Model · zephyr-7b-beta
Zephyr 7B β
Developer: Hugging Face (H4 team)
Availability: available · checked 2026-09-24 · source
Raw record: /data/models/zephyr-7b-beta.json
Fields
- id
- zephyr-7b-beta
- identifiers
- huggingface
- HuggingFaceH4/zephyr-7b-beta
- developer
- Hugging Face (H4 team)
- release_date
- 2023-10-25partial · sourcenote: Date of the paper's arXiv v1; the weights' own publication date was not confirmed.
- weights_status
- open
- availability
- available · checked 2026-09-24 · source
- license
- MITrecorded · source
- architecture
- family
- decoder_only
- n_layers
- 32recorded · source
- hidden_size
- 4096recorded · source
- n_heads
- 32recorded · source
- vocab_size
- 32000recorded · source
- positional_encoding
- rotary (RoPE)recorded · source
- context_length
- 32768recorded · sourcenote: Config max_position_embeddings, inherited from Mistral 7B's config; the Mistral paper gives 8192 as the design context length.
- n_kv_heads
- 8recorded · source
- sliding_window
- 4096recorded · source
- training_data
- Filtered UltraChat (synthetic dialogues 'generated by ChatGPT') for distilled SFT; UltraFeedback (64k prompts with model completions 'ranked by GPT-4') for distilled DPO.recorded · source
- techniques
- instruction-tuningdpo
- primary_sources
- record_history
- date:2026-09-24 · change:created from primary sources (Phase 1 seed, batch 1) · by:wilson-pruitt + claude ·date:2026-09-24 · change:note on context_length provenance · by:wilson-pruitt + claude ·date:2026-09-24 · change:availability checked and recorded · by:wilson-pruitt + claude ·date:2026-09-24 · change:added fine_tuned_from edge to mistral-7b-sft-beta (SFT stage), per that card · by:claude (Sonnet 5), tranche-2 follow-up ·
Parents
Weights descend
- fine_tuned_from → Mistral 7B (v0.1) declared source Developer's own model card: 'fine-tuned version of mistralai/Mistral-7B-v0.1'.
- fine_tuned_from → Mistral 7B SFT β declared source Developer's card for the SFT model: 'It is the SFT model that was used to train Zephyr-7B-β with Direct Preference Optimization.' The Zephyr card's own 'fine-tuned version of Mistral-7B-v0.1' edge stays: this edge records the intermediate SFT stage, so Zephyr's path is Mistral-7B-v0.1 -> mistral-7b-sft-beta -> zephyr-7b-beta.
Training data
- trained_on → UltraChat declared source Distilled SFT step; card: 'filtered and preprocessed' UltraChat.
- trained_on → UltraFeedback declared source DPO step on GPT-4's rankings of model completions.
Children
No edges recorded.
Read in
No station on the reading path has touched this record yet.