Model · bloom-7b1
BLOOM 7B1
Developer: BigScience
Availability: available · checked 2026-09-24 · source
Raw record: /data/models/bloom-7b1.json
Fields
- id
- bloom-7b1
- identifiers
- huggingface
- bigscience/bloom-7b1
- developer
- BigScience
- release_date
- 2022-07-11partial · sourcenote: Card: 'Release Date Estimate: Monday, 11.July.2022'.
- weights_status
- open
- availability
- available · checked 2026-09-24 · source
- license
- BigScience RAIL License v1.0 (open access with use restrictions)recorded · source
- architecture
- family
- decoder_only
- note
- Card: 'Modified from Megatron-LM GPT2' (the Megatron codebase, not OpenAI's weights). Paper: ALiBi instead of position embeddings, plus a LayerNorm on the embedding layer. GeLU activation.
- n_layers
- 30recorded · source
- hidden_size
- 4096recorded · source
- n_heads
- 32recorded · source
- vocab_size
- 250680recorded · sourcenote: Config vocab_size is 250880 (padded).
- positional_encoding
- ALiBi (attention bias by distance; no position embeddings)recorded · source
- context_length
- 2048recorded · source
- training_data
- ROOTS corpus: 46 natural and 13 programming languages; 1.5TB of text, 350B unique tokens.recorded · source
- techniques
- transformer-decoderalibi
- primary_sources
- record_history
- date:2026-09-24 · change:created from primary sources (Phase 1 seed, batch 2) · by:wilson-pruitt + claude ·date:2026-09-24 · change:availability checked and recorded · by:wilson-pruitt + claude ·
Parents
Training data
- trained_on → ROOTS corpus declared source
Children
Weights descend
- ← fine_tuned_from bloomz-7b1 declared source Card: 'Architecture: Same as bloom-7b1'; fine-tuned 1000 steps / 4.19B tokens on xP3 (crosslingual task mixture, 46 languages).
Read in
No station on the reading path has touched this record yet.