Model · starcoder

StarCoder

Developer: BigCode project (open scientific collaboration led jointly by Hugging Face and ServiceNow Research)
Availability: gated · checked 2026-09-24 · source
HF gate: auto.

Raw record: /data/models/starcoder.json

Fields

id

starcoder

identifiers

huggingface

bigcode/starcoder

developer

BigCode project (open scientific collaboration led jointly by Hugging Face and ServiceNow Research)

release_date

2023-05-04recorded · source

note:  Release post published May 4, 2023 announces StarCoder and StarCoderBase together. HF repo created 2023-04-24 (access-gated, likely pre-release).

weights_status

open

availability

gated · checked 2026-09-24 · source

note:  HF gate: auto.

license

BigCode OpenRAIL-M v1recorded · source

note:  Card metadata `license: bigcode-openrail-m`; paper: 'We release StarCoder under an OpenRAIL-M license agreement'.

architecture

family

decoder_only

note

Config.json is gated (not retrievable). Values from the StarCoder paper Table 11 and section 5.2; the paper states multi-query attention (a single shared key/value head by definition; the paper gives no count) and 'learned absolute positional embeddings'. Fine-tuning (StarCoder from StarCoderBase) changes no architecture field.

n_layers

40recorded · source

note:  Paper Table 11 'Num. of hidden layers'.

hidden_size

6144recorded · source

note:  Paper Table 11.

n_heads

48recorded · source

note:  Paper Table 11 'Num. of attention heads'.

vocab_size

49152recorded · source

note:  Paper sec. 5.1: 'vocabulary size of 49,152 tokens—including the sentinel tokens'. WizardCoder's HF config shows 49,153 (one added token).

positional_encoding

learned absoluterecorded · source

note:  Paper sec. 5.2: 'decoder-only Transformer with Multi-Query-Attention, and learned absolute positional embeddings'.

context_length

8192recorded · source

note:  Paper Table 11 'Max. position embeddings' 8192; card: 'a context window of 8192 tokens'.

training_data

StarCoderBase (1T tokens, The Stack v1.2), then fine-tuned for 2 epochs on the Python subset of the training data: 35B Python tokens.recorded · source

note:  Paper: 'We fine-tuned StarCoderBase on another 35B Python tokens, resulting in the creation of StarCoder'; sec. 5.6 'Starting from StarCoderBase, we fine-tuned a Python variant of the model for 2 epochs on the Python subset of the training data ... 8,500 steps'.

techniques

primary_sources

https://huggingface.co/bigcode/starcoder

record_history

date:2026-09-24 · change:ingested as candidate from HF (bigcode/starcoder@827cd7721231872c152e3faef5cae1b22964ed58) · by:ingest_hf.py ·

date:2026-09-24 · change:preparer: developer, release date (blog), license, architecture (paper), training data, 3 edge entries (fine_tuned_from starcoderbase; trained_on starcoderdata; the-stack-dedup rejected); sources: card, arXiv 2305.06161, release post, dataset API · by:claude (preparer, Sonnet 5) ·

date:2026-09-24 · change:reviewed and promoted from staging (2 edge(s) accepted) · by:Wilson Pruitt ·

Parents

Weights descend

• fine_tuned_from → StarCoderBase declared source Paper abstract: 'We fine-tuned StarCoderBase on 35B Python tokens, resulting in the creation of StarCoder.' Release post: 'We fine-tuned StarCoderBase model for 35B Python tokens, resulting in a new model that we call StarCoder.' Developers' own words.

Training data

• trained_on → StarCoderData declared source Dataset card (public API description): 'the dataset used for training StarCoder and StarCoderBase'. StarCoder's own fine-tuning used the Python subset of that training data (paper sec. 5.6).

Children

Weights descend

• ← fine_tuned_from WizardCoder-15B-V1.0 declared source Card fine-tuning section: 'We fine-tune StarCoder-15B with the following hyperparameters' and the reproduce command `--model_name_or_path "bigcode/starcoder"`; paper: 'we fine-tune StarCoder'.

Read in

No station on the reading path has touched this record yet.