Model · pythia-chat-base-7b-v0-16

Pythia-Chat-Base-7B-v0.16

Developer: Together Computer (with LAION and Ontocord.ai)
Availability: available · checked 2026-09-24 · source

Raw record: /data/models/pythia-chat-base-7b-v0-16.json

Fields

id

pythia-chat-base-7b-v0-16

identifiers

huggingface

togethercomputer/Pythia-Chat-Base-7B-v0.16

developer

Together Computer (with LAION and Ontocord.ai)

release_date

2023-03-22partial · source

note:  HF repo creation date; the card gives a version (v0.16) but no release date.

weights_status

open

availability

available · checked 2026-09-24 · source

license

apache-2.0recorded · source

note:  Card: 'License: Apache 2.0'. Uploader is the developer.

architecture

family

decoder_only

note

family read from config 'architectures': ['GPTNeoXForCausalLM'].

n_layers

32recorded · source

hidden_size

4096recorded · source

n_heads

32recorded · source

vocab_size

50432recorded · source

context_length

2048recorded · source

positional_encoding

rotary (RoPE), partial: 25% of dimsrecorded · source

note:  Propagated from pythia-6-9b-deduped, the base per the OpenChatKit README; config matches (32 layers, hidden 4096).

training_data

OIG-43M: 'a collection of 43 million high-quality instructions', curated by Together with LAION and Ontocord.ai (OpenChatKit models 'were trained on the OIG-43M training dataset'); plus 'a small amount of feedback data' (unspecified) in a further fine-tuning step.recorded · source

note:  The feedback data is described only as 'a small amount'; its source is not_recorded.

techniques

primary_sources

https://huggingface.co/togethercomputer/Pythia-Chat-Base-7B-v0.16

https://huggingface.co/togethercomputer/Pythia-Chat-Base-7B-v0.16/raw/97aa918c383820e1a69f042801091d7deb996c20/config.json

https://huggingface.co/togethercomputer/Pythia-Chat-Base-7B-v0.16/blob/97aa918c383820e1a69f042801091d7deb996c20/README.md

https://github.com/togethercomputer/OpenChatKit/blob/main/README.md

record_history

date:2026-09-24 · change:ingested as candidate from HF (togethercomputer/Pythia-Chat-Base-7B-v0.16@97aa918c383820e1a69f042801091d7deb996c20) · by:ingest_hf.py ·

date:2026-09-24 · change:preparer: developer, license, positional encoding (propagated from pythia-6-9b-deduped), training data, 2 edges prepared; sources: card, OpenChatKit README · by:claude (preparer, Sonnet 5) ·

date:2026-09-24 · change:reviewed and promoted from staging (2 edge(s) accepted) · by:Wilson Pruitt ·

Parents

Weights descend

• fine_tuned_from → pythia-6.9b-deduped declared source OpenChatKit README: 'Pythia-Chat-Base-7B is a 7B-parameter fine-tuned variant of Pythia-6.9B-deduped from Eleuther AI'; the HF card says 'fine-tuned from EleutherAI's Pythia 7B' (no variant named). The README is what pins the deduped 6.9B.

Training data

• trained_on → Open Instruction Generalist (OIG) declared source README: 'The chat model was trained on the OIG dataset built by LAION, Together, and Ontocord.ai'; card: 43M-instruction collection (OIG-43M). The `oig` record covers the LAION OIG collection ('currently at 44M'); the 43M snapshot vs the current collection is not distinguished.

Children

No edges recorded.

Read in

No station on the reading path has touched this record yet.