Model · guanaco-7b
guanaco-7b
Developer: Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer (QLoRA authors; repo uploaded by timdettmers)
Availability: available · checked 2026-09-24 · source
Raw record: /data/models/guanaco-7b.json
Fields
- id
- guanaco-7b
- identifiers
- huggingface
- timdettmers/guanaco-7b
- developer
- Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, Luke Zettlemoyer (QLoRA authors; repo uploaded by timdettmers)
- release_date
- 2023-05-22partial · sourcenote: HF repo creation date (2023-05-22). The QLoRA paper (arXiv 2305.14314) is dated 2023-05-23.
- weights_status
- open
- availability
- available · checked 2026-09-24 · source
- license
- Apache-2.0 (adapter weights only); use also requires the LLaMA licenserecorded · sourcenote: Card: 'Guanaco adapter weights are available under Apache 2 license. Note the use of the Guanaco adapter weights, requires access to the LLaMA model weighs. Guanaco is based on LLaMA and therefore should be used according to the LLaMA license.'
- architecture
- family
- decoder_only
- n_layers
- 32recorded · source
- hidden_size
- 4096recorded · source
- n_heads
- 32recorded · source
- vocab_size
- 32000recorded · source
- positional_encoding
- rotary (RoPE)recorded · source
- context_length
- 2048recorded · source
- note
- Propagated from llama-7b: the card says Guanaco is 'obtained through 4-bit QLoRA tuning of LLaMA base models' and its usage example loads huggyllama/llama-7b with this adapter. The adapter changes no architecture field.
- training_data
- OASST1 (OpenAssistant Conversations), per the card: 'obtained through 4-bit QLoRA tuning of LLaMA base models on the OASST1 dataset'.recorded · source
- techniques
- primary_sources
- record_history
- date:2026-09-24 · change:ingested as candidate from HF (timdettmers/guanaco-7b@cad9de16fb306d5bb1feb901333fe0aa7bd700d8) · by:ingest_hf.py ·date:2026-09-24 · change:preparer: developer, license (Apache-2.0 adapter, card), architecture (propagated from llama-7b), training data (OASST1), 2 edges; sources: card, adapter_config.json, arXiv 2305.14314 · by:claude (preparer, Sonnet 5) ·date:2026-09-24 · change:reviewed and promoted from staging (2 edge(s) accepted) · by:Wilson Pruitt ·
Parents
Weights descend
- adapter_on → LLaMA 7B declared source Card: 'open-source finetuned chatbots obtained through 4-bit QLoRA tuning of LLaMA base models'; 'Lightweight checkpoints which only contain adapter weights'; usage loads huggyllama/llama-7b (a third-party re-upload of LLaMA-7B) with adapters timdettmers/guanaco-7b. adapter_config.json: LoRA r 64, targets q,k,v,o,gate,up,down proj; its base path is a local path (/gscratch/zlab/llama/7B). QLoRA trains the adapter against a frozen 4-bit quantized base; the adapter itself is not a quantization of LLaMA, so not quantized_from.
Training data
- trained_on → OpenAssistant Conversations (OASST1) declared source Card: 'on the OASST1 dataset'; paper abstract: Guanaco outperforms previous open models on the Vicuna benchmark. The card says evaluation used March 2023 ChatGPT/Bard outputs; that is evaluation, not training, so no influence edge.
Children
No edges recorded.
Read in
No station on the reading path has touched this record yet.