LiteGPT-Llama-V2-Instruct

The chat version of LiteGPT V2: a 201M-parameter model built from scratch, with every step on record.

LiteGPT V2 Base after three epochs of chat fine-tuning on curated conversations. It passed its registered quality gate, and follows instructions far better than LiteGPT V1-Instruct. Results, charts and run records: github.com/kmkrofficial/LiteGPT.

Highlights

• Follows instructions 55% better than V1-Instruct on IFEval (0.129 against 0.083), clearing its registered gate by 2.56 standard errors.
• Writes some working code: 7 of 164 HumanEval problems solved and 4.2% of MBPP, where V1-Instruct solved none.
• Small: 201M parameters and a 4,096-token context. It runs comfortably on a laptop.
• Loads in Transformers with no remote code, with its chat template included.

Results

Every benchmark, including comparisons with other chat models: results.

How it compares

LiteGPT V2 Instruct doesn't reach the top score among these chat models. What it brings is a chat model built entirely from scratch, small enough for a laptop, and open down to every run record.

Higher is better.

Model

Use

With Transformers:

from transformers import AutoModelForCausalLM, AutoTokenizer

name = "kmkrworks/LiteGPT-Llama-V2-Instruct"
tokenizer = AutoTokenizer.from_pretrained(name)
model = AutoModelForCausalLM.from_pretrained(name)
inputs = tokenizer.apply_chat_template([{"role": "user", "content": "What is the capital of France?"}], add_generation_prompt=True, return_tensors="pt", return_dict=True)
output = model.generate(**inputs, max_new_tokens=48)
print(tokenizer.decode(output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True))

With LiteGPT's own engine, from a clone of the repository:

uv sync --locked
uv run litegpt chat --model LiteGPT-Llama-V2-Instruct

Benchmarks were scored through LiteGPT's engine.

Intended use: research and learning. It has no safety tuning, so add your own safeguards before putting it in front of people.

Training

3 epochs, 14,000 steps, assistant-only loss, peak learning rate 6e-4. Held-out loss fell every epoch: 1.1323, 1.0518, 1.0238. Details: training.md.

Training data

Everything in the Base card, then these chat conversations (385,677 in all, 497M tokens). Attributions: the NOTICE file.

Licence

Apache-2.0 for the weights, MIT for the code, with the NOTICE file.

Built with Llama. Built with Qwen. Llama 3.1 is licensed under the Llama 3.1 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved. Use of this model should follow the Llama Acceptable Use Policy.