A Llama Chat Model of 160M Parameters
Recommended Prompt Format
<|im_start|>system
{system_message}<|im_end|>
<|im_start|>user
{user_message}<|im_end|>
<|im_start|>assistant
Recommended Inference Parameters
penalty_alpha: 0.5
top_k: 4
repetition_penalty: 1.01
Usage Example
from transformers import pipeline
generate = pipeline("text-generation", "Felladrin/Llama-160M-Chat-v1")
messages = [
{
"role": "system",
"content": "You are a helpful assistant who answers user's questions with details and curiosity.",
},
{
"role": "user",
"content": "What are some potential applications for quantum computing?",
},
]
prompt = generate.tokenizer.apply_chat_template(
messages, tokenize=False, add_generation_prompt=True
)
output = generate(
prompt,
max_new_tokens=1024,
penalty_alpha=0.5,
top_k=4,
repetition_penalty=1.01,
)
print(output[0]["generated_text"])
Old Open LLM Leaderboard Evaluation Results
Detailed results can be found here
