How to use it
# pip install transformers accelerate
from transformers import AutoTokenizer
import transformers
import torch
model = "kingabzpro/llama-2-7b-chat-guanaco"
prompt = "Who is Leonardo Da Vinci?"
tokenizer = AutoTokenizer.from_pretrained(model)
pipeline = transformers.pipeline(
"text-generation",
model=model,
torch_dtype=torch.float16,
device_map="auto",
)
sequences = pipeline(
f'<s>[INST] {prompt} [/INST]',
do_sample=True,
top_k=10,
num_return_sequences=1,
eos_token_id=tokenizer.eos_token_id,
max_length=200,
)
for seq in sequences:
print(f"Result: {seq['generated_text']}")
Output
Result: <s>[INST] Who is Leonardo Da Vinci? [/INST] Leonardo da Vinci (1452-1519) was a famous Italian polymath, artist, and inventor. (polymath: a person who knows a lot about many different subjects). He is considered one of the greatest painters of all time, and his paintings have had a big influence on many people who have lived after him.
As an artist, Leonardo da Vinci painted beautiful paintings using his skills and knowledge. Some of his most famous paintings are:
* The Mona Lisa: a painting of a woman who is smiling mysteriously, and is considered one of the greatest works of art of the Renaissance.
* The Last Supper: a mural painting depicting the last meal Jesus shared with his twelve disciples before he died.
* The Virgin and Child with St. Anne and St. John the