T3Q-LLM-MG-v1.0
Model Developers Chihoon Lee(chihoonlee10), T3Q
Python code
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, TextStreamer
MODEL_DIR = "chihoonlee10/T3Q-LLM-MG-v1.0"
model = AutoModelForCausalLM.from_pretrained(MODEL_DIR, torch_dtype=torch.float16).to("cuda")
tokenizer = AutoTokenizer.from_pretrained(MODEL_DIR)
streamer = TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)
s = "한국의 수도는 어디?"
conversation = [{'role': 'user', 'content': s}]
inputs = tokenizer.apply_chat_template(
conversation,
tokenize=True,
add_generation_prompt=True,
return_tensors='pt').to("cuda")
_ = model.generate(inputs, streamer=streamer, max_new_tokens=1024)
hf (pretrained=chihoonlee10/T3Q-LLM-MG-v1.0), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
T3Q-LLM/T3Q-LLM-sft1.0-dpo1.0