Patched LLama 3.2 8B from LLaMA 3.2 11B Model
Here’s the complete, refined code for patching the weights:
# Import required libraries
from transformers import AutoProcessor, AutoTokenizer, AutoModelForImageTextToText, AutoModelForCausalLM
# Load the 11B Vision-Instruct model
processor = AutoProcessor.from_pretrained("meta-llama/Llama-3.2-11B-Vision-Instruct")
model = AutoModelForImageTextToText.from_pretrained("meta-llama/Llama-3.2-11B-Vision-Instruct")
# Load the 8B text-only model
s_tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
s_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")
# Prepare input text for testing
input_text = "Write me a poem about Machine Learning."
input_ids = s_tokenizer(input_text, return_tensors="pt")
# Test the original 8B model
outputs = s_model.generate(**input_ids, do_sample=False, max_new_tokens=10)
print("8B Model Output:", s_tokenizer.decode(outputs[0]))
# Patch weights from the 11B model into the 8B model
model_weight = model.state_dict()
s_model_dict = s_model.state_dict()
skip_layer = 0 # Track skipped layers
for key in s_model_dict.keys():
if "layers." in key:
layer_idx = int(key.split("layers.")[1].split(".")[0]) # Extract layer index
try:
s_model_dict[key] = model_weight[
"language_model." + key.replace(f"layers.{layer_idx}.", f"layers.{layer_idx + skip_layer}.")
]
except KeyError:
skip_layer += 1
s_model_dict[key] = model_weight[
"language_model." + key.replace(f"layers.{layer_idx}.", f"layers.{layer_idx + skip_layer}.")
]
else:
s_model_dict[key] = model_weight["language_model." + key]
# Test the patched 8B model
outputs = s_model.generate(**input_ids, do_sample=False, max_new_tokens=10)
print("Patched 8B Model Output:", s_tokenizer.decode(outputs[0]))
# Test the original 11B model
outputs = model.generate(**input_ids, do_sample=False, max_new_tokens=10)
print("11B Model Output:", s_tokenizer.decode(outputs[0]))
Prompt: "Write me a poem about Machine Learning."
Outputs:
8B Model Output (Before Patching):
<|begin_of_text|>Write me a poem about Machine Learning.
Artificial minds, born from code,
Learning
Patched 8B Model Output:
<|begin_of_text|>Write me a poem about Machine Learning.
In silicon halls, where data reigns
11B Model Output:
<|begin_of_text|>Write me a poem about Machine Learning.
In silicon halls, where data reigns
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
Use the code below to get started with the model.
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
BibTeX:
[More Information Needed]
APA:
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
[More Information Needed]
model-00004-of-00007.safetensors 5.0 GBmodel-00006-of-00007.safetensors 5.0 GBmodel-00003-of-00007.safetensors 5.0 GBmodel-00001-of-00007.safetensors 4.9 GBmodel-00005-of-00007.safetensors 4.8 GBmodel-00002-of-00007.safetensors 4.8 GBmodel-00007-of-00007.safetensors 2.6 GB有文件?提交种子。
提交种子