中文Sign up

inference-optimization/Inkling-0.6B-A0.6B

🤗 Hugging Face sourceimage-text-to-textmit644M params600M activated2.6 GBsafetensors✓ 13 checksumsupdated today
Submit in one command

Run it next to your model folder. It makes the torrent, checks your files against Hugging Face, and submits it. You just start seeding and paste your key from your account. It only reads your files and never changes them. Read the script first if you like.

curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo inference-optimization/Inkling-0.6B-A0.6B ./model-folder
Needs a seeder →

Inkling-0.6B-A0.6B

This is a tiny version of thinkingmachines/Inkling created for testing and development.

Model Details

  • Base Model: thinkingmachines/Inkling
  • Architecture: inkling_mm_model (InklingForConditionalGeneration)
  • Total Parameters: 0.644B
  • Activated Parameters: 0.602B

Configuration Changes

The following parameters were reduced from the original model:

Parameter Original Tiny
text_config.num_hidden_layers 66 12
text_config.hidden_size 6144 1024
text_config.intermediate_size 24576 4096
text_config.num_attention_heads 64 8
text_config.num_key_value_heads 8 2
text_config.swa_num_attention_heads 64 8
text_config.swa_num_key_value_heads 16 4
text_config.n_routed_experts 256 8
text_config.num_experts_per_tok 6 4
text_config.moe_intermediate_size 3072 512
text_config.num_mtp_layers 8 1
vision_config.n_layers 4 1
vision_config.hidden_size 1024 256
vision_config.decoder_dmodel 6144 1024
audio_config.decoder_dmodel 6144 1024

Layer type patterns are preserved: 2 repetitions of [5× hybrid_sliding + 1× hybrid], with the first 2 MLP layers as dense and the rest as sparse (MoE).

Checkpoint Structure

Single safetensors file (model.safetensors). Key naming matches the original checkpoint format (model.llm.*, model.audio.*, model.visual.*).

Usage

from transformers.models.inkling import InklingForConditionalGeneration
from transformers import AutoTokenizer

model = InklingForConditionalGeneration.from_pretrained("inference-optimization/Inkling-0.6B-A0.6B", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("inference-optimization/Inkling-0.6B-A0.6B")

input_ids = tokenizer("According to all known laws", return_tensors="pt").input_ids.to(model.device)
output = model.generate(input_ids, max_new_tokens=20)
print(tokenizer.decode(output[0]))

Creation Process

This model was created using the llm-compressor create-tiny-model claude skill.

  1. Config inspected via inspect_config.py
  2. Tiny model created via modified save_tiny_model.py — all-zero params fixed post init_weights
  3. Fine-tuned on copypasta dataset; reached perplexity 1.45 (target: ≤3.0) at lr=5e-4
  4. Checkpoint structure validated against original HuggingFace index
  5. Inference validated via validate_tiny_model.py

Notes

  • The embed_tokens weights require explicit re-initialization after init_weights() (they initialize to zero in this architecture). The save script applies a fixup: any all-zero, non-finite, or extreme-valued parameter is re-initialized with kaiming_uniform / normal / ones as appropriate.
  • MTP (Multi-Token Prediction) layers present in the original checkpoint (model.mtp.*) are not included, as InklingForConditionalGeneration does not expose them through its standard interface.
  • Validation output: Success: 1.4451 <= 10.0