TinyLlama-1.1B-Chat — ONNX (FP16)
ONNX export of TinyLlama-1.1B-Chat-v1.0 (1.1B parameters, FP16 weights) with KV cache support for efficient autoregressive generation.
Converted for use with inference4j, an inference-only AI library for Java.
Original Source
- Repository: TinyLlama/TinyLlama-1.1B-Chat-v1.0
- License: Apache 2.0
Usage with inference4j
try (var gen = OnnxTextGenerator.tinyLlama().build()) {
GenerationResult result = gen.generate("What is Java?");
System.out.println(result.text());
}
Model Details
| Property | Value |
|---|---|
| Architecture | LlamaForCausalLM (1.1B parameters, 22 layers, 2048 hidden, 32 heads, 4 KV heads) |
| Task | Text generation (instruction-tuned, Zephyr chat template) |
| Precision | FP16 |
| Context length | 2048 tokens |
| Vocabulary | 32,000 tokens (SentencePiece BPE) |
| Chat template | Zephyr (`< |
| Original framework | PyTorch (transformers) |
| Export method | Hugging Face Optimum (with KV cache, FP16) |
License
This model is licensed under the Apache License 2.0. Original model by TinyLlama.