Miamnus-M1-Nano-v1.1
Miamnus-M1-Nano-v1.1 is a compact, high-efficiency hybrid causal language model engineered by Miamnus AI. Built on a 3:1 recurring hybrid architecture pairing KDA linear attention with MLA attention layers across 24 transformer blocks.
This release package is completely independent, self-contained, and written in pure PyTorch. It contains ONLY the code required for this model, with zero external Triton or CUDA kernel compilation dependencies. It runs out of the box on Apple Silicon (MPS), CPU, and NVIDIA CUDA.
Specifications
| Parameter | Value |
|---|---|
| Model Name | Miamnus-M1-Nano-v1.1 |
| Version | v1.1 |
| Parameters | 22.4M (22,396,872 weights) |
| Architecture | 3:1 Hybrid (KDA + MLA) |
| Layers | 24 (18 KDA + 6 MLA) |
| Hidden Dim ($d_{\text{model}}$) | 256 |
| Attention Heads | 4 Query Heads, 2 Key/Value Heads |
| Head Dim | 64 |
| FFN Intermediate Dim | 704 (SwiGLU) |
| Context Length | 1,024 tokens |
| Vocabulary Size | 4,096 tokens (Byte-Level BPE) |
| Pretraining Dataset | MiamnusAI/CuratedCorpus-5B |
| License | Apache 2.0 |
Quickstart (Hugging Face Transformers)
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "MiamnusAI/Miamnus-M1-Nano-v1.1"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_id, trust_remote_code=True)
inputs = tokenizer("Once upon a time in a sunny valley,", return_tensors="pt")
output = model.generate(inputs["input_ids"], max_new_tokens=60, temperature=0.7)
print(tokenizer.decode(output[0]))
Local Standalone Inference
python generate.py --prompt "Once upon a time" --max_tokens 80 --temperature 0.75
Citation
@article{miamnus2026m1nano,
title={Miamnus M1 Nano: Efficient 3:1 Hybrid Linear Attention Language Models},
author={Miamnus AI Research Team},
journal={Miamnus Technical Report Series},
year={2026},
url={https://huggingface.co/MiamnusAI/Miamnus-M1-Nano-v1.1}
}