The model card, file list, and checksums are here. Have the files and permission to share them? Submit a torrent so others can download from peers.
Getting the source files requires Hugging Face approval.[!TIP]
Exclusive Model Access
This model repository is available exclusively to my Pro Patreon Members.
- Become my Pro Patreon Member:
2. Navigate to this Patreon model post containing a secret link to a form. 3. Input your Hugging Face username into the form to unlock access instantly to this repo and download model weights.
Direct requests submitted through the native Hugging Face form above are NOT approved manually. Access is granted ONLY through an active Patreon subscription.
The following website pages were generated by this model running locally in Unsloth Desktop at Q5_K_M GGUF quantization.
Prompt: "Create a beautiful animated one-file landing page website with HTML, CSS and JavaScript selling a book about the history of AI. No photos."
No web search, no tool-calls and no additional system prompts were used during this test. Run with multiple different seeds to mitigate luck and cherry-picking.
Landing Page Demo 1: Seed 1234Landing Page Demo 1: Seed 777Prompt: "Create a beautiful one-file website landing page with 3D-animations for a fake linux distro called CrabOS that uses a blazingly fast linux kernel fully rewritten in Rust, a desktop environment written in OCaml and a package manager written in Haskell. Don't use photos."
MiMo-HERO-UIUX-Designer-9B is a surgical merge of two Qwen 3.5 9B fine-tunes:
By grafting HERO's high-efficiency decision vectors directly into MiMo's reasoning core, the model cures MiMo's tendency to go into thinking loops unlocking its real potential.
[!NOTE] The result is a that excels at frontend/UI design and website creation while retaining tool-call capabilities for agentic tasks.
NOTE: Unlike my previous iteration (pragmaticcs/MiMo-HERO-Frontier-Agentic-Coder-9B) this merge has a higher share/strength of HERO, which resulted in a model better at frontend design.
| Parameter | Specification |
|---|---|
| Total Base Parameters | 8.9B (Vision tower preserved; Multi-Token Prediction excised) |
| Foundation Architecture | Hybrid Recurrent-Attention Causal LM (Qwen3_5ForCausalLM) |
| Recurrent Topology | Gated DeltaNet linear-attention layers interleaved with standard attention |
| Attention Configuration | Grouped-Query Attention (GQA) with YaRN RoPE scaling |
| Context Window | 262,144 tokens native |
| Native Precision | bfloat16 |
| Primary Capabilities | Agentic coding and software engineering |
Direct linear blending, spherical interpolation (SLERP), and uniform DARE perturbations degrade hybrid DeltaNet networks. Linear attention recurrence relies on strictly contractive transition states; arbitrary parameter shifts introduce runaway eigenvalues that destroy long-range memory and break special control tokens.
To prevent degradation, the parameter integration is governed by non-interfering geometric constraints.
Let W0 denote the underlying base model parameters (Qwen3.5-9B), WMiMo the primary anchor, and WHERO the donor. The task vectors are defined as:
$$ \Delta_{\text{MiMo}} = W_{\text{MiMo}} - W_{0} $$
$$ \Delta_{\text{HERO}} = W_{\text{HERO}} - W_{0} $$
To guarantee that HERO does not overwrite the fundamental representations discovered during MiMo's distillation, HERO's update is projected orthogonally away from the dominant singular directions of MiMo's task vector.
For any weight matrix with singular value decomposition:
$$ \Delta_{\text{MiMo}} \approx U_k \Sigma_k V_k^T $$
The overlapping projection tensor is computed across rank k = 32 and tapered according to spectral energy:
$$ \Omega = \left(\frac{\operatorname{diag}(\Sigma_k)}{\tau}\right)^2 $$
$$ \Delta_{\text{HERO}}^{\perp} = \Delta_{\text{HERO}} - U_k \left( \Omega \odot (U_k^T \Delta_{\text{HERO}} V_k) \right) V_k^T $$
Components of the agentic donor that directly compete with MiMo's primary singular subspace are eliminated, allowing new procedural capabilities to occupy unutilized parameter dimensions.
When fine-tuning objectives pull individual weights in opposite directions, blind averaging causes catastrophic cancellation and syntax corruption. For coordinates where updates conflict:
$$ \operatorname{sign}\left(\Delta_{\text{MiMo}}^{(i, j)}\right) \neq \operatorname{sign}\left(\Delta_{\text{HERO}}^{\perp (i, j)}\right) $$
An antagonistic penalty coefficient $\Phi$ is evaluated against relative weight salience:
$$ \Phi_{(i, j)} = 1.0 - 0.50 \cdot \left( \frac{|\Delta_{\text{MiMo}}^{(i, j)}|}{|\Delta_{\text{MiMo}}^{(i, j)}| + |\Delta_{\text{HERO}}^{\perp (i, j)}| + \epsilon} \right) $$
$$ \widetilde{\Delta}{\text{HERO}}^{(i, j)} = \Phi{(i, j)} \cdot \Delta_{\text{HERO}}^{\perp (i, j)} $$
Antagonistic donor noise is suppressed, ensuring that settled code generation rules are never inverted.
To eliminate low-magnitude parameter drift, the bottom 4% energy of the sanitized update is zeroed out. The surviving parameters are rescaled by the inverse square root of the retained energy ratio to preserve exact variance:
$$ \widehat{\Delta}{\text{HERO}} = \operatorname{Mask}\left(\widetilde{\Delta}{\text{HERO}}, , p=0.96\right) \cdot \sqrt{\frac{1.0}{0.96}} $$
Layer-wise integration follows a smooth transition profile across the network's depth index l over total layers L:
$$ g(l) = \begin{cases} 0.50 + 0.50 \cdot S\left(\frac{l}{0.20 L}\right) & 0 \le l < 0.20 L \ 1.00 & 0.20 L \le l \le 0.80 L \ 1.00 - 0.30 \cdot S\left(\frac{l - 0.80 L}{0.20 L}\right) & 0.80 L < l \le L \end{cases} $$
where S(t) represents the smooth cubic step function:
$$ S(t) = 3t^2 - 2t^3 $$
To prevent distribution collapse, the effective step size is bound simultaneously by Frobenius norm ratio $\rho$ and spectral radius $\sigma_{\max}$:
$$ s = \min\left( s_{\max}, ; \rho \cdot \frac{|\Delta_{\text{MiMo}}|F}{|\widehat{\Delta}{\text{HERO}}|F}, ; \kappa \cdot \frac{\sigma{\max}(\Delta_{\text{MiMo}})}{\sigma_{\max}(\widehat{\Delta}_{\text{HERO}})} \right) $$
Final parameter state:
$$ W_{\text{final}} = W_{\text{MiMo}} + \big(s \cdot g(l)\big) \cdot \widehat{\Delta}_{\text{HERO}} $$
| Subsystem | Target Tensors | Policy | Objective |
|---|---|---|---|
| Embeddings & Heads | embed_tokens, lm_head |
Strict Anchor Lock | Prevents vocabulary misalignment and maintains exact output logit distribution. |
| Vision Subsystem | visual.* |
Preserved Intact | Keeps multimodal parsing and visual document understanding fully functional. |
| Recurrent Core | linear_attn.* |
Conservative Gain (0.80) | Protects Gated DeltaNet state-space stability across 128k context steps. |
| Attention Routing | self_attn.q_proj, k_proj |
Attenuated Gain (0.85) | Prevents attention-head defocusing during long-sequence tool tracking. |
| Value & Projections | self_attn.v_proj, o_proj |
Elevated Gain (0.95) | Facilitates smooth transfer of semantic agentic representations. |
| MLP Blocks | mlp.down_proj, up_proj |
Full Strength (1.00) | Maximizes transfer of practical code implementation heuristics. |
| Outlier Coordinates | |W| > 8 · rms(W) | Coordinate Shield | Protects critical emergent outlier features from perturbation. |
This model integrates the Improved Chat Template for Qwen 3.x directly into its tokenizer configuration and Jinja runtime environment.
<thought>...</thought> delimiters prior to tool calling or terminal emission.For repository-level software engineering, bug localization, and automated tool use:
| Parameter | Recommended Value | Description |
|---|---|---|
| Temperature | 0.6 |
Balances strict syntactic validity with algorithmic path exploration. |
| Top-P | 0.95 |
Eliminates low-probability token tails while preserving alternative logic paths. |
| Top-K | 20 |
Restricts token candidate pools to prevent architectural syntax drift. |
| Min-P | 0.0 (Off) |
Disabled in favor of explicit Top-K / Top-P governance. |
| Repetition Penalty | 1.0 (Off) |
Disabled to prevent syntax degradation in repetitive code patterns (indentation, braces). |
| Presence Penalty | 0.0 |
Prevents naming mutations across long-context symbol resolution. |
Thanks to the excision of multi-token prediction layers, the model deploys natively without custom engine flags:
vllm serve pragmaticcs/MiMo-HERO-UIUX-Designer-9B \
--dtype bfloat16 \
--max-model-len 32768 \
--gpu-memory-utilization 0.90 \
--enable-auto-tool-choice \
--tool-call-parser hermes \
--enable-reasoning \
--reasoning-parser qwen3
python -m sglang.launch_server \
--model-path pragmaticcs/MiMo-HERO-UIUX-Designer-9B \
--port 30000 \
--host 0.0.0.0 \
--context-length 32768
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "pragmaticcs/MiMo-HERO-UIUX-Designer-9B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id, torch_dtype=torch.bfloat16, device_map="auto"
)
conversation = [
{
"role": "user",
"content": "Create a beautiful one-file landing page for a travel agency using HTML, CSS and JavaScript.",
},
]
inputs = tokenizer.apply_chat_template(
conversation, add_generation_prompt=True, enable_thinking=True, return_tensors="pt"
).to(model.device)
outputs = model.generate(
inputs, max_new_tokens=65536, temperature=0.6, top_p=0.95, top_k=20, do_sample=True
)
response = tokenizer.decode(outputs[0][inputs.shape[-1] :], skip_special_tokens=True)
print(response)
@article{yu2024dare,
title={Language Models are Super Mario: Absorbing Abilities from Homologous Models as a Free Lunch},
author={Yu, Le and Yu, Bowman and Yu, Haiyang and Huang, Fei and Li, Yongbin},
journal={arXiv preprint arXiv:2311.03099},
year={2023}
}
@inproceedings{yadav2023ties,
title={Resolving Interference When Merging Models},
author={Yadav, Prateek and Tam, Derek and Choshen, Leshem and Raffel, Colin and Bansal, Mohit},
booktitle={Advances in Neural Information Processing Systems (NeurIPS)},
volume={36},
pages={7093--7115},
year={2023}
}
@article{yang2024deltanet,
title={Gated Delta Networks: Improving Mamba2 with Delta Rule for Linear Recurrent Attention},
author={Yang, Songlin and Wang, Bailin and Shen, Yikang and Panda, Rameswar and Kim, Yoon},
journal={arXiv preprint arXiv:2412.06464},
year={2024}
}
model-00007-of-00008.safetensors 2.7 GBmodel-00005-of-00008.safetensors 2.7 GBmodel-00003-of-00008.safetensors 2.7 GBmodel-00004-of-00008.safetensors 2.6 GBmodel-00006-of-00008.safetensors 2.6 GBmodel-00002-of-00008.safetensors 2.6 GBmodel-00001-of-00008.safetensors 2.0 GBmodel-00008-of-00008.safetensors 920 MBQwen/Qwen3.5-9B