MiMo-Ornith-9B-AGSI is a non-linear parameter-space synthesis of two leading fine-tuned models derived from the Qwen 9B architecture:
Rather than relying on naive linear averaging (LERP) or global spherical interpolation (SLERP)—which flatten parameter tensors into uncalibrated vectors—this model was fused using Adaptive Geodesic Spectral Interpolation (AGSI). AGSI executes row-wise hyperspherical geodesics on decoupled directional manifolds, enforces second-order spectral energy conservation, shields against anti-phase gradient interference, and dynamically modulates parameters across depth via a $C^2$-continuous quintic smoothstep curve.
Standard model interpolation techniques often suffer from:
AGSI resolves these issues through a five-stage manifold interpolation framework:
Linear layers compute transformations $y = x W^T$, where individual rows $W_i \in \mathbb{R}^{D_{\text{in}}}$ represent the hyperplanes of specific neurons. AGSI isolates radial feature scale from directional orientation:
$$m^{(i)} = |W^{(i)}|2 = \sqrt{\sum{j=1}^{D_{\text{in}}} (W_{ij})^2}$$
$$u^{(i)} = \frac{W^{(i)}}{m^{(i)} + \epsilon}, \quad \text{where } u^{(i)} \in S^{D_{\text{in}}-1}$$
By constraining directional updates to the unit hypersphere $S^{D_{\text{in}}-1}$, the model preserves the angular separation of neuron receptive fields.
For each individual neuron row $i$, the angular geodesic distance between checkpoint trajectories is calculated directly in its tangent space:
$$\theta_i = \arccos\left(\text{clamp}\left(\langle u_A^{(i)}, u_B^{(i)} \rangle, -1 + \delta, 1 - \delta\right)\right)$$
Rather than using Euclidean displacement, the directional basis moves along the great-circle arc:
$$u_{\text{fused}}^{(i)} = c_A^{(i)} u_A^{(i)} + c_B^{(i)} u_B^{(i)}$$
where the geodesic velocity coefficients are defined as:
$$c_A^{(i)} = \frac{\sin((1 - t)\theta_i)}{\sin\theta_i + \epsilon}, \quad c_B^{(i)} = \frac{\sin(t \theta_i)}{\sin\theta_i + \epsilon}$$
If $\sin\theta_i \to 0$ (quasi-collinear vectors), the interpolation smoothly transitions to normalized linear blending:
$$\lim_{\theta \to 0} u_{\text{fused}}^{(i)} = (1 - t)u_A^{(i)} + t u_B^{(i)}$$
When RL optimization (Ornith) and distillation gradients (MiMo) pull in opposing directions ($\theta_i > 90^\circ, \cos\theta_i < 0$), standard spherical combination results in substantial cancellation:
$$|u_A + u_B| = \sqrt{2 + 2\cos\theta} < \sqrt{2} \approx 1.414 \quad (\text{vs. } 2.0 \text{ when aligned})$$
AGSI monitors the directional dot product against a critical interference bound ($\tau = -0.05$, corresponding to $\theta \approx 92.86^\circ$). If destructive cancellation occurs:
$$\text{Conflict Condition: } \langle u_A^{(i)}, u_B^{(i)} \rangle < \tau$$
$$\gamma_A^{(i)} = \frac{m_A^{(i)}}{m_A^{(i)} + m_B^{(i)} + \epsilon}$$
$$u_{\text{fused}}^{(i)} = \begin{cases} 0.85, u_A^{(i)} + 0.15, u_B^{(i)}, & \text{if } \gamma_A^{(i)} \ge 0.5 \ 0.15, u_A^{(i)} + 0.85, u_B^{(i)}, & \text{if } \gamma_A^{(i)} < 0.5 \end{cases}$$
This dominance gate projects conflicting features toward the checkpoint exhibiting higher parameter variance, preventing the formation of dormant neurons.
In deep autoregressive networks normalized by RMSNorm, layer output variance relies heavily on parameter norm preservation:
$$\mathbb{E}[|W x|2^2] \approx \frac{1}{D{\text{in}}} |W|_F^2 , \mathbb{E}[|x|_2^2]$$
To maintain stable forward-pass activations without logit saturation or signal decay, AGSI scales the interpolated direction vector by the root-mean-square energy:
$$m_{\text{target}}^{(i)} = \sqrt{(1 - t)(m_A^{(i)})^2 + t(m_B^{(i)})^2}$$
$$W_{\text{candidate}}^{(i)} = m_{\text{target}}^{(i)} \cdot \frac{u_{\text{fused}}^{(i)}}{|u_{\text{fused}}^{(i)}|_2}$$
Finally, a global Frobenius norm correction is applied across the full matrix:
$$W_{\text{final}} = W_{\text{candidate}} \times \left( \frac{\sqrt{(1 - t)|W_A|_F^2 + t|W_B|F^2}}{|W{\text{candidate}}|_F + \epsilon} \right)$$
The mixing coefficient $t$ is non-static. It is governed by a $C^2$-continuous quintic polynomial function across the 32 transformer layers, augmented by functional block biases:
$$\xi = \frac{l}{L - 1} \in [0, 1], \quad l \in {0, 1, \dots, 31}$$
$$S(\xi) = 6\xi^5 - 15\xi^4 + 10\xi^3$$
$$t(l, \text{block}) = \text{clamp}\left(t_{\text{base}} + \Delta t \cdot \left(S(\xi) - 0.5\right) + \beta_{\text{block}}, , 0.0, , 1.0\right)$$
q, k, v, o, linear_attn): $\beta_{\text{attn}} = +0.04$ (Ornith priority for context selection and environment state tracking).gate, up, down_proj): $\beta_{\text{mlp}} = -0.04$ (MiMo priority for factual retention and associative coding memories).# The Golden SOTA Preset applied during parameter synthesis
BASE_ORNITH_RATIO = 0.48 # Foundational balance (52% MiMo / 48% Ornith)
DEPTH_MODULATION = 0.14 # Dynamic span across transformer depth
ATTN_ROUTING_BIAS = 0.04 # Attention block offset (favors Ornith)
MLP_ROUTING_BIAS = -0.04 # MLP block offset (favors MiMo)
ANTI_PHASE_THRESHOLD = -0.05 # Critical angle cutoff (θ = 92.86°)
SPECTRAL_ENERGY_EXPONENT = 2.0 # Second-order L2 moment conservation
This model is compatible with systems supporting the Qwen 9B architecture (e.g., standard Hugging Face transformers, vLLM, SGLang).
To run the model with vLLM, including support for streaming reasoning tokens (<think>) and agentic tool calls:
vllm serve DEST_REPO_ID \
--port 8000 \
--model DEST_REPO_ID \
--trust-remote-code \
--tensor-parallel-size 1 \
--max-model-len 131072 \
--gpu-memory-utilization 0.92 \
--reasoning-parser qwen3 \
--tool-call-parser hermes
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "DEST_REPO_ID"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
trust_remote_code=True
)
messages = [
{
"role": "system",
"content": "You are a master systems engineer and competitive programmer. Solve problems using precise step-by-step reasoning enclosed in <think> tags, then execute terminal commands or output production code."
},
{
"role": "user",
"content": "Write an optimized eBPF program in C that traces network socket latency outliers (>100ms) and provides a Python BCC analysis script."
}
]
prompt = tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
**inputs,
max_new_tokens=4096,
temperature=0.6,
top_p=0.95,
repetition_penalty=1.05
)
response = tokenizer.decode(outputs[0][inputs.input_ids.shape[1]:], skip_special_tokens=True)
print(response)
This model uses the unified Qwen chat template, configured to separate internal reasoning steps from final outputs:
<|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Write a bash script to monitor memory usage.<|im_end|>
<|im_start|>assistant
<think>
1. Identify target metrics: available vs. used memory.
2. Use /proc/meminfo or 'free -m' for portability.
3. Handle logging and alert thresholds.
</think>
Here is the monitoring script:
#!/usr/bin/env bash
set -euo pipefail
THRESHOLD=85
CURRENT=$(free | awk '/Mem:/ {printf("%.0f"), $3/$2 * 100}')
if [ "$CURRENT" -gt "$THRESHOLD" ]; then
echo "WARNING: Memory usage at ${CURRENT}%"
fi
<|im_end|>
model-00004-of-00004.safetensors 4.8 GBmodel-00003-of-00004.safetensors 4.8 GBmodel-00002-of-00004.safetensors 4.8 GBmodel-00001-of-00004.safetensors 3.6 GBmodel-00005-of-00005.safetensors 1.4 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo OliviaRossi/MiMo-Ornith-9B-AGSI ./model-folder