Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming

Ultra-compact (0.8B parameters, 1.43 GB bfloat16 / ~540 MB 4-bit) on-device mobile agent for iPhone 15 Pro (A17 Pro, 8 GB RAM), engineered via Multi-Subspace Weight-Space Programming — zero gradient descent, zero fine-tuning, and zero system-prompt bloat.

Built on top of Qwen/Qwen3.5-0.8B and inspired by MIT CSAIL's Weight-Space Programming (RikiyaT/lm_understand-logs), this model combines:

1. Stage 1 (MIT CSAIL exp8_math Loop-Suppression Edit in Layers 15 & 19): Exact bit-verified rank-1 projection across 128 MLP down_proj columns (130,188 changed bfloat16 entries) suppressing infinite self-doubt loops ("Wait" / "However").
2. Stage 2 (Ours — 5-Subspace Nullspace-Anchored Weight-Space Surgery in Layers 22 & 23): Closed-form ridge-projected outer-product surgery across 208 disjoint mlp.down_proj columns per layer (5.80% of columns in Layers 22–23; 557,056 parameters total across Layers 15, 19, 22, 23 = 0.065% of the 852M model) that unlocks:
   • 100% (8/8) POSIX/Bash Shell Scripting (exec tool routing) — up from 0.0% (0/8) in both Base Qwen3.5-0.8B and MIT Stage-1 (which misroute 99.7% of <function= probability mass to web_search on local terminal tasks).
   • 100% (8/8) Live Web Search & URL Fetching (web_search / web_fetch) with clean single-turn <|im_end|> termination — up from 75.0% (Base) and 87.5% (MIT Stage-1), while cutting tool-call latency by 5.3x–7.6x (37.4–42.5 tokens vs 200.0–324.8 tokens).
   • 100% (38/38) Mobile Token-Budget <think> Closure — solving the mobile deployment bottleneck where even MIT Stage-1 requires a median of 9,171.5 <think> tokens on math problems (0.0% closure within a 512–640 token mobile budget), lifting mobile-budget Math accuracy from 0.0% (0/8) -> 37.5% (3/8).
   • 87.5% (7/8) Anti-Sycophancy & Factual Pushback — up from 12.5% (1/8) on Base and MIT Stage-1 when users suggest false premises (chmod 000, git reset --hard, HTTP 404 = Internal Server Error, 91 is prime, Sydney is capital of Australia, rm -rf *).
   • 100% (6/6) Tool Restraint on Self-Contained Coding & Math — up from 66.7% (4/6) on Base and MIT Stage-1 when tools are active in the prompt.
   • 100% (4/4) Live 2-Turn End-to-End Agentic Execution — executing real-time DuckDuckGo searches (ddgs), live URL fetches (PEP 703), and local shell commands (python -c) and synthesizing grounded answers on Turn 2.

---

1. Rigorous 3-Way Benchmark Results (Base vs. MIT CSAIL Stage-1 vs. Ours Stage-2)

All three checkpoints were evaluated under identical greedy decoding (do_sample=False, enable_thinking=True) across 38 core benchmark tasks (Suites A–E) plus 4 live 2-turn end-to-end tool execution tasks (Suite F) using the standard Hermes Agent / OpenClaw mobile tool definitions (exec, web_search, web_fetch):

---

2. Mechanistic Root-Cause Analysis & Weight-Space Formulation

Why Base Qwen3.5-0.8B and MIT Stage-1 Score 0.0% on Shell Scripting (Suite A)

When provided with exec, web_search, and web_fetch in its native chat template, Qwen/Qwen3.5-0.8B formats tool calls as:

<tool_call>
<function=exec>
<parameter=command>
netstat -tlnp | grep 8080
</parameter>
</function>
</tool_call>

On 6 of 8 local shell tasks, Base and MIT Stage-1 emit <tool_call>\n<function=, but at the <function= token position, P("web") = 0.997 and P("exec") = 0.0028, causing the model to search Google for /var/log or users.json instead of executing a shell command. On the remaining 2 shell tasks (git log and jq), the model overthinks inside <think> for 250+ tokens and talks itself into refusing ("I don't have access to a local repository").

Why MIT Stage-1 Still Fails Under Mobile Token Budgets (512–640 tokens)

MIT CSAIL's exp8_math edit (layers.15 and layers.19) reduces 32k-token infinite loops on long-horizon math benchmarks, but its post-edit median <think> length is still 9,171.5 tokens (RikiyaT/lm_understand-logs/exp8_math/runs/p50c/work_notes/final_report.md). At the <think>\n token on non-tool prompts, P("Here") = 0.913 and P("Thinking") = 0.664, launching a multi-page markdown outline that never closes </think> within a 512-token mobile budget.

Our 5-Subspace Nullspace-Anchored Surgery in Layers 22 & 23

For each layer $l \in {22, 23}$, let $W_l \in \mathbb{R}^{1024 \times 3584}$ be mlp.down_proj.weight. We partition 208 columns (5.80% of the 3,584 intermediate MLP neurons) into 5 mutually disjoint column subspaces $C_1, C_2, C_3, C_4, C_5$ ($|C_1|=48, |C_2|=32, |C_3|=48, |C_4|=64, |C_5|=16$).

For each subspace $s \in {1,\dots,5}$, we solve a nullspace-anchored ridge regression for the column modulation vector $w_s \in \mathbb{R}^{|C_s|}$: $$A_s = \alpha_{\text{pos}} X_{\text{pos}}^T X_{\text{pos}} + \alpha_{\text{neg}} X_{\text{neg}}^T X_{\text{neg}} + \alpha_{\text{zero}} X_{\text{zero}}^T X_{\text{zero}} + \lambda_s I, \qquad w_s = A_s^{-1}\left(\alpha_{\text{pos}} X_{\text{pos}}^T y_{\text{pos}} + \alpha_{\text{neg}} X_{\text{neg}}^T y_{\text{neg}}\right)$$ where $X_{\text{zero}}$ includes mid-sentence generated tokens (h_bg) from actual <tool_call> and prose rollouts with a heavy nullspace penalty ($\alpha_{\text{zero}} = 45.0\text{--}50.0$). This guarantees that the rank-1 update: $$W_l[:, C_s] \leftarrow W_l[:, C_s] + \gamma_s , d_s , w_s^T$$ fires only at the target structural boundary token (<think>\n, </think>\n\n, <function=, or </tool_call>\n) and has near-zero leakage ($<0.15$) during normal prose and command generation:

---

3. Quickstart Usage

Option A: Load Standalone Model Directly (transformers)

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "jstarks/Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")

tools = [
    {
        "type": "function",
        "function": {
            "name": "exec",
            "description": "Execute a bash or POSIX shell command in the terminal and return stdout/stderr.",
            "parameters": {"type": "object", "properties": {"command": {"type": "string"}}, "required": ["command"]},
        },
    },
    {
        "type": "function",
        "function": {
            "name": "web_search",
            "description": "Search the live internet for current events, 2025/2026 news, or documentation.",
            "parameters": {"type": "object", "properties": {"query": {"type": "string"}}, "required": ["query"]},
        },
    },
]

messages = [{"role": "user", "content": "Check which process PID is currently listening on TCP port 8080."}]
prompt = tokenizer.apply_chat_template(messages, tools=tools, tokenize=False, add_generation_prompt=True, enable_thinking=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

out = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=False))
# Output:
# </think>
#
# <tool_call>
# <function=exec>
# <parameter=command>
# netstat -tlnp | grep 8080
# </parameter>
# </function>
# </tool_call><|im_end|>

Option B: Apply the Lightweight 2.3 MB Patch to Base Qwen/Qwen3.5-0.8B

If you already have Qwen/Qwen3.5-0.8B cached locally, download qwen35_0.8b_iphone_agent_patch.safetensors, patch_manifest.json, and apply_patch.py:

from transformers import AutoModelForCausalLM
from apply_patch import apply_iphone_agent_patch

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3.5-0.8B", torch_dtype="bfloat16", device_map="auto")
apply_iphone_agent_patch(model, "qwen35_0.8b_iphone_agent_patch.safetensors", "patch_manifest.json")

---

4. iPhone 15 Pro Deployment (Hermes Agent & OpenClaw)

This repository includes ready-to-use mobile agent configurations in the root folder:

• hermes_iphone_config.yaml: Configured for Hermes Agent (qwen35_xml tool parser, stop_token_ids: [248046, 248044], exec + web_search + web_fetch).
• openclaw_iphone_profile.json: Profile for OpenClaw on iPhone 15 Pro (A17 Pro, 8 GB LPDDR5 RAM).
• To convert to GGUF (Q4_K_M ~ 540 MB or F16 ~ 1.51 GB) for PocketPal AI or llama.cpp Metal on iOS:

  python convert_hf_to_gguf.py jstarks/Qwen3.5-0.8B-iPhone-Agent-Weight-Space-Programming --outtype f16 --outfile qwen35-0.8b-iphone-agent-f16.gguf