Canopy-258M-R3 v5: Frontier Recurrent MoE & Autonomous Browser Agent

Canopy-258M-R3 v5 is a state-of-the-art 258.56M parameter Recurrent Mixture-of-Experts (MoE) model optimized for high-speed edge reasoning, long-trace memory efficiency, and robust web automation.

v5 incorporates groundbreaking architectural advances synthesized from late August / September 2026 frontier research:

Prefix Sliding KV-Cache Engine (Stanford / UW / Prime): Bounded memory test-time scaling (1.50x faster token generation during long reasoning bursts).
SMELT Recurrent Residual Scaling (Tsinghua / ByteDance): $1/\sqrt{2}$ stabilization across recurrent layer loops, eliminating attention sink saturation.
Curriculum Model Averaging (CMA) (Tsinghua / Pengcheng): Optimal geometric-interpolated weights between deep reasoning and high-precision browser dispatch stages.
sPTC Speculative Programmatic Tool Calling (spec-ptc): Parallel element coordinate pre-computation, achieving 1.46x faster form fills.
Prime Agent Resilient Membrane (Princeton / Prime): Self-healing execution harness with automated modal backdrop dismissal and re-grounding.
SPADE Synthetic Procedural Environments (UW / Stanford / Choi): Verifiable multi-step synthetic Gym environments.

Performance Benchmarks: v3 vs. v5

1. Model Latency & Memory Scaling

Architectural Engine	Metric	v3 Baseline	v5 (Current)	Improvement
Prefix Sliding KV-Cache	512-token reasoning burst	851.6 ms	567.0 ms	1.50x FASTER (33.4% lower latency)
KV-Cache Memory Footprint	Bounded context memory	Linear growth	Capped (Prefix 128 + Window 512)	Stable $O(1)$ memory bound
Residual Variance (SMELT)	Recurrent layer stability	Norm Drift	$1/\sqrt{2} \approx 0.7071$ scaling	Zero attention sink saturation

2. Browser Execution & Autonomous Tool Calling

Mechanism	Benchmark Task	v3 Serial	v5 Optimized	Speedup
sPTC Speculative Tool Caller	4-Field Form Fill	431.2 ms	295.7 ms	1.46x FASTER
Prime Agent Membrane	Modal Overlay Recovery	Timeout / Abort	166.9 ms Self-Healing	100% Recovery
SPADE Procedural Env	Multi-Item Order Flow	N/A	742.9 ms (100% Verified)	Ground Truth Verified

3. Complex Chained Actions Benchmark (Multi-Stage Battery)

Chained Action Scenario	Total Actions	v3 Latency	v5 Latency	Speedup
Chain 1: E-Commerce Multi-Stage Cart & Checkout	12 actions	1,774.4 ms	1,330.5 ms	25.0% FASTER
Chain 2: ETL Bulk Filter & Modal Dispatch	7 actions	1,313.8 ms	1,116.2 ms	15.0% FASTER
Chain 3: Spatial Grounding & Dynamic Extraction	4 actions	453.0 ms	323.8 ms	28.5% FASTER
Cumulative Chained Total	23 Complex Actions	3,541.2 ms (3.54s)	2,770.4 ms (2.77s)	21.8% FASTER OVERALL

Model Architecture Specifications

Hyperparameter	Value	Description
Total Parameters	258,555,654	Standalone weights with tied embeddings
Active Parameters	~112,000,000	Active parameter compute per token
Recurrent Layers	18 effective layers	3 Prelude + 6 Recurrent (visited 2x) + 3 Coda
Recurrent Scaling	$1/\sqrt{2} \approx 0.7071$	SMELT recurrence variance stabilization
KV-Cache Engine	Prefix Sliding	128 prefix tokens + 512 sliding window tokens
MoE Routing	Top-2 of 8 Experts	Dense first 3 layers, MoE middle/coda layers
Tokenwise Thought Bus	192 channels	Persistent reasoning state across recurrent passes
Context Window	2,048 tokens	RoPE position embeddings
Vocabulary Size	49,152	Byte-level BPE tokenizer (Cosmo-2)

Quickstart: Python Inference & Web Automation

1. Model Loading with Prefix Sliding

import torch
from canopy_r3.config import CanopyConfig
from canopy_r3.model import CanopyForCausalLM
from transformers import AutoTokenizer

model_id = "psikosen/canopy-258m-r3-v5"
tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)

config = CanopyConfig(
    enable_prefix_sliding=True,
    prefix_tokens_len=128,
    sliding_window_len=512,
)
model = CanopyForCausalLM.from_pretrained(model_id, config=config, torch_dtype=torch.bfloat16).cuda()

prompt = "<|im_start|>user\nDescribe the sPTC speculative tool calling algorithm.<|im_end|>\n<|im_start|>assistant\n"
inputs = tokenizer(prompt, return_tensors="pt").cuda()
outputs = model.generate(**inputs, max_new_tokens=256, temperature=0.2)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. Fast Browser Agent Execution (`miniswardbower`)

import asyncio
from miniswardbower.browser.controller import BrowserController
from miniswardbower.core.config import BrowserConfig
from miniswardbower.core.schemas import BrowserAction, BrowserActionType

async def run_agent():
    controller = BrowserController(BrowserConfig(headless=True))
    await controller.start()
    try:
        await controller.goto("https://news.ycombinator.com")
        
        # Speculative chunk execution with atomic DOM fills
        chunk = [
            BrowserAction(op=BrowserActionType.TYPE, target="input[name='q']", text="Canopy MoE", stream_input=False),
            BrowserAction(op=BrowserActionType.PRESS, key="Enter")
        ]
        results = await controller.execute_chunk(chunk)
        print("Chunk executed in record time:", results)
    finally:
        await controller.stop()

asyncio.run(run_agent())

License

Released under the Apache 2.0 License.

psikosen/canopy-258m-r3-v4