88-Virtual-Layer Encode-Think-Decode Looped MoE Transformer retrofitted onto unsloth/Qwen3.6-35B-A3B-NVFP4 (40 physical layers, 35B total parameters, 256 routed experts/layer).
88L Virtual Depth): Prelude [0, 12) (x1) + Recurrent Core [12, 36) (x3) + Coda [36, 40) (x1).PassRouterAdapters) and cross-pass expert repulsion allow Pass 2+ to wake up distinct Top-8 experts from the frozen 32.2B MoE pool at each recurrent pass (expanding unique consulted parameters from ~3.0B up to ~4.5B-5.9B per token without adding base weight memory).mHC) Bridge: Doubly-stochastic Birkhoff-Sinkhorn mixing with prelude anchor injection and zero-gated step-0 bit-exact identity.HSSD v3.2): Upgrades all 30 linear-attention layers with a strictly disjoint local SWA + multi-timescale M=8 Top-2 routed page bank (write_mode="full", learnable SWA recency slope, low-rank 2D phase rotation, and chunkwise WY Tensor-Core parallelization).(layer, pass) compact lagged-replay HSSD state caches across recurrent core passes, paired with 1.0x pass-1 FP8 KV + per-pass D-LSA bucket sharing on Gated Full Attention layers (zero KV memory multiplication at 1M context).D-LSA): Position-free NoPE head indexing with active-slot compacted Ward clustering and hybrid sparse+KL calibration for sub-quadratic 1M-token retrieval.s2_joint_bptt (step 250)2026-11-22T02:22:26Z| Stage | Status | Step | LM Loss | Depth Gain (L_d1 - L_d2) |
Pass-2 Reroute Rate | Updated (UTC) |
|---|---|---|---|---|---|---|
s0_probe |
completed | 0 |
- |
- |
- |
2026-11-14T22:37:44Z |
s1_loop_a6b |
completed | 1000/1000 |
0.7498 |
0.0609 |
0.848 |
2026-11-16T21:16:01Z |
s1b_hssd_distill |
completed | 1500/1500 |
0.7253 |
0.0626 |
0.830 |
2026-11-19T04:33:49Z |
s3_long_1m_lsa |
completed | 600/600 |
2.7642 |
0.1414 |
- |
2026-11-20T20:09:13Z |
s2_joint_bptt |
in_progress | 250/1200 |
2.8368 |
0.0726 |
0.812 |
2026-11-22T02:22:26Z |
s4_rl |
pending | 0 |
- |
- |
- |
- |
from qloop.hub import load_qloop_from_hub
from qloop.eval.generate import Generator
stack = load_qloop_from_hub("TassP0lar/Qwen3.6-35B-A3B-Loop64")
gen = Generator(stack.model, stack.tokenizer, loop_controller=stack.loop)
with gen.depth(2): # 64 virtual layers with cross-pass dynamic expert routing
print(gen.generate(["Solve over Z: x^3 + y^3 + z^3 = 42."], max_new_tokens=2048)[0].text)
qloop_adapters.pt 542 MBstages/s2_joint_bptt/latest/trainable.pt 542 MBstages/s3_long_1m_lsa/latest/trainable.pt 332 MBstages/s1b_hssd_distill/latest/trainable.pt 308 MBstages/s1_loop_a6b/latest/trainable.pt 73 MBstages/s1b_hssd_distill/latest/rng.pt 10.6 kBstages/s2_joint_bptt/latest/rng.pt 10.6 kBstages/s1_loop_a6b/latest/rng.pt 10.6 kBstages/s3_long_1m_lsa/latest/rng.pt 10.5 kBunsloth/Qwen3.6-35B-A3B-NVFP4