Qwen3.5-9B-GLiDE-LoRA (r=128 Teacher-ASVD + Full Norm-Folding + CodeReadout)
Rank-128 PEFT LoRA adapter and 255-way CodeReadout head extracted from fastino/GLiDE-9B onto Qwen/Qwen3.5-9B (and compatible with quantized Qwen3.5-9B backbones such as nchapman/Qwen3.5-9B-W4A16-HGPTQ).
Highlights
- Single-Engine Dual Serving (
warp-34+ PEFT): Serve standard causal chat completions (lora_id = -1) and non-causal/v1/systemonestructured decision queries (lora_id = "glide") from a single loadedQwen3.5-9Bbackbone. - Shared Fused Input Projections (
lora_ADeduplication): Fused sibling projections (q_proj/k_proj/v_proj,gate_proj/up_proj,in_proj_qkv/in_proj_z,in_proj_b/in_proj_a) share an identicallora_A.weightcomputed via Joint Teacher-ASVD (128calibration prompts fromSargeDev/jev-distill-corpus-v3withGLiDE-9Bteacher activations), reducing fused LoRAAprojection FLOPs and VRAM by 2x-3x when deduplicated by the inference engine. - Exact Offline
GemmaRMSNorm+RMSNormGatedFolding: Pre-norm scale shifts(1 + g_glide) / (1 + g_base)(input_layernorm,post_attention_layernorm),linear_attn.norm(g_glide / g_baseintolinear_attn.out_proj), and finalmodel.language_model.norm(folded intoreadout.safetensors) are absorbed into the adapter and readout weights with zero runtime overhead.
JevBench Public Eval (231 Cases: 48 Easy / 72 Original / 111 Hard)
Evaluated on kshitijthakkar/nirnaya-jevbench-public-eval-v1 using the native GLiDE quick-empty-think-medium-v1 prompt template and noncausal_full_attention:
| Model | Overall Acc | Easy (48) |
Original (72) |
Hard (111) |
NLL ($\downarrow$) | Brier ($\downarrow$) |
|---|---|---|---|---|---|---|
Base Qwen/Qwen3.5-9B (passes=1) |
194 / 231 (83.98%) |
48 / 48 (100.0%) |
70 / 72 (97.22%) |
76 / 111 (68.47%) |
0.4190 |
0.2212 |
Base Qwen/Qwen3.5-9B (passes=2 midloop) |
195 / 231 (84.42%) |
48 / 48 (100.0%) |
70 / 72 (97.22%) |
77 / 111 (69.37%) |
0.4188 |
0.2214 |
Qwen3.5-9B + GLiDE-LoRA (r=128, passes=1 single-pass) |
205 / 231 (88.74%) |
48 / 48 (100.0%) |
72 / 72 (100.0%) |
85 / 111 (76.58%) |
0.2819 |
0.1592 |
Qwen3.5-9B + GLiDE-LoRA (r=128, passes=2 midloop) |
207 / 231 (89.61%) |
48 / 48 (100.0%) |
72 / 72 (100.0%) |
87 / 111 (78.38%) |
0.2554 |
0.1410 |
Full fastino/GLiDE-9B (passes=1 single-pass) |
211 / 231 (91.34%) |
48 / 48 (100.0%) |
72 / 72 (100.0%) |
91 / 111 (81.98%) |
0.2068 |
0.1173 |
Full fastino/GLiDE-9B (passes=2 midloop) |
214 / 231 (92.64%) |
48 / 48 (100.0%) |
72 / 72 (100.0%) |
94 / 111 (84.68%) |
0.2049 |
0.1103 |
Repository Contents
adapter_config.json&adapter_model.safetensors: Standard PEFT LoRA weights (r=128,lora_alpha=128,rank_pattern&alpha_pattern=64onlinear_attn.in_proj_{a,b},bfloat16).decision_config.json:GLiDE-9Bdecision runtime metadata (attention_mode: "noncausal_full_attention",prompt_format: "quick-empty-think-medium-v1",midloop: {"blocks": [[12, 17], [18, 23]], "passes": 2},255option codesA..GC).readout.safetensors: 255-wayCodeReadouthead ([255, 4096]inbfloat16, with finalGemmaRMSNormratio folded in).