Qwen3.5-27B-GLiDE-LoRA (r=128 Teacher-ASVD + Full Norm-Folding + CodeReadout)
Rank-128 PEFT LoRA adapter and 255-way CodeReadout head extracted from fastino/GLiDE-27B onto Qwen/Qwen3.5-27B.
Highlights
- 100% of Full
GLiDE-27B(passes=2) JevBench Accuracy (213 / 231 = 92.21%): Matches the full 54 GBfastino/GLiDE-27Bcheckpoint on JevBench (213/231,NLL = 0.1988vs0.1972,Brier = 0.1053vs0.1033) in a 1.6 GB LoRA adapter (+5cases and-0.0180Brier over unweighted Joint-SVD). Even in single-pass mode (passes=1, withoutmidloop), it reaches211 / 231(91.34%,Brier = 0.1141). - Single-Engine Dual Serving (
warp-34+ PEFT): Serve standard causal chat completions (lora_id = -1) and non-causal/v1/systemonestructured decision queries (lora_id = "glide") from a single loadedQwen3.5-27Bbackbone. - Shared Fused Input Projections (
lora_ADeduplication): Fused sibling projections (q_proj/k_proj/v_proj,gate_proj/up_proj,in_proj_qkv/in_proj_z,in_proj_b/in_proj_a) share an identicallora_A.weightcomputed via Joint Teacher-ASVD (128calibration prompts fromSargeDev/jev-distill-corpus-v3withGLiDE-27Bteacher activations), reducing fused LoRAAprojection FLOPs and VRAM by 2x-3x when deduplicated by the inference engine. - Exact Offline
GemmaRMSNorm+RMSNormGatedFolding: Pre-norm scale shifts(1 + g_glide) / (1 + g_base)(input_layernorm,post_attention_layernorm),linear_attn.norm(g_glide / g_baserepeated across48value heads intolinear_attn.out_proj), and finalmodel.language_model.norm(folded intoreadout.safetensors) are absorbed into the adapter and readout weights with zero runtime overhead.
JevBench Public Eval (231 Cases: 48 Easy / 72 Original / 111 Hard)
Evaluated on kshitijthakkar/nirnaya-jevbench-public-eval-v1 using the native GLiDE-27B glide-v1 prompt template and noncausal_full_attention:
| Model | Overall Acc | Easy (48) |
Original (72) |
Hard (111) |
NLL ($\downarrow$) | Brier ($\downarrow$) |
|---|---|---|---|---|---|---|
Base Qwen/Qwen3.5-27B (passes=1 single-pass) |
204 / 231 (88.31%) |
48 / 48 (100.0%) |
69 / 72 (95.83%) |
87 / 111 (78.38%) |
0.3236 |
0.1738 |
Base Qwen/Qwen3.5-27B (passes=2 midloop) |
204 / 231 (88.31%) |
48 / 48 (100.0%) |
70 / 72 (97.22%) |
86 / 111 (77.48%) |
0.2947 |
0.1525 |
Qwen3.5-27B + GLiDE-LoRA (r=128 Joint-SVD, passes=1) |
206 / 231 (89.18%) |
48 / 48 (100.0%) |
70 / 72 (97.22%) |
88 / 111 (79.28%) |
0.2754 |
0.1522 |
Qwen3.5-27B + GLiDE-LoRA (r=128 Joint-SVD, passes=2) |
208 / 231 (90.04%) |
48 / 48 (100.0%) |
71 / 72 (98.61%) |
89 / 111 (80.18%) |
0.2275 |
0.1233 |
Qwen3.5-27B + GLiDE-LoRA (r=128 Teacher-ASVD, passes=1) |
211 / 231 (91.34%) |
48 / 48 (100.0%) |
71 / 72 (98.61%) |
92 / 111 (82.88%) |
0.2147 |
0.1141 |
Qwen3.5-27B + GLiDE-LoRA (r=128 Teacher-ASVD, passes=2) |
213 / 231 (92.21%) |
48 / 48 (100.0%) |
71 / 72 (98.61%) |
94 / 111 (84.68%) |
0.1988 |
0.1053 |
Full fastino/GLiDE-27B (passes=2 midloop) |
213 / 231 (92.21%) |
48 / 48 (100.0%) |
72 / 72 (100.0%) |
93 / 111 (83.78%) |
0.1972 |
0.1033 |
Full fastino/GLiDE-27B (passes=1 single-pass) |
214 / 231 (92.64%) |
48 / 48 (100.0%) |
72 / 72 (100.0%) |
94 / 111 (84.68%) |
0.1933 |
0.1022 |
Repository Contents
adapter_config.json&adapter_model.safetensors: Standard PEFT LoRA weights (r=128,lora_alpha=128,rank_pattern&alpha_pattern=96onlinear_attn.in_proj_{a,b},bfloat16).decision_config.json:GLiDE-27Bdecision runtime metadata (attention_mode: "noncausal_full_attention",prompt_format: "glide-v1",midloop: {"blocks": [[24, 35], [36, 47]], "passes": 2},255option codesA..GC).readout.safetensors: 255-wayCodeReadouthead ([255, 5120]inbfloat16, with finalGemmaRMSNormratio folded in).