ThinkingCap Qwen3.8-27B — SVD-LoRA (base: Qwen3.8-27B)

Token-efficient reasoning as a LoRA adapter. This adapter reproduces the behaviour of bottlecapai/ThinkingCap-Qwen3.8-27B on top of Qwen/Qwen3.8-27B, so you can load one base model and hot-swap thinking styles with --lora.

What is ThinkingCap?

ThinkingCap is BottleCap AI's fine-tune of Qwen3.8-27B for token-efficient reasoning: it cuts thinking tokens by ~37% on average (11–66% depending on benchmark) while holding accuracy near the base model (85.8% vs 86.6% average). It shines on long-context retrieval, where it cuts reasoning by 39% with +2.3pp accuracy.

Original model card: bottlecapai/ThinkingCap-Qwen3.8-27B (Polyform Small Business license).

How this LoRA was extracted (weight-diff SVD)

Given two models sharing an identical architecture:

W_base   = Qwen3.8-27B weights
W_think  = ThinkingCap weights
Δ = W_think − W_base        ← fine-tune signal only, base cancels out
Δ ≈ U_r Σ_r V_rᵀ  (truncated SVD, rank r = 32)
lora_A = Σ_r^½ V_rᵀ ,  lora_B = U_r Σ_r^½

Pipeline (all computed on CPU in float32, then stored as the GGUF/PEFT adapter):

Architecture check — both repos expose the same 1,199 tensor names and identical shapes (qwen3_5_text, 64 layers, hidden 5120, intermediate 17408, MTP head, vocab 248320). No remapping needed.
Norm scan — Δ computed for all 866 non-visual tensors. 257 tensors had ‖Δ‖ ≥ 0.05; everything else was within fine-tune noise (lm_head and embed_tokens deltas were < 0.001, so embeddings are already covered by Qwen's).
Exact truncated SVD at uniform rank 32 on those 257 tensors → 257 lora_A/lora_B pairs.
Reconstruction validation — for a 40-tensor sample, the adapter's implied delta B@A vs the true delta Δ: mean relative error 0.88.

Important honesty note: this is an approximation

ThinkingCap is a diffuse fine-tune, not a low-rank edit. Even full-rank energy analysis shows:

rank	fraction of Δ energy captured (largest tensor per module)
16	21–29%
32	27–33%
64	34–37%
128	42–44%
256	47–48%

So this LoRA captures the dominant components of the token-efficiency fine-tune (the large singular directions that carry most of the behavioural change), but it is not an exact merge. Expect: noticeably shorter reasoning traces on Qwen3.8-27B, with some loss of ThinkingCap's fine-grained calibration. If you need bit-level fidelity, use the original full model instead.

Target	Tensor count	Δ norm range	energy retained @ r=32
`mlp.gate_proj`	64	up to 0.186	~29%
`mlp.up_proj`	64	up to 0.184	~27%
`mlp.down_proj`	64	up to 0.092	~16%
`self_attn.q_proj`	16	up to 0.149	~33%
`self_attn.o_proj`	16	up to 0.096	~24%
`linear_attn.out_proj`	48	up to 0.052	~23%
`linear_attn.in_proj_qkv`	48	up to 0.060	~19%

Files

File	Format	Notes
`adapter_model.safetensors`	PEFT LoRA	r=32, alpha=32, float32, 257 pairs
`adapter_config.json`	JSON	Standard PEFT config, `base_model: Qwen/Qwen3.8-27B`
`thinkingcap_qwen3.8-27B_svd_lora.gguf`	llama.cpp LoRA GGUF	load with `--lora` against a Qwen3.8-27B GGUF
`extraction_stats.json`	JSON	Per-tensor Δ norms, ranks, retained energy
`validation_report.json`	JSON	Reconstruction error on 40-tensor sample

Usage

llama.cpp (GGUF)

llama-cli -m Qwen3.8-27B.gguf \
    --lora thinkingcap_qwen3.8-27B_svd_lora.gguf \
    -p "..."

Or scaled (0.5 = halfway back toward stock Qwen behaviour):

llama-server -m Qwen3.8-27B.gguf \
    --lora-scaled thinkingcap_qwen3.8-27B_svd_lora.gguf:0.5 \
    --n-gpu-layers 99 --ctx-size 32768 -fa on

Python (PEFT + transformers)

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "Qwen/Qwen3.8-27B"
model = AutoModelForCausalLM.from_pretrained(base, torch_dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "hotdogs/thinkingcap_qwen3.8-27B_svd_lora")

License

Derivative of bottlecapai/ThinkingCap-Qwen3.8-27B, which carries the Polyform Small Business License 1.0.0. This adapter is intended for non-commercial research/evaluation use; commercial use should respect the upstream license. Base model Qwen3.8-27B is Apache-2.0.