Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-GGUF

Project

• GitHub · LynnMerkyor/Lynn

LCB note: LCB is the standard abbreviation for LiveCodeBench. It selects competitive-programming problems published after the evaluated model's training-data cutoff and grades executable solutions with test cases to reduce training-data contamination. LCB100 on this card is the fixed 100-problem coding evaluation used for all compared tiers.

Why the featured 16GB LynnStyle tier is not uniform Q3

The released architecture remains 40 MoE blocks × 256 routed experts per block, with top-8 experts activated per token. The 16GB tier is a mixed-precision GGUF, not a uniform 3-bit conversion: it keeps the 256-expert architecture and reserves error budget for routing, convergence, and long-chain state paths.

Benchmark-REAP routing evidence was collected by forward-only observation on 128 MMLU, 64 GPQA, and 64 LCB prompts. It records router workloads only and does not use answer labels, canonical code, or execution outcomes to choose protected layers. A separate 24-chunk, 900-entry imatrix constrains tensor error inside the explicit protection schedule.

The three task families expose different hot blocks. Q3 v2 raises L8/L19/L21/L28 to Q3_K and L20/L25 to Q4_K to prioritize long-chain GPQA reasoning. These plots are routing evidence, not benchmark scores.

Actual protection schedule for the featured 16GB Q3 LynnStyle tier

For Q3 v2: embeddings/output and full-attention q/k/v/o use Q5_K; router inputs, norms, biases, and small controls remain F32; 40 shared-expert gates remain BF16; shared-expert and SSM paths use Q5_K/Q4_K. Routed experts use Q5_K on L2–L3, Q4_K on L0–L1/L20/L25/L39, Q3_K on L8/L19/L21/L28, IQ3_XXS on eight shoulder layers, and IQ2_S on the remaining 21 layers. The actual 673-tensor distribution is F32 261, BF16 40, Q5_K 69, Q4_K 204, Q3_K 12, IQ3_XXS 24, and IQ2_S 63.

This repository contains LynnStyle GGUF tiers for Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent plus an optional official vision projector.

Highlights

• Featured 16GB tier: Q3 LynnStyle v2, 14.28 GB (13.30 GiB).
• Versus the former Q3, v2 keeps MMLU unchanged, improves GPQA by 1.01pp, and moves LCB from 61/100 to 59/100 while reducing length/empty/runtime anomalies.
• This tier is reasoning-first. Avoid heavy code generation on Q3 v2; choose Q4 for coding.
• An optional official Q8_0 vision projector is provided and was functionally validated with the exact Q4 file. No MTP sidecar is included. This card does not claim a strict Pareto improvement.

Training lineage and data role

Full evaluation

Primary evaluation settings: open thinking, maximum output of 32768 tokens, and up to 1800 seconds per problem. MMLU500, GPQA198, and LCB100 are scored from final outcomes; parse, length, timeout, runtime, and empty failures are never silently converted into correct answers.

Complete Agent100 evaluation: Q4 and Q3 v2 each completed 100 real trajectories under the same C4, 32K-per-slot, 10-step, 8192-token-per-step, 900-second-timeout setup, followed by an independent gpt-5.6-sol (xhigh) final judge. Q4 scores 33/100 and Q3 v2 scores 32/100; false-verify counts are 12/20, protocol-error trajectory flags are 5/5, and judge errors are zero. A protocol-error flag describes the task trajectory; it is not a download, transport, or judge-service failure.

Q3 v2 is selected because MMLU stays flat and GPQA gains two correct answers over the former Q3. LCB loses two points, but average completion tokens fall 22.93%, length finishes improve from 16 to 10, empty code from 14 to 9, and runtime failures from 9 to 7. The remaining coding loss is the 13GB tradeoff, not a new generation-instability regression.

The actual Q3 16GB tradeoff

Quantization design

The REAP figures above document the task-family routing differences that inform the LynnStyle layer/path protection schedule. Direct quant, or imatrix alone without explicit core-path protection, is not a LynnStyle tier in this repository.

Download tiers

Availability: all four files below passed their own exact-file SHA, GGUF-parse, and runtime-generation gates. The clean full-evaluation table on this card remains limited to Q3/Q4; no Q5/Q8 benchmark score is inferred from an artifact gate.

Why the ModelScope sidebar mislabels the mixed quants

The GGUF panel in the right sidebar is generated by ModelScope's automatic scanner. It cannot fully represent a LynnStyle file that mixes Q5_K, Q4_K, Q3_K, IQ families, and preserved tensors by layer and tensor family. It may therefore label the mixed-quant Q3/Q4/Q5 files as unknown / IMATRIX; use the exact filename in the file list instead.

Open the repository's file list, download exactly one complete GGUF by the filename in the table above, and launch it with llama.cpp. Do not use the sidebar's unknown / IMATRIX entry to infer quantization type, file size, or VRAM requirements.

Actual context capacity depends on KV-cache format, backend, concurrency, and non-model VRAM use. This repository provides an optional official vision projector validated with the exact Q4 file, but no MTP sidecar.

llama.cpp launch

Featured Q3 16GB tier:

llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q3_16G-LynnStyle-imatrix.gguf \
  -ngl 999 -c 16384 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

This repository does not provide an MTP sidecar. Do not add --model-draft or treat the vision projector/any other GGUF as a draft model.

Q4 + official vision projector (tested)

This repository provides the optional official vision projector mmproj-Qwen3.6-35B-A3B-Q8_0.gguf (614.19 MB, SHA256 904cbf8c8e876220066ab3bf676c7efa40f3da372276fdaf8b01d2fb2a37a51d). It is neither a main model nor an MTP sidecar; pass it to llama.cpp together with the main model. Compatibility was validated only with the exact Q4 file in this repository; Q3/Q5/Q8 vision paths are not marked as tested.

The command below is the CPU-only, single-concurrency command that actually passed. It proves joint loading and functional image input; it is not a GPU TPS or VRAM claim:

CUDA_VISIBLE_DEVICES= llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q4-imatrix.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf \
  -ngl 0 --no-mmproj-offload --fit off \
  --image-min-tokens 1024 -c 3072 -np 1 -n 512 \
  --host 127.0.0.1 --port 18283 \
  --metrics --no-webui --cache-ram 0

Validated with llama.cpp b9922 (commit 230ea9d21): joint model/projector load and health passed; OCR returned exact Z8K4; the shape test returned three red circles and one blue square; the text regression returned exact TEXT PATH OK.

Limitations

• Vision compatibility was smoke-tested only for the exact Q4 + official Q8_0 projector. This is not a full vision benchmark, and Q3/Q5/Q8 vision paths are not marked as validated.
• Q3 v2 prioritizes MMLU/GPQA and scores 59/100 on LCB. For long, code-heavy work, prefer Q4.
• The model may still declare completion too early when evidence is insufficient. Preserve tool output and independently review high-risk tasks.

---

中文

LCB 说明： LCB 是 LiveCodeBench 的标准缩写。它按照被测模型训练数据的截止日期，选取该日期之后发布的竞赛编程题，并通过执行测试用例判定代码正确性，以降低训练数据污染。本卡的 LCB100 为固定 100 题同口径编码测试。

为什么 Q3 v2 选择推理优先

35B-A3B 的实际结构是 40 个 MoE block × 每层 256 个 routed experts，单 token 激活 top-8 experts。因此，Q3 16G v2 不是把整模型统一转成一种 3-bit 格式：它保留完整的256专家架构，让专家大矩阵承担主要压缩，同时保护路由、共享路径、full-attention、SSM及任务热点层。

这份策略有两层依据：Benchmark-REAP路由观测使用MMLU 128题、GPQA 64题和LCB 64题，只记录前向router负载，用来识别三类任务各自的热点层；24块、900 entries的imatrix再约束量化误差。REAP产物不保存题目、答案、标准代码或执行结果，也不使用正确性标签决定保护层。

三类任务的热点并不重合。v2因此把L8/L19/L21/L28提升到Q3_K，并把L20/L25提升到Q4_K，优先换回GPQA长链推理；MMLU维持旧Q3水平。图是前向路由证据，不是测评分数。

主推 16GB Q3 LynnStyle 的实际保护表

v2的673个tensor实际分布为：F32 261、BF16 40、Q5_K 69、Q4_K 204、Q3_K 12、IQ3_XXS 24、IQ2_S 63。标准GGUF把同一层的256个routed experts堆在一个张量中，所以当前公开档采用层级/路径保护，而不是在同一张量内给单个专家指定不同类型。

仅做 direct quant，或只有 imatrix 而没有这张显式保护表，都不属于本仓的 LynnStyle 档位。

这是 Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent 的 LynnStyle GGUF 量化仓，并提供一个可选官方视觉投影器。全部主模型档位都使用 LynnStyle MoE 分层混合精度，而不是对所有张量统一降位。

先看结论

• 16GB 主推档：Q3 LynnStyle v2，14.28 GB（13.30 GiB）。
• 相对旧Q3：MMLU持平，GPQA提高1.01pp；LCB从61/100降至59/100，但截长、空代码和runtime异常均减少。
• v2明确选择通识与重推理优先。13GB预算下的代码能力下降不再用额外量化补救；请规避用该档承担重度代码生成。
• 本仓新增可选官方 Q8_0 视觉投影器；已与精确 Q4 文件完成真实加载、OCR、图形识别与文本回归。
• 本仓仍不提供 MTP sidecar；不要把视觉投影器当作 draft model。
• ModelScope 源仓已通过发布前/发布后校验并开放下载。

项目

• GitHub · LynnMerkyor/Lynn

训练谱系与数据作用

同口径全量评测

主评测口径：open-thinking、最大输出 32768 tokens、单题最长 1800 秒；MMLU500 / GPQA198 / LCB100 均按最终结果计分，parse/length/timeout/runtime/empty 不会被静默改判为正确。

Agent100完整补测： Q4 与 Q3 v2 均完成100条真实轨迹，使用相同C4、每路32K、最多10步、单步最多8192 tokens与900秒超时设置，并由 gpt-5.6-sol（xhigh）按同一规则独立终判。Q4为 33/100，Q3 v2为 32/100；false-verify分别为12/20，protocol-error轨迹标记均为5，判分器errors均为0。protocol-error是任务轨迹结果，不是下载、传输或判分服务错误。

Q3 16G 的真实取舍

v2选择理由很直接：MMLU不退，GPQA多答对2题，平均思考tokens基本持平；虽然LCB少2题，但平均completion tokens下降22.93%，截长16→10、空代码14→9、runtime 9→7、timeout仍为1。也就是说，代码分数下降并不是生成异常恶化，而是13GB预算下代码能力本身的取舍。需要稳定写代码时请选择Q4；Q3 v2建议用于通识、推理和非重度代码场景。

下载档位

当前文件： 下列四份文件均已通过各自的同文件 SHA、GGUF 解析与真实生成门禁。下方 clean 全量评测表仍只报告 Q3/Q4；不会把文件门禁推导成 Q5/Q8 的评测分数。

为什么右侧自动量化档位显示不准确

ModelScope右侧的GGUF量化卡片来自平台自动扫描，无法准确表达LynnStyle这种逐层、逐tensor混合精度文件：Q3/Q4/Q5同时包含Q5_K、Q4_K、Q3_K、IQ系列和保留精度张量，平台因此可能标为unknown / IMATRIX。请以文件列表中的完整文件名为准。

请进入“模型文件”页，严格按上表完整文件名下载单个GGUF，并使用下方llama.cpp命令启动；不要依据右侧unknown / IMATRIX入口判断量化类型、文件大小或显存需求。

显存档位是实用选型，不等于文件大小；实际可用上下文依赖 KV cache 格式、后端、并发和系统显存占用。

llama.cpp 启动

Q3 16G 主推档：

llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q3_16G-LynnStyle-imatrix.gguf \
  -ngl 999 -c 16384 -np 1 \
  --host 0.0.0.0 --port 8080 \
  --jinja --reasoning on --reasoning-format deepseek

本仓不提供 MTP sidecar，不要添加 --model-draft 或把视觉投影器/任何其他 GGUF 当作 draft model。

Q4 + 官方视觉投影器（已实测）

本仓提供可选官方视觉投影器 mmproj-Qwen3.6-35B-A3B-Q8_0.gguf（614.19 MB，SHA256 904cbf8c8e876220066ab3bf676c7efa40f3da372276fdaf8b01d2fb2a37a51d）。它不是主模型，也不是 MTP sidecar；必须与主模型同时传给 llama.cpp。当前只对本仓精确 Q4 文件完成兼容性验收，未把 Q3/Q5/Q8 视觉能力标为已测。

以下是本次实际通过的 CPU-only 单并发命令；它用于证明联合加载与图像链路可用，不代表 GPU TPS 或显存成绩：

CUDA_VISIBLE_DEVICES= llama-server \
  -m Qwen3.6-35B-A3B-DSV4Pro-SFT-GPT56Sol-RL-Agent-LynnStyle-Q4-imatrix.gguf \
  --mmproj mmproj-Qwen3.6-35B-A3B-Q8_0.gguf \
  -ngl 0 --no-mmproj-offload --fit off \
  --image-min-tokens 1024 -c 3072 -np 1 -n 512 \
  --host 127.0.0.1 --port 18283 \
  --metrics --no-webui --cache-ram 0

验收环境为 llama.cpp b9922（commit 230ea9d21）：联合加载与健康检查通过，OCR 精确读取 Z8K4，图形题精确识别 3 个红圆和 1 个蓝方块，纯文本回归精确输出 TEXT PATH OK。

限制

• 视觉兼容性仅对本仓精确 Q4 + 官方 Q8_0 投影器完成 smoke；这不是完整视觉基准，也不代表 Q3/Q5/Q8 视觉路径已验收。
• Q3 v2优先保住MMLU/GPQA；LCB为59/100，不建议用于重度代码长链任务，代码优先请选择Q4。
• 模型仍可能在证据不足时过早宣告完成；高风险任务要保留工具输出并独立复核。