Qwen3.8-27B → 14B for code — experimental CMF

Qwen3.8-27B compressed for one set of tasks with a single cortiq compress command. The task set is code in six languages, HTML/three.js layouts, and short RU/EN reasoning and tool-use questions. Every FFN keeps the 24 % of its neurons the task uses, and the rest is folded into them. The model was then healed on the 27B's answers and quantized to 4 bits.

• Size: 14.4B parameters in a 7.6 GB file. The 27B takes 14.3 GB in the same 4-bit codec.
• Speed: 89 tok/s with the retrained speculative-decoding draft, 77 tok/s without it. The 27B decodes 32 tok/s on the same card in q8_2f.

Experimental sample. This file shows what task compression does on a large model. It is not a drop-in replacement for the 27B: its code has the same form and structure as the 27B's, but the content is less accurate (see Quality). Outside the task, expect losses.

English · Русский · 中文

Quick start

cargo install cortiq-cli          # or a prebuilt binary from the releases page
hf download infosave/Qwen3.8-27B-Code-14B-Experimental-cmf qwen38-27b-code-14b-q4tp.cmf --local-dir .
CMF_GRAPH_SPEC_K=2 cortiq run qwen38-27b-code-14b-q4tp.cmf --no-think --max-tokens 1024 \
  --prompt "Write a Go hash map with open addressing, error handling and unit tests."

• Use cortiq 0.8.46 or later. Prebuilt binaries for Linux, macOS and Windows are on the releases page.
• --no-think matters: the model was healed on direct answers, not on reasoning.
• CMF_GRAPH_SPEC_K=2 fixes the speculative draft depth at 2, where the retrained draft is accepted 87 % of the time (see Speed).
• On a Mac add CMF_GRAPH_SPEC=1: on native Metal the speculative draft runs only when asked for.
• Task data: the 483 prompts and the ready teacher dataset are in data/.

Files

What changed against Qwen3.8-27B:

• FFN: every one of the 64 layers keeps 4224 of its 17408 neurons. The dropped neurons are folded into the kept ones.
• Unchanged: the depth, the attention and GatedDeltaNet blocks, the embeddings, the head and the chat template.

Task data

The prompts and the teacher's answers this file was compressed with:

• Repeat this compression without API calls: pass the dataset to cortiq compress in place of the prompts (see How it was made).
• Another source model: re-tokenize the text answers with its tokenizer, cortiq teacher-data import data/q27-code-t27.tdata.answers.jsonl my.tdata --tokenizer my-model.cmf --split 80/10/10.
• Prompt sources: 222 of the prompts are aider-polyglot exercises, which come from Exercism (MIT license).

Results

Quality

Negative log-likelihood of 88 held-out answers of the 27B teacher, on prompts the compression never saw:

• Healing: it removes 39 % of the loss the fold adds.
• 4-bit quantization: GPTQ adds 0.009 on top of the healed model.
• Quality gates pass. In greedy generations from 16 different held-out prompts, this file loops on one, and the 27B loops on none.

What the answers look like:

• Structure matches the 27B. On 15 of 16 prompts the code is coherent and shaped like the 27B's: the same error declarations and types in the Go hash map, and C++ perfect numbers ending by itself after 223 tokens (27B: 229).
• One prompt loops. On a Java rate limiter the model invents an import (java.util.concurrent.locks.RWLock) and repeats the import list.
• Accuracy is lower. For Exercism "gigasecond" the 27B answers in 62 tokens (return start + 1000000000;). The 14B works through the conversion in comments, gets the arithmetic wrong, and runs out of its token limit.

Speed

One stream, greedy decoding, 384 tokens, two coding prompts (Go hash map / Java bank account), tok/s:

• Same text: speculative decoding verifies the draft's tokens with the full model, so the output equals plain greedy decoding.
• Why the draft was retrained: after compression the original 27B draft was accepted only 54 % of the time, and speculation no longer paid. cortiq mtp-heal retrained it on this model's own predictions over a 3-step chain (38 min). It is now accepted 87 % of the time at depth 2.
• Adaptive depth: the engine's default adaptive depth climbs to 3–5 and gives only 81.8 / 76.7 tok/s, hence CMF_GRAPH_SPEC_K=2.
• On a Mac (cortiq 0.8.46):
  • The numbers are two runs of the Go prompt with CMF_GRAPH_SPEC=1 CMF_GRAPH_SPEC_K=2.
  • The draft is accepted 87 % of the time, as on Vulkan, and the text equals plain decoding.
  • The adaptive depth gives 13.2 tok/s there.
  • cortiq 0.8.45 runs this file on a Mac without speculation.

How it was made

Data. 483 prompts:

• 222 aider-polyglot exercises in JavaScript, Java, Go, Python, Rust and C++;
• 177 other coding tasks (backend, frontend, DevOps, algorithms, tests, HTML, three.js, …);
• 84 RU/EN questions on knowledge, math, logic and tools.

The teacher was Qwen3.8-27B via OpenRouter, without thinking. It gave two answers per prompt (966 answers) for $2.38 in 15 minutes. The split is 80/10/10 by prompt, so both answers to a prompt stay on one side.

Both are published in data/: the prompts as data/prompts.jsonl, the generated dataset as data/q27-code-t27.tdata.

# how the dataset was generated
OPENROUTER_API_KEY=… cortiq teacher-data generate openrouter:qwen/qwen3.8-27b data/prompts.jsonl code.tdata \
  --tokenizer qwen38-27b-tokenizer.cmf --samples 2 --no-think \
  --sampling temperature=0.6,top_p=0.95,top_k=20 --split 80/10/10

# the compression, here from the published dataset (no API calls)
hf download infosave/Qwen3.8-27B-Code-14B-Experimental-cmf --include "data/*" --local-dir .
cortiq compress hf/Qwen3.8-27B data/q27-code-t27.tdata qwen38-27b-code-14b-q4tp.cmf q4tp --size 14B \
  --min-width 0.2 --illum-tokens 200000 --keep-grams off --heal fcd --heal-loss ce --heal-epochs 2

• Hardware: one RTX PRO 6000 (96 GB), 116 GB RAM, 27 CPU cores.
• Total: 1 h 34 min without the draft.
• With the cortiq 0.8.45 defaults the heal stops once its held-out loss stops improving, and the same recipe took 1 h 10 min.
• Full write-up: the stage-by-stage article, Compressing Qwen3.8-27B to 14B on one RTX PRO 6000 (also in Russian and Chinese).

Compress a model for your own task

The same command turns any supported model into a specialist for your prompts. Some uses:

• a coding assistant for your stack;
• a support or documentation bot for your product;
• a tool-calling agent;
• a model that has to fit a smaller GPU, a laptop or a phone.

1. Describe the task with prompts. Write one JSON object per line, with prompt or messages; id and category are optional. A few hundred prompts that look like real traffic work better than thousands of generic ones.

{"id": "go_lru", "category": "go", "prompt": "Implement a thread-safe LRU cache in Go with tests."}
{"id": "sql_q3", "category": "sql", "messages": [{"role": "user", "content": "Rewrite this query to use a window function: ..."}]}

2. Run one command. The source can be a .cmf, a local Hugging Face folder or a hub repo id. By default the source answers the prompts itself. --teacher takes a larger model instead: another .cmf with the same tokenizer, or any OpenAI-compatible API.

# a quarter smaller, then 4-bit; the model teaches itself
cortiq compress Qwen/Qwen3.5-2B prompts.jsonl qwen2b-task.cmf q4tp --ratio 0.75

# a larger teacher behind an API (OpenRouter, OpenAI, vLLM, SGLang, llama.cpp, Ollama, …)
OPENROUTER_API_KEY=… cortiq compress Qwen/Qwen3.8-27B prompts.jsonl my-14b.cmf q4tp --size 14B \
  --teacher openrouter:qwen/qwen3.8-27b --no-think --gen-samples 2

• Size target: --ratio 0.75 keeps 75 % of the parameters, --size 14B sets a parameter count, --ffn 0.5 keeps half of the FFN neurons.
• Codec: the last positional argument (q4tp, q8_2f, …) quantizes the result with GPTQ on the same task data.
• --plan-only prints the widths and parameter count before anything runs.

3. Read the report. Each stage prints its progress with an estimate of the time left. An interrupted run resumes with the same command, and finished stages are cached in OUTPUT.work/.

• OUTPUT.work/S6.eval/report.md compares the source, the fold, the healed model and the final file on held-out answers.
• The output is written only if it passes the quality gates; --write-on-fail keeps a rejected one.
• --compare other.cmf scores another file on the same test, for example a plain 4-bit quant.

Hardware. The heavy stages run on cortiq's GPU trainer: illumination, fold, heal, GPTQ calibration, evaluation and the draft's retraining.

• GPUs: NVIDIA through Vulkan, or Apple Silicon through Metal.
• Model families:
  • The trainer runs the Qwen3.5 / 3.6 / 3.8 dense layer set: GatedDeltaNet, gated attention and SwiGLU.
  • Other dense architectures heal on the CPU.
  • Mixture-of-experts models drop the experts the task never routes to; they are not healed yet.

Rules of thumb (measured).

• Quantize first, cut structure on top. On Qwen3.5-2B, q4tp alone gives 1.03 GB at NLL 0.225, while fold + q4tp gives 0.91 GB at 0.354. Task compression adds to 4-bit quantization; it does not replace it.
• The heal decides the result.
  • Choosing neurons with DTG-MA masks gives a better fold than illumination, but after healing the two are equal.
  • Baking the task skill in with DTG-MA before compressing raised the 27B's own accuracy by 6 %, yet ended at the same 0.398 against 0.396.
• Tokenize the teacher's data with the target model's exact tokenizer. Even one model family differs: Qwen3.8-27B has 7 more special tokens than Qwen3.5-0.8B.
• Generate the teacher data with --no-think if you will serve without thinking. It is cheaper, and the answers are complete.
• Retrain the MTP draft after any structural compression. This is automatic with --mtp-heal auto.

Full guide: docs/COMPRESS.md (also in Russian and Chinese); the method and its measurements: task-compression article.

Limitations

• One target point (14B). Wider points (17B, 20B) were folded but not healed or scored.
• Task-specific scores. NLL and the gates measure our task set, not general benchmarks. General knowledge outside the task was not measured; expect losses there.
• Thinking and long context. The model was healed without thinking and on 2048-token windows. Its thinking mode and long-context quality are untested.
• Speed sample. Speed was measured on two prompts.

Patents and license

The compression methods are the subject of US patent applications by Oleg Urevich Kirichenko:

• cortiq is Apache-2.0. Its §3 patent grant covers these claims as practiced by the shipped software (PATENTS.md).
• The model weights inherit Qwen3.8-27B's Apache-2.0 license.

Verify the download

sha256sum -c qwen38-27b-code-14b-q4tp.cmf.sha256
cortiq info qwen38-27b-code-14b-q4tp.cmf

sha256: d4f28a8c4c2416b075df8125cb3aaa1d17a5dc695708b158bb95253719473631

---

Документация на русском

Qwen3.8-27B, сжатая одной командой cortiq compress под набор задач: код на шести языках, вёрстка HTML/three.js, короткие RU/EN-задачи на рассуждение и инструменты.

• Что сделано: в каждом FFN остались 24 % нейронов, которые использует задача, остальные вшиты в них свёрткой. Затем модель вылечена на ответах 27B и квантована в 4 бита.
• Размер: 14.4 млрд параметров, 7.6 ГБ; у 27B в том же кодеке 14.3 ГБ.
• Скорость: 89 ток/с с дообученным черновиком MTP, 77 ток/с без него; у 27B в q8_2f на той же карте 32.

Экспериментальный образец. Файл показывает, что даёт сжатие под задачу на большой модели. Заменой 27B он не является: по форме и структуре код похож на ответы 27B, по точности содержания модель слабее. Вне задачи она теряет больше.

Быстрый старт

cargo install cortiq-cli
hf download infosave/Qwen3.8-27B-Code-14B-Experimental-cmf qwen38-27b-code-14b-q4tp.cmf --local-dir .
CMF_GRAPH_SPEC_K=2 cortiq run qwen38-27b-code-14b-q4tp.cmf --no-think --max-tokens 1024 \
  --prompt "Напиши на Go хеш-таблицу с открытой адресацией, обработкой ошибок и тестами."

• Версия: cortiq 0.8.46 или новее.
• --no-think обязателен: модель лечили на прямых ответах.
• CMF_GRAPH_SPEC_K=2 задаёт глубину черновика 2, на которой он принимается в 87 % случаев.
• На Mac добавьте CMF_GRAPH_SPEC=1: на нативном Metal черновик включается только по запросу.

Данные задачи

Повторить это сжатие без вызовов API — передать датасет вместо промптов:

hf download infosave/Qwen3.8-27B-Code-14B-Experimental-cmf --include "data/*" --local-dir .
cortiq compress hf/Qwen3.8-27B data/q27-code-t27.tdata qwen38-27b-code-14b-q4tp.cmf q4tp --size 14B \
  --min-width 0.2 --illum-tokens 200000 --keep-grams off --heal fcd --heal-loss ce --heal-epochs 2

• Для другой исходной модели текстовые ответы нужно перетокенизировать её токенизатором: cortiq teacher-data import data/q27-code-t27.tdata.answers.jsonl my.tdata --tokenizer my-model.cmf --split 80/10/10.
• Источник промптов: 222 из них — упражнения aider-polyglot, взятые из Exercism (лицензия MIT).

Результаты

• Метрика: NLL посчитан на 88 отложенных ответах учителя.
• Ворота качества пройдены. В жадных генерациях с 16 разных отложенных промптов модель зацикливается на одном, 27B — ни на одном.
• Черновик MTP: после сжатия исходный черновик принимался в 54 % случаев. cortiq mtp-heal дообучил его цепочкой из 3 шагов, теперь на глубине 2 он принимается в 87 %. Текст тот же, что без спекуляции.
• MacBook Air M4 24 ГБ (Metal, cortiq 0.8.46):
  • без черновика 11.3–11.6 ток/с;
  • с CMF_GRAPH_SPEC=1 CMF_GRAPH_SPEC_K=2 15.6–17.4 ток/с, черновик принимается в 87 %, текст тот же;
  • cortiq 0.8.45 запускает файл на Mac без спекуляции.

Как сжать модель под свою задачу

# промпты: по одному JSON на строку с полем prompt или messages
cortiq compress Qwen/Qwen3.5-2B prompts.jsonl qwen2b-task.cmf q4tp --ratio 0.75

# учитель покрупнее через API
OPENROUTER_API_KEY=… cortiq compress Qwen/Qwen3.8-27B prompts.jsonl my-14b.cmf q4tp --size 14B \
  --teacher openrouter:qwen/qwen3.8-27b --no-think --gen-samples 2

Что происходит:

1. Учитель отвечает на промпты.
2. Засветка показывает, какие нейроны использует задача.
3. Лишние нейроны вшиваются в оставшиеся (осмотическая свёртка).
4. Модель лечится на ответах учителя.
5. GPTQ квантует её на тех же данных, черновик MTP дообучается.
6. Отложенные ответы оценивают результат.

Как следить: у каждой стадии видны прогресс и оценка оставшегося времени. Прерванный запуск продолжает та же команда. Отчёт — OUTPUT.work/S6.eval/report.md.

Где работает: тяжёлые стадии идут на GPU-тренере — NVIDIA через Vulkan или Apple Silicon через Metal.

Что подтвердили замеры:

• Сначала квант, потом структура. На 2B просто q4tp даёт 1.03 ГБ и NLL 0.225, а свёртка + q4tp — 0.91 ГБ и 0.354.
• Итог определяет лечение. Ни выбор нейронов, ни запекание навыка DTG-MA до сжатия после лечения не дают выигрыша.
• Токенизатор данных учителя — точно от целевой модели.

Подробно:

• руководство по cortiq compress;
• статья о сжатии 27B → 14B.

Патенты

Методы сжатия — предмет заявок на патенты США (автор — Олег Юревич Кириченко):

• US 19/731,402 — осмотическая свёртка удалённых нейронов FFN в оставшиеся (§[0025]–[0026]), лечение свёрнутой сети (§[0022]) и контейнер CMF;
• US 19/738,763 — восстановление по учителю: FFN обучается на тексте учителя, остальное заморожено;
• US 19/452,464 — DTG-MA, маски активации под задачу (--select dtgma, cortiq dtgma-bake; в этом файле не использовались).

cortiq распространяется под Apache-2.0; его §3 даёт патентную лицензию на эти пункты в составе поставляемого ПО (PATENTS.md). Веса наследуют лицензию Apache-2.0 модели Qwen3.8-27B.

---

中文说明

Qwen3.8-27B 用一条 cortiq compress 命令针对一组任务压缩而成：六种语言的代码、 HTML/three.js 页面布局，以及俄语/英语的简短推理与工具调用问题。

• 做法： 每个 FFN 只保留任务用到的 24% 神经元，其余神经元渗透折叠 （osmotic folding）进保留的神经元；随后在 27B 的回答上修复（healing）， 并量化为 4 位。
• 大小： 144 亿参数，文件 7.6 GB；27B 在同一编码格式下为 14.3 GB。
• 速度： RTX PRO 6000 上配合重新训练的 MTP 草稿为 89 token/s，不用草稿为 77 token/s；27B（q8_2f）在同一显卡上为 32 token/s。

实验性样本。 本文件展示任务压缩在大模型上的效果，并不能替代 27B：代码的 形式和结构与 27B 相似，但内容准确性较低；任务之外的损失更大。

快速开始

cargo install cortiq-cli
hf download infosave/Qwen3.8-27B-Code-14B-Experimental-cmf qwen38-27b-code-14b-q4tp.cmf --local-dir .
CMF_GRAPH_SPEC_K=2 cortiq run qwen38-27b-code-14b-q4tp.cmf --no-think --max-tokens 1024 \
  --prompt "Write a Go hash map with open addressing, error handling and unit tests."

• 版本： cortiq 0.8.46 及以上。
• --no-think 必须加：模型是在直接回答上修复的。
• CMF_GRAPH_SPEC_K=2 把推测解码草稿的深度固定为 2，此时草稿接受率为 87%。
• 在 Mac 上再加 CMF_GRAPH_SPEC=1：原生 Metal 仅在显式请求时启用草稿。
• 任务集： 压缩只针对俄语/英语提示，中文提示未经评估。

结果

• 指标： NLL 在 88 条留出的教师回答上计算。
• 质量门槛通过： 在 16 个不同留出提示的贪心生成中，本模型在 1 个提示上出现 循环，27B 为 0 个。
• MTP 草稿： 压缩后原草稿的接受率只有 54%。cortiq mtp-heal 以 3 步链式训练 后，深度 2 的接受率为 87%，输出文本与不用推测解码时相同。
• MacBook Air M4 24 GB（Metal，cortiq 0.8.46）： 不用草稿 11.3–11.6 token/s； 使用 CMF_GRAPH_SPEC=1 CMF_GRAPH_SPEC_K=2 时为 15.6–17.4 token/s，接受率 87%， 文本相同。cortiq 0.8.45 在 Mac 上运行此文件时不启用推测解码。

任务数据

无需调用 API 即可复现本次压缩，把数据集代替提示传入：

hf download infosave/Qwen3.8-27B-Code-14B-Experimental-cmf --include "data/*" --local-dir .
cortiq compress hf/Qwen3.8-27B data/q27-code-t27.tdata qwen38-27b-code-14b-q4tp.cmf q4tp --size 14B \
  --min-width 0.2 --illum-tokens 200000 --keep-grams off --heal fcd --heal-loss ce --heal-epochs 2

• 换用其他源模型： 用该模型的分词器重新分词文本回答： cortiq teacher-data import data/q27-code-t27.tdata.answers.jsonl my.tdata --tokenizer my-model.cmf --split 80/10/10。
• 提示来源： 其中 222 条为 aider-polyglot 练习，来自 Exercism（MIT 许可）。

为自己的任务压缩模型

# 提示：每行一个 JSON，含 prompt 或 messages 字段
cortiq compress Qwen/Qwen3.5-2B prompts.jsonl qwen2b-task.cmf q4tp --ratio 0.75

# 通过 API 使用更大的教师模型
OPENROUTER_API_KEY=… cortiq compress Qwen/Qwen3.8-27B prompts.jsonl my-14b.cmf q4tp --size 14B \
  --teacher openrouter:qwen/qwen3.8-27b --no-think --gen-samples 2

流程：

1. 教师模型回答提示。
2. 照明（illumination）统计找出任务使用的神经元。
3. 多余的神经元折叠进保留的神经元（渗透折叠）。
4. 模型在教师回答上修复。
5. 用同一数据做 GPTQ 量化，并重新训练 MTP 草稿。
6. 用留出回答评估结果。

如何跟踪： 每个阶段都显示进度和剩余时间估计；中断后用同一条命令继续； 报告位于 OUTPUT.work/S6.eval/report.md。

运行环境： 繁重的阶段运行在 GPU 训练器上——NVIDIA（Vulkan）或 Apple Silicon（Metal）。

实测结论：

• 先量化，再压缩结构。 在 2B 上单纯 q4tp 得到 1.03 GB、NLL 0.225，而折叠 + q4tp 为 0.91 GB、0.354。
• 结果由修复决定。 无论是神经元的选择方式，还是压缩前先用 DTG-MA 烘焙技能， 修复之后都没有带来收益。
• 教师数据必须用目标模型自己的分词器分词。

详细资料：

• cortiq compress 指南；
• 面向任务压缩的方法文章；
• 27B → 14B 压缩逐步记录。

专利与许可

压缩方法是 Oleg Urevich Kirichenko 的美国专利申请的主题：

• US 19/731,402：把移除的 FFN 神经元渗透折叠进保留的神经元（§[0025]–[0026]）、 折叠后网络的修复（§[0022]）以及 CMF 容器；
• US 19/738,763：基于教师的恢复——FFN 在教师文本上训练，其余部分冻结；
• US 19/452,464：DTG-MA，任务引导的掩码激活（--select dtgma、 cortiq dtgma-bake；本文件未使用）。

cortiq 采用 Apache-2.0 许可，其第 3 条在随附软件实施这些权利要求的范围内授予 专利许可（PATENTS.md）。 模型权重沿用 Qwen3.8-27B 的 Apache-2.0 许可。

校验下载

sha256sum -c qwen38-27b-code-14b-q4tp.cmf.sha256
cortiq info qwen38-27b-code-14b-q4tp.cmf

sha256：d4f28a8c4c2416b075df8125cb3aaa1d17a5dc695708b158bb95253719473631