👁️ Vision Support Added — This model now supports image input! Download a mmproj projector file from the file list and add
--mmprojto enable vision. See the Vision Support section below for details.
Hard runtime requirement: use the custom AEON ik-llama fork:
https://github.com/noonr48/qwen36-aeon-ik-llama
The root GGUF files in this repo are built for that fork. Treat this as an
ik-llamarelease, not a stockllama.cppmodel drop.
Newer fine-tunes built on this base — SignalLatch (a behaviour fine-tune) and PatchCode (an agentic-coder distil on top of SignalLatch). This repo is the non-finetuned base.
This model supports vision/image input. Qwen3.6-27B is natively a vision-language model. Download one of the mmproj (multimodal projector) files below and pass it with
--mmprojto enable image understanding.
The projector is extracted from the official Qwen/Qwen3.6-27B base model. Since text fine-tuning does not modify the vision encoder, one projector works across all three RYS variants (base, SignalLatch, PatchCode).
| File | Precision | Size | Link |
|---|---|---|---|
mmproj-Qwen3.6-27B-base-f32.gguf |
F32 (full precision) | 1.8 GB | ⬇ Download |
mmproj-Qwen3.6-27B-base-f16.gguf |
F16 (half precision) | 885 MB | ⬇ Download |
mmproj-Qwen3.6-27B-base-q8_0.gguf |
Q8_0 (8-bit quantized) | 601 MB | ⬇ Download |
Recommended: mmproj-Qwen3.6-27B-base-f16.gguf — best balance of quality and size.
Add --mmproj to your llama-server command:
./build/bin/llama-server -m Qwen3.6-27B-AEON-RYS-Agentic-Coder-PatchCode.IQ4_NL.gguf \
--mmproj mmproj-Qwen3.6-27B-base-f16.gguf \
--jinja -ngl 999 -c 200000
Then send images via the standard OpenAI-compatible API:
curl http://localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"messages":[{"role":"user","content":[
{"type":"image_url","image_url":{"url":"data:image/jpeg;base64,..."}},{"type":"text","text":"Describe this image"}
]}]}'
For higher-resolution images, add --image-max-tokens 16384 (default is 4096). Requires an ik-llama / llama.cpp build from May 2026 or later with Qwen3VL mtmd support.
The full transparent build/test/decision record for this non-finetuned release is here:
https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-15-20/index.html
That page is the detailed companion to this model card. It covers the layer-selection process, quantization-survival checks, runtime profile, practical-test notes, caveats, and the reasoning behind centering this release on the IQ4_NL GGUF. This Hugging Face README is the quick download/runtime page; the linked record is where the comprehensive details live.
For future planned RYS releases, this is the expected pattern: a concise model card for use, plus a public companion record for transparency.
SignalLatch behavior fine-tune page:
https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-signallatch/index.html
The broader goal of this model series is practical local work: capable, functional models that fit realistic hardware and can simply get tasks done. Built on the AEON uncensored base, this release is meant to stay low-friction: less lecturing, less getting in the way, and more focus on completing the task in front of it.
RYS and IQ4_NL are the practical part of that goal: preserving useful reasoning and coding behavior in a compact form factor. One 24 GB-class GPU can be enough for a serious local worker. If you have more GPUs, the natural deployment pattern is simple: run more workers in parallel and get more work done at once.
Default: use Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf for normal inference.
The BF16 artifacts are included for exploration, inspection, conversion, and continued work. The release itself is aimed at practical use through the IQ4_NL GGUF: a much smaller file that keeps the model's reasoning and coding behavior close to the BF16 source.
The point of RYS here is not a huge BF16 headline gain. In our testing, the RYS BF16 model only gave roughly a 1% gain over the Unsloth/official-style BF16 baseline. The useful result was quantization resistance: this specific AEON RYS 15,20 branch, custom runtime path, and IQ4_NL quantization held up far better after compression.
We had tried the same broad quantization approach on the official/non-RYS model before, but the outcome was poor: reasoning and coding quality degraded noticeably. The RYS branch was selected because it preserved those behaviors much better under the practical quantized release path.
Experimental: use the MTP GGUF only if you specifically want to test ik_llama.cpp multi-token prediction on this model.
Result: the MTP GGUF loads with the expected MTP tensors, but MTP mode was slower than the no-MTP path in our internal graph-split tests, and practical long-output checks favored the normal non-MTP IQ4_NL file.
Runtime: this is an ik-llama-targeted release, not a stock llama.cpp release.
Required runtime URL: https://github.com/noonr48/qwen36-aeon-ik-llama
Download/build that fork first; the recommended GGUF is intended for this runtime path.
RYS weight representation note: this release uses materialized RYS. The copied 15,20 window is exported as normal GGUF layer tensors, so output layers 20..24 have their own tensors copied from source layers 15..19 instead of aliasing those source buffers at runtime. That is intentional for this release: it keeps the checkpoint, GGUF conversion, quantization path, and downstream modification/fine-tune workflows explicit. A procedural/aliased runtime could be explored in future work to reduce duplicate weight memory, but the current IQ4_NL artifact has already been tested as stable and reliable, so I do not plan to change its representation without rebuilding and re-verifying the release.
Use the custom AEON ik-llama fork and start with this public deployment shape for the recommended non-MTP IQ4_NL GGUF:
./build/bin/llama-server \
-m /path/to/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf \
-c 65536 \
-ngl 999 \
-np 1 \
-fa on \
-sm graph \
--temp 0.7 \
--jinja \
--reasoning-format deepseek \
--reasoning-budget 0
For long-context use, the same profile can be run with a larger context. The default KV type is f16; we have also tested the f16 KV path up to about 160k context without seeing the earlier failure pattern:
-c 131072
Practical single-GPU deployment note: the Q4_NL release is small enough for practical use on a single RTX 3090 / 24 GB-class card. In an observed reference profile, roughly 160k context with default/FP16 KV fit at about 20.3 GiB total VRAM. Treat this as a deployment reference point, not a guaranteed cross-hardware memory benchmark.
For the later 131k practical canvas-agent comparison, we used FP32 KV as a conservative isolation setting to make sure any failure would not be caused by KV precision:
-c 131072 \
-ctk f32 \
-ctv f32
Runtime notes:
-c 65536 is the practical baseline profile; -c 131072 is the long-context comparison profile.-sm graph is the tested strength-sweep / comparison path. If you are running a RAM-cache parallel deployment, use the separate setup below.--ctx-checkpoints 0 for RAM prompt-cache deployments. The prompt may still match, but without recurrent context checkpoints the server can fall back to full prompt reprocessing.The custom AEON ik-llama fork now includes a runtime adjustment for explicit slot lanes and RAM prompt-cache reuse. This matters for agent harnesses such as OpenCode, where you may want one long-running worker on slot0 and another on slot1 instead of letting requests bounce between slots.
Generic two-slot server shape:
./build/bin/llama-server \
-m /path/to/Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf \
-c 360000 \
-np 2 \
-ngl 999 \
-b 256 \
-ub 64 \
-fa on \
-sm layer \
-ctk f16 \
-ctv f16 \
--temp 0.7 \
--jinja \
--reasoning-format deepseek \
--reasoning-budget 0 \
--alias qwen36-aeon-rys-q4nl \
-cram 65536
How to read that setup:
-np 2 creates two parallel slots.-c 360000 is the total server context budget; with two slots, each lane receives roughly half.-cram 65536 enables RAM prompt cache. The value is MiB and is server-wide, so adjust it to your host memory budget.--ctx-checkpoints 0 if you want RAM prompt-cache restores to avoid full prompt reprocessing.-sm layer for this two-GPU / two-slot RAM-cache path. On this recurrent/hybrid path, -sm graph disables the checkpoints needed for actual restore.--alias, then point the agent harness at <alias>-slot0 and <alias>-slot1.Minimal OpenCode-style provider shape:
{
"provider": {
"aeon-rys-q4nl-p2": {
"npm": "@ai-sdk/openai-compatible",
"name": "AEON RYS Q4_NL p2",
"options": {
"baseURL": "http://<host>:<port>/v1",
"timeout": false
},
"models": {
"qwen36-aeon-rys-q4nl-slot0": {
"name": "AEON RYS Q4_NL slot0",
"reasoning": true,
"interleaved": { "field": "reasoning_content" },
"limit": { "context": 180096, "input": 171904, "output": 8192 }
},
"qwen36-aeon-rys-q4nl-slot1": {
"name": "AEON RYS Q4_NL slot1",
"reasoning": true,
"interleaved": { "field": "reasoning_content" },
"limit": { "context": 180096, "input": 171904, "output": 8192 }
}
}
}
}
}
The important part is the model-name suffix. If your server alias is qwen36-aeon-rys-q4nl, the OpenAI-compatible request model names qwen36-aeon-rys-q4nl-slot0 and qwen36-aeon-rys-q4nl-slot1 pin requests to the matching server slot.
Rationale: a long-running agent benefits from a stable context lane. If a harness sends related work sometimes to slot 0 and sometimes to slot 1, the live KV/recurrent checkpoint state for that lane may not match the next request and the server may need to reprocess more of the prompt. Slot suffixes make the lane explicit. In the unpatched pinned-slot path we tested, those suffixes selected the slot but skipped RAM prompt-cache save/load. The current fork fixes that by running the RAM prompt-cache path for pinned slots too.
Verification we performed: direct A/B/A requests against the current fork showed the expected cache behavior. The repeated A request produced found better prompt, prompt cache load took ..., and restored context checkpoint in the server log instead of a full prompt reprocess. When --ctx-checkpoints 0 was present, the prompt match could still be found, but the server had no checkpoint state to restore and fell back to reprocessing. That is why the RAM-cache profile above leaves checkpoints enabled.
Full runtime and release details are in the companion record:
https://noonr48.github.io/qwen36-aeon-ik-llama/qwen36-aeon-rys-15-20/index.html
Behavior fine-tune note: a newer minimal behavioral fine-tune of this RYS line is available as SignalLatch:
https://huggingface.co/jackasda211233/Qwen3.6-27B-AEON-RYS-SignalLatch-GGUF
It keeps the same practical IQ4_NL deployment idea, but adds a small checkpoint-386 / strength-0.10 behavior merge trained around a Review -> Align -> Latch -> Repair -> Confirm loop for coding agents: scoped context review, goal and constraint alignment, waiting for concrete tool/command signals, targeted repair from evidence, and focused validation.
Qwen3.6-27B-AEON-RYS-MaxThinkCoder-IQ4_NL-ik-llama-custom-mixed.gguf
Runtime: qwen36-aeon-ik-llamaQwen3.6-27B-AEON-RYS-MaxThinkCoder-SpeedBoosted-IQ4_NL-MTP-Experimental.gguf
Runtime: qwen36-aeon-ik-llama with Qwen3.5/Qwen3.6 MTP supportQwen3.6-27B-AEON-RYS-MaxThinkCoder-BF16.ggufbf16-safetensors/The released IQ4_NL GGUF is about 70% smaller on disk than the BF16 GGUF:
54G BF16 -> 16G IQ4_NL.
This is a mixed validation snapshot, not a broad leaderboard benchmark. It compares the RYS BF16 source artifact against the released RYS IQ4_NL file.
The important read is the compression behavior: after the drop from BF16 to IQ4_NL, the mixed four-probe score changed by less than 1% relative.
| probe | RYS BF16 | released RYS IQ4_NL |
|---|---|---|
| mixed 4-probe mean | 0.7299 |
0.7244 |
math_16 |
0.8421 |
0.7897 |
eq_16 |
0.7123 |
0.7111 |
math_4 |
0.4851 |
0.5170 |
gsm8k_5 |
0.8800 |
0.8800 |
Mixed four-probe snapshot change: -0.0055 absolute, about -0.75% relative.
That is the intended claim for this release: RYS made this practical Q4_NL deployment path much more resistant to reasoning/coding degradation. The BF16 RYS model is available for exploration, but the main artifact is the compressed IQ4_NL GGUF.
Practical coding-agent test: we also used this non-finetuned RYS IQ4_NL model in an isolated Krita-like canvas-agent task: a local app with layers, brushes, transforms, opacity controls, and a local-only AI image generation stub. Under temp 0.7, 131072 context, conservative FP32 KV cache, flash-attention, graph-split setup, the first attempt hit an invalid tool/diff formatting failure before producing usable root files, but a clean retry completed with rc=0 and verifier score 1.0. FP32 KV was used here to isolate the model/task behavior from KV precision questions; separate long-context checks with FP16/default KV have reached about 160k context without showing the earlier failure pattern. This is a useful practical result: the compressed RYS path can complete the coding-agent task, while the later SignalLatch behavior fine-tune was added to improve reliability and completion discipline on top of this base.
These are context-specific internal snapshots, not a normalized cross-hardware benchmark or a public speed claim. Hardware identifiers and local machine details are intentionally omitted here; treat the rows as relative notes for the tested runtime shapes only.
The first two rows compare the custom runtime path against an internal, non-released patched upstream-style comparison path. The MTP rows are a separate graph-split runtime experiment and should not be read as a direct hardware comparison.
| check | setup | decode speed |
|---|---|---|
recommended custom ik-llama deployment |
graph split, total server ctx=409600, np=2 (~204800 token KV budget/slot), f32/f32 KV |
39.37 tok/s |
| internal non-released patched upstream-style comparison path | layer-style comparison, ctx=4096, np=1, f16 KV |
22.51 tok/s |
| no-MTP short generation | graph split, 192-token check | 48.68 tok/s |
| naive MTP draft-1 short generation | graph split, same 192-token check | 38.16 tok/s |
| adaptive MTP short generation | graph split, same 192-token check | 45.36 tok/s |
| adaptive MTP longer generation | graph split, 768-token check | 46.95 tok/s vs 48.71 tok/s no-MTP |
Note: the SpeedBoosted filename is legacy packaging wording for the custom ik_llama release context. Treat this file as experimental MTP, not as a speedup claim.
MaxThinkCoderAEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensoredblocks:15,20qwen36-aeon-ik-llamadocs/rys-layer-duplication-guidellama.cpp releaseTool-calling note: use a build of the linked ik-llama fork that includes the May 2026 duplicate tool_call filtering. It deduplicates repeated identical tool_calls server-side.
An optional MTP-capable IQ4_NL GGUF is included for testing ik_llama.cpp multi-token prediction:
Qwen3.6-27B-AEON-RYS-MaxThinkCoder-SpeedBoosted-IQ4_NL-MTP-Experimental.gguf
Use it only with an ik_llama.cpp build that has Qwen3.5/Qwen3.6 MTP support. The tested MTP path used:
-sm graph -mtp --draft-max 1 --draft-p-min 0.0 --draft-min 0 -b 128 -ub 32 -ctk f16 -ctv f16 -fa on -gr
What our tests found:
qwen35.nextn_predict_layers = 1 and the blk.69.nextn.* tensors.blk.69 matrix tensors.38.16 tok/s naive MTP and 45.36 tok/s adaptive MTP vs 48.68 tok/s non-MTP on the 192-token check.46.95 tok/s adaptive MTP vs 48.71 tok/s non-MTP.What this means in practice:
ik-llama fork has Qwen3.5/Qwen3.6 MTP support and still supports the non-MTP GGUFIQ4_NL-ik-llama-custom-mixed GGUFRecommendation: download the MTP file if you specifically want to test MTP. For normal use, use the original non-MTP IQ4_NL-ik-llama-custom-mixed GGUF.
A BF16 GGUF is also included for people who want to explore the unquantized GGUF-side artifact from the same released 15,20 RYS branch:
Qwen3.6-27B-AEON-RYS-MaxThinkCoder-BF16.gguf
Use this if you want a GGUF reference build, local conversion/testing, or to compare quantization behavior against the released IQ4_NL GGUF. It is here for exploration and downstream work. For normal inference, the IQ4_NL file is the practical target. For Transformers/LoRA/SFT workflows, use the bf16-safetensors/ folder instead.
Size note:
54G16GThe quality/compression snapshot is shown at the top of this card. The short version is that the released IQ4_NL file is the practical inference target, while the BF16 artifacts are included for reference, training workflows, and conversion work.
The original HF-format BF16 checkpoint for the released 15,20 RYS branch is included here:
Use the files in that folder for Transformers-based work such as LoRA, SFT, continued training, or conversion into another training format. Use the GGUF file in the repo root for ik-llama inference.
Loading example:
from transformers import AutoModelForCausalLM, AutoTokenizer
repo_id = "jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF"
subfolder = "bf16-safetensors"
tok = AutoTokenizer.from_pretrained(repo_id, subfolder=subfolder, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo_id,
subfolder=subfolder,
torch_dtype="bfloat16",
device_map="auto",
trust_remote_code=True,
)
RYS note for anyone rebuilding or modifying this checkpoint: the config is part of the model. The BF16 folder keeps the corrected hybrid-stack metadata for the 15,20 insert, including text_config.num_hidden_layers = 69 and a 69-entry text_config.layer_types list. Do not change the layer count without remapping layer_types to the same layer order as the tensors.
We did build and benchmark an internal standard-typed comparison artifact.
We are not releasing it as a public llama.cpp file.
Why:
ik-llama runtimellama.cpp, not clean stock mainlinellama.cpp support, this release does not include that internal comparison artifactSo the intended reading is simple:
ik-llama-targeted modelllama.cpp, this is not that releaseThis is a deliberately narrow project. The target is a practical Q4-class English-first model optimized for coding, technical reasoning, academic-style writing, and long-context work.
This release came from the AEON-derived 15,20 RYS branch. That branch was the practical release target because it quantized cleanly and held up as the best balanced candidate for this experiment.
The honest summary is that RYS was not a dramatic BF16-only upgrade in this run. The BF16-side gain over the Unsloth/official-style BF16 baseline was only around 1%. The practical value was that the RYS branch made this model much more resistant to quality loss under the IQ4_NL quantization path than the non-RYS/official-style model had been.
The source path is uncensored, so this is not a generic safe-default chat model.
The quantization was deliberately biased toward reasoning and technical work.
Heuristic calibration breakdown:
math_reasoning: 5,688 chunks, 1,706,070 chars (36.0%)code_technical: 3,518 chunks, 1,343,392 chars (28.4%)experiment_docs: 808 chunks, 224,169 chars (4.7%)writing_chat: 387 chunks, 164,097 chars (3.5%)other: 5,139 chunks, 1,249,396 chars (26.4%)Practical read:
Recommended:
Not recommended as a generic safe-default chat model. This branch came from an uncensored source path.
bf16-safetensors/model-00001-of-00011.safetensors 5.4 GBbf16-safetensors/model-00008-of-00011.safetensors 5.3 GBbf16-safetensors/model-00002-of-00011.safetensors 5.3 GBbf16-safetensors/model-00005-of-00011.safetensors 5.3 GBbf16-safetensors/model-00009-of-00011.safetensors 5.3 GBbf16-safetensors/model-00006-of-00011.safetensors 5.3 GBbf16-safetensors/model-00010-of-00011.safetensors 5.3 GBbf16-safetensors/model-00007-of-00011.safetensors 5.3 GBbf16-safetensors/model-00003-of-00011.safetensors 5.3 GBbf16-safetensors/model-00011-of-00011.safetensors 5.3 GBbf16-safetensors/model-00004-of-00011.safetensors 5.2 GBHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo jackasda211233/Qwen3.6-27B-AEON-RYS-15-20-GGUF ./model-folderAEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16