This repository holds a ninfer pack of LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15. That model is DavidAU/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored with the V15 episodic memory, which needs no training; its card explains how the memory works.
The pack holds:
--spec dflash2 --draft-tokens 7;The companion config.json and tokenizer.json provide V15 memory settings and turn markers to ninfer-serve. The Python model files in the repository remain available for the reference implementation.
| File | What it is |
|---|---|
Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-FP8.ninfer |
the pack, 30.3 GB |
config.json, configuration_qwen_dense_v15.py, modeling_qwen_dense_v15.py |
the V15 configuration and core, the same as in the BF16 repository |
tokenizer.json, tokenizer_config.json, vocab.json, chat_template.jinja, *_config.json |
tokenizer, chat template, generation and media settings |
ninfer-v15-patch/ |
the V15 patch for ninfer (upstream commit 5a5c2c1), with its own README and license |
You need an NVIDIA Blackwell GPU (sm_120a). It was built and measured on one RTX PRO 6000 (96 GB).
Download the 27B FP8 pack and start the native server in Docker. The volume keeps the episodic store when the container exits.
hf download LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-ninfer-FP8 --local-dir v15-ninfer
docker run --rm --gpus all --stop-timeout 150 -p 19421:19421 \
-v "$PWD/v15-ninfer:/model:ro" -v ninfer-v15-memory:/memory \
lovewin10/ninfer-v15:5a5c2c1-cu13-v15 --model-id qwen3.8-27b-v15
The image defaults to FP8 KV, MTP with three draft tokens, vision, a 262144-token context, automatic KV capacity and eight lanes. Pass server flags after the image name to override those defaults. Set -e SPEC=none to disable speculative decoding.
Install CUDA 13.0, CMake 3.28 or newer, Ninja, and the development packages for FFmpeg (libavcodec, libavformat, libavutil, libswscale), libcurl and SQLite 3.
Build ninfer with the patch:
hf download LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-ninfer-FP8 \
--local-dir v15-ninfer
git clone https://github.com/MerkyorLynn/ninfer-rollout ninfer && cd ninfer
git checkout 5a5c2c1
git apply ../v15-ninfer/ninfer-v15-patch/ninfer_v15.patch
cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_CUDA_ARCHITECTURES=120a -DNINFER_BUILD_APPS=ON
cmake --build build --target ninfer-serve
Serve from the ninfer checkout. Keep the memory directory between runs.
PACK=../v15-ninfer/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-FP8.ninfer
./build/apps/ninfer-serve "$PACK" --v15-memory ./v15-memory --v15-model-dir ../v15-ninfer \
--host 0.0.0.0 --port 19421 --model-id qwen3.8-27b-v15 \
--vision --spec mtp --draft-tokens 3 --kv-dtype fp8 \
--max-context 262144 --kv-capacity auto --max-concurrency 8
The server offers OpenAI-compatible Chat Completions and Responses, and Anthropic Messages.
--spec mtp --draft-tokens 3 with --spec dflash2 --draft-tokens 7. Only one speculative backend runs per server.--v15-memory and --v15-model-dir, or set MEMORY=0 in Docker; the pack serves as a plain ninfer model.--kv-capacity auto includes the memory buffers in the Program's planned GPU allocation."memory": false in a request serves it without memory.--v15-sleep-after SECONDS controls the idle interval. The default is 600 seconds. --v15-no-sleep turns sleep off.--v15-namespace and --v15-storage-path select an existing store. --v15-trace logs votes.On an RTX PRO 6000 (96 GB), the packaged native server recalled a red mug in a later text-only chat using a copy of an existing FORMAT 6 store. With the same model, question and store contents, the previous build answered steel blue. The fix restores the picture alongside its recalled description within the configured memory-token budget. A separate green lunchbox was also recalled as green.
The red-mug check used the 27B FP8 pack, FP8 KV, MTP with three draft tokens, a 262144-token context and eight configured lanes, with one request at a time.
The 27B MTP reader recovered the missing second fact in a two-hop question from an existing FORMAT 6 store. With the same store and prompts, accuracy rose from 11/12 to 12/12 on six updates and six two-hop questions. On ten independently specified questions and controls, it rose from 7/10 to 8/10. The scoring used visible answers. The model still missed two items in that second set.
Single-request tests on the previous native build (77d3033, RTX PRO 6000 96 GB, FP8 KV, eight configured lanes, 262144-token context) averaged 105.4 decoded tokens/s with MTP at three draft tokens, 52.1 with DFlash2 at seven and 53.6 with speculative decoding disabled. These are means over three streamed 400-token replies, excluding first-token latency. DFlash2 accepted 3.05% of its proposed tokens on this fine-tune. The matched Coder390 pack averaged 106.6 tokens/s with DFlash2 on the same prompts.
In an end-to-end sleep run on that native build, the update score improved from 5/6 to 6/6 with either backend. The historical exact-string two-hop score changed from 4/6 to 4/6 with MTP and from 4/6 to 5/6 with DFlash2. Each run also answered “星期一” correctly where the scorer required the synonymous “週一”.
One RTX PRO 6000 Blackwell (96 GB) under Docker Desktop, 2026-11-17. The runs used this pack, the patch at commit dc51f5e, FP8 KV and the Python memory service (262144-token context, 8 lanes, --kv-capacity auto), with a new memory store for each run.
| MTP (default) | DFlash2 | |
|---|---|---|
| ready after | 121 s | 127 s |
KV cache (auto) |
1,648,128 tokens | 1,496,960 tokens |
| facts recalled | 6 of 6 | 6 of 6 |
| unrelated questions answered | 4 of 4 | 4 of 4 |
| eight chats telling and asking at once | 7 of 8 (one answered with a tool call instead of the number); 8 of 8 in a second run of this test | 8 of 8 |
| a fact inside a 9,442-token prompt | found | found |
| an image | named | named |
| server after an aborted request | alive | alive |
| one 320-token reply, first token included | 4.0 to 7.2 s (44 to 80 tokens/s) | 7.0 to 10.0 s (32 to 46 tokens/s) |
| eight 200-token replies at once, in all | 69 to 112 tokens/s | 44 to 112 tokens/s |
| drafted tokens accepted | 42 to 76 % | 2.3 to 4.3 % |
Two more runs, for comparison:
Use MTP. With this model DFlash2's draft is rarely accepted, so DFlash2 decodes more slowly than MTP, with memory or without.
The draft comes from nerkyor/Qwen3.8-27B-Coder390-…-MTP-DFlash2, folder FP8/DFlash2-FP8 (Apache-2.0). It is 5 layers and 1.92 B parameters.
That folder stores the draft in Qwen's block-FP8 layout (FP8 weights with 128×128 scales), which ninfer's converter does not read. So before the conversion it was expanded to BF16:
In the pack, the draft's projections are stored as q8_g32_fp16, as ninfer's own recipes do.
The draft was trained for another Qwen3.8-27B fine-tune (Coder390), not for this model. It drafts for this model too; how many of its tokens are accepted is in the measurements above.
DFlash2's draft was trained for Coder390. Its acceptance rate and the throughput of eight simultaneous requests in the earlier Python-service build are reported above. The engine runs on one Blackwell GPU (sm_120a). The base model card describes the limits of V15 episodic memory.
From the BF16 repository, with the patched ninfer checkout (tools/convert, commit dc51f5e of the V15 branch):
python3 -m tools.convert --model <the BF16 V15 directory> \
--recipe tools/convert/recipes/qwen_v15_fp8.py \
--source dflash2=<the draft expanded to BF16> \
--components text,vision,mtp,dflash2 --proposal \
--name Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-FP8 \
--out Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15-FP8.ninfer
The recipe converts the text projections to fp8_e4m3fn_row_bf16 (row maxabs, from BF16). The GDN a/b projections and the norms stay BF16. Vision, MTP and DFlash2 follow ninfer's official recipe choices.
LTM-Lab/Qwen3.8-27B-TWIN-TURBO-Fable-Cold-Fusion-709-ULTRA-HERETIC-Uncensored-v15