Qwen3.8-Flash-Next · Splash · UD-Q4_K_XL with a Q8_0 PLE table

Same model, listed at its in-memory size: this body is the Q4-PLE8 folder of the family repository (--model Vardaan97/Qwen3.8-Flash-Next-Splash-GGUF:Q4-PLE8). There the Hub's hardware panel shows it at 82.5 GB, which fits a 128 GB Mac, and a Q4-PLE4 folder offers the 4-bit table.

Our recommended default for Splash. The body is unsloth's UD-Q4_K_XL, unchanged. The only difference from unsloth's file is the per-layer embedding (PLE) table: here it is a standard Q8_0 conversion of the official BF16 table, in place of unsloth's IQ4_NL table. Nothing in the body was re-quantized; this repository is an assembly of existing files for Splash 1.4.0. Base model: Qwen/Qwen3.8-Flash-Next. The family repository carries the builds that fit the Hub's 128 GB line; the other repositories of this release are linked below.

Quick start

Splash 1.4.0 or newer, a 128 GB Apple-silicon Mac, macOS 26.4 or later:

brew install incoai/tap/splash
splash serve --model Vardaan97/Qwen3.8-Flash-Next-UD-Q4_K_XL-PLE8-GGUF:UD-Q4_K_XL-PLE8

Splash resolves :UD-Q4_K_XL-PLE8 to the four split parts in that folder. The first start downloads them (136.9 GB) and the vision projector (0.91 GB) from this repository. Splash then adds the DFlash2 draft it releases for Qwen3.8-Flash-Next, incoai/Qwen3.8-Flash-Next-DFlash2 (1.21 GB), by itself. It picks the draft from the model family it reads in the GGUF, not from the repository name, so you don't pass --draft-model; this repository ships no draft. Later starts reuse the downloads.

• Vision is on by default, through the BF16 projector at the repository root. Add --language-only to serve text only.
• Memory and context are planned automatically from the GPU working set. --ple-memory defaults to auto, which streams the PLE table from the SSD when memory can't hold it. --max-memory caps Splash to leave room for other applications.
• Wired limit. All our measurements used iogpu.wired_limit_mb=110592 on an M5 Max 128 GB (sudo sysctl iogpu.wired_limit_mb=110592, resets on reboot). A Mac at the default limit is untested.

At a glance

Hub GGUF panel. The panel groups split files by the quant tag in their filenames and shows the total download size, including the PLE table, which streams from the SSD and is not held in memory. It reads Qwen3.8-Flash-Next-UD-Q4_K_XL-PLE8-00001-of-00004.gguf as UD-Q4_K_XL (PLE8 is not a recognized tag), so expect one UD-Q4_K_XL entry of about 137 GB, of which 54.4 GB is the streamed table. The family repository lists its builds at their in-memory size, which fit a 128 GB Mac. The memory actually needed is the resident body in the memory table.

Which one should I pick

Measured on the earlier Splash build with native MTP (2026-10-19), not on Splash 1.4.0. The KLD figures don't depend on the runtime.

All nine variants are the same model. They trade disk, memory, speed and weight error. On agent tasks the four tested PLE8 variants are indistinguishable: 7–8/16 strict and 8–9/16 with the lenient FinQA re-grade, a one-task spread that is within noise for 16 single-trial tasks. PLE5 and PLE4 were not task-tested.

Our default: UD-Q4_K_XL-PLE8. It had the fastest decode, the lowest memory and the quickest load, and its agent-task scores are within noise of the others. We keep Quality-PLE8 as the quality reference, since its measured weight error is lowest.

The PLE table streams from the SSD (a 2 GiB cache in these runs), so the table choice changes disk size, not the resident body; PLE5/PLE4 runtime memory is in the release E2E table. Sweep decode rates come from each model's own agent workload, so they are descriptive rather than a controlled A/B. On an older build, UD was also faster on prose (122.8 vs 114.4–115.3 tok/s) but not on the 2K greedy city test (157.7 vs 158.3–161.9).

These custom variants were designed around 128 GB unified memory and PLE streaming. A full 262,144-token base context is a model capability, not an end-to-end validation for each quant.

Call it

Splash serves an OpenAI-compatible API on 127.0.0.1:8000. This is the release E2E's tool-call check, which expects two parsed get_weather calls:

curl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
  "model": "Vardaan97/Qwen3.8-Flash-Next-UD-Q4_K_XL-PLE8-GGUF:UD-Q4_K_XL-PLE8",
  "messages": [{"role": "user", "content": "Call get_weather exactly twice: once for Paris and once for Tokyo, both in celsius. Do not answer in prose."}],
  "tools": [{"type": "function", "function": {"name": "get_weather", "description": "Read weather for one city.",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}, "unit": {"type": "string", "enum": ["celsius"]}},
                   "required": ["city", "unit"], "additionalProperties": false}}}],
  "tool_choice": "auto", "parallel_tool_calls": true,
  "reasoning_effort": "high", "max_tokens": 4096
}'

Optional settings. None of these is required on 1.4.0. They reproduce the conditions of our earlier measurements.

• --max-context 176K (180,224 tokens) is the server limit we tested; the agent harness kept each request within 131,072 tokens. Without it, Splash plans the context itself. The native 262,144-token window is not validated for these files.
• --ple-memory 2G gives the 2 GiB PLE row cache our runs used; the default auto sizes the cache itself.
• --disable-ane, --idle-release off and --max-cache-disk 0 (SSD cache off) match the benchmarks. With the default --idle-release 10m, the weights are freed after 10 idle minutes and reloaded by the next request (loading took 12.9 s in the sweep).
• int8 KV and fp32 state are Splash's defaults and what the measurements used.
• --served-model-name flash-next adds a short model name for API calls.
• Reasoning: the benchmark requested "reasoning_effort": "high" through a custom chat template with a distinct high level. The GGUFs' own template supports low, medium and xhigh, so Splash renders high as xhigh. "none" turns reasoning off; --default-reasoning-effort sets a server default. Sweep sampling was temperature 1.0, top_p 0.95, top_k 20, min_p 0.

Offline. After one online start has installed the model, add --offline (or set HF_HUB_OFFLINE=1) to start it without contacting the Hub. A 40-hex --revision keeps Splash on one repository commit. To pre-fetch the files only:

hf download Vardaan97/Qwen3.8-Flash-Next-UD-Q4_K_XL-PLE8-GGUF --include "UD-Q4_K_XL-PLE8/*.gguf" "mmproj-*"

Memory and storage

Measured on the earlier Splash build with native MTP (2026-10-19), not on Splash 1.4.0.

Text-only (no projector), 176K context, int8 KV, fp32 state, 2 GiB PLE cache and a 110,592 MB wired limit. Download adds the 0.908 GB projector to the target shards; the DFlash2 draft (1.21 GB) comes from its own repository, and imatrix_unsloth.gguf_file (0.580 GB) is provenance only, which Splash does not download. Resident body is the target tensor bytes without the streamed table, from the GGUF headers. Peak after load is Splash's Metal allocation high-water mark after loading plus a 64-token warmup; sweep peak is the highest over the roughly one-hour agent run. Neither is process RSS. Whole-machine memory in use peaked at 124.2–124.6 GiB during the four PLE8 sweep runs, including the sandboxed harness, and swap never grew above its starting baseline (about 1.0 GiB). Weights loaded is from the server log; the OS file cache was not controlled. The release E2E table also loads the projector.

Benchmarks (M5 Max 128 GB)

Splash 1.4.0 install check

Passed, 2026-10-22. This ran on stock Splash 1.4.0 (1482f6bb) on an M5 Max 128 GB with the wired limit at 110,592 MB.

• How it ran: exactly as a user runs it, splash serve --model REPO:FOLDER, with Splash's defaults (vision on, automatic memory and context) plus --max-cache-disk 0.
• What it downloaded: the Hugging Face cache already held these exact files, matched by SHA-256, so the large shards weren't downloaded again. On the first start, Splash fetched the 1.21 GB DFlash2 draft from its own repository by itself, and nothing else.
• Context: its automatic plan held the model's full 262,144-token window.

The checks:

• 17 × 23 answered 391.
• A code planted in an ~8K-token prompt was retrieved.
• Two get_weather calls (Paris and Tokyo, celsius) came back as parsed tool calls.
• A synthetic receipt image was read into a JSON schema with all 6 fields correct.

The time to Ready includes loading the weights; the first start also includes the draft download. Peak memory is Splash's Metal allocation high-water mark after the checks, not process RSS.

The family repository's UD-Q4_K_XL-PLE8 folder holds the same bytes as this repository's (identical SHA-256).

DFlash2 speed before 1.4.0, 2026-10-20

A DFlash2 speed run of UD-Q4_K_XL with a Q8_0 table, on the upstream main branch just before 1.4.0 (text only, 176K context), is reported on the UD-Q5_K_XL card: 134.0 tok/s for one stream, 174.7 tok/s in total with 4 concurrent streams. It is not a 1.4.0 result.

Scored agent sweep, 2026-10-18/19

Measured on the earlier Splash build with native MTP (2026-10-19), not on Splash 1.4.0.

Method. Nine local Flash-Next builds ran one at a time through the same sandboxed agent harness on the M5 Max. Each faced 16 deterministically graded tasks from public benchmarks (one trial each), four repeat attempts (F1 and H3 twice more) and one separately judged finance write-up (F3). All used the same Splash build with native MTP, text-only serving, reasoning high, temperature 1.0 / top_p 0.95 / top_k 20, identical task order and seeds, a fresh server per model, a 131,072-token working context, per-task time caps set from one UD pilot, and a two-hour window per model. The four rows below are the released PLE8 files: the same shard bytes, except that the custom bodies' tensor-free part 1 differs only by a shortened importance-matrix path. No score is transferred to PLE5 or PLE4.

Per-task primary outcomes: ✓ pass · ✗ wrong or invalid · ⏱ hit the time cap · ✓ˡ fails the strict final-line rule, passes the lenient re-grade.

Tasks: M1 AIME 2026 and M2 HMMT Feb 2026 (math, no tools); C1/C2 BigCodeBench-Hard and T1 Terminal-Bench (coding); F1 FinQA and F2 FinanceMath (finance); H1/H2 LongHealth at about 96K/64K input tokens, H3/H4 MedXpertQA (health); I1/I2 IFBench (instructions); B1–B3 BFCL v4 (tools).

Noise. n = 16 single-trial tasks. Eleven tasks had the same outcome for all nine sweep models: H1, H2, H4, I2 and B3 always passed; M1, H3, F2, B1, B2 and T1 always failed. Among these four variants, strict scores differ only on C2 (Quality passed) and on F1's answer format (Balanced passed strict); with the lenient F1 re-grade only C2 differs. A one-task difference is within noise, and no paired uncertainty analysis was completed. The whole nine-model sweep spanned 5–9/16 strict. These are results of one harness, not general capability rankings.

Notes. The lenient re-grade changes only F1's format rule: it accepts source-reference lines after the numeric FINAL ANSWER: line. It is a post-hoc sensitivity check; strict stays the primary score. F3, an adapted FAB financial-variance write-up, is judged separately and is not in /16. All four scored 3/5 and failed the combined gate: the Sonnet 5.5 judge passed all five criteria, but the deterministic signed-figure gate rejected two. UD's F3 attempt also used the whole 131,072-token working context. B1/B2 failed for every model, and the harness showed tool results to the model as render errors in those tasks, so the tools column says little about tool-use ability. T1 and most M1 attempts hit their caps. Wall includes load, warmup, solving, archival and cleanup (Balanced's includes substantial teardown) and excludes cooldowns. Decode is Σ(output × logged tok/s)/Σoutput over completed server requests with ≥500 output tokens (50/52/53/61 requests), excluding warmup and cancelled requests. Metal peak is Splash's allocation high-water mark, not process RSS or whole-machine memory. MTP counters are proposal acceptance, not speedup.

Neutral KLD and perplexity

Same BF16 reference and neutral text, 87 × 4096-token chunks, 178089 scored tokens; base PPL 4.045181. Chunk SE is used for KLD uncertainty; PPL ratios are the measured tool summary. These are distribution comparisons, not task scores, and they don't depend on the runtime.

Balanced-PLE5 adds 0.000457 KLD (+3.44%) versus PLE8 while saving 16.000 GB. Quality-PLE5 (0.013128) and Max-PLE5 (0.012848) were measured on 2026-10-19 with the same scorer. Quality and Max PLE8 are effectively tied (0.6 paired SE). Full method, raw baseline logs and paired comparisons remain in KLD evidence.

1M context

Measured on an earlier Splash YaRN build without MTP, not on Splash 1.4.0. Splash 1.4.0 turns on YaRN when --max-context goes past the 256K native window, up to 1,010,000 tokens; this test has not been rerun on it.

Original unsloth UD-Q4_K_XL with its stock IQ4_NL table, factor 4, max context 1010000, int8 KV, fp32 state, 2 GiB PLE cache: at 1000000 prompt tokens it retrieved 8/8 planted facts and returned NOT_FOUND for the absent fact, with 614.3 s cold TTFT, 1628 prefill tok/s, 28.5 decode tok/s and 91.9 GiB Splash peak, no new swap. Retrieval was also 8/8 at 262K and 520K. The sum-of-amounts test failed at 520K and 1M. This does not validate arbitrary long-context reasoning, other quants, or this release's Q8-table UD assembly at 1M. Reproduction.

Release E2E smoke, 2026-10-19

Measured on the earlier Splash build with native MTP (2026-10-19), not on Splash 1.4.0.

The E2E decode figure is a single cold 300-token sample with reasoning off, so it reads lower than the agent-sweep decode rates in the glance box. The two numbers measure different things.

Completed 2026-10-19T07:14:16Z on the exact upload-source assembly, the stock MTP head of that build and the shipped projector; int8 KV, fp32 state, 2 GiB PLE cache. Prose timing disables reasoning and EOS for exactly 300 completion tokens; the other checks request high reasoning. Load timing is process launch to ready, with the existing OS file-cache state (not a certified cold load). The six-prompt smoke battery covers text/math, executed code assertion, two tool calls, an 8K-token needle, exact instruction format and a 300-token prose decode. Projector presence/header checks do not establish image/PDF correctness. These checks do not replace the scored sweep or validate streaming, Responses API, full-context correctness or broader hardware.

Variants

One repository per expert body, with PLE8/PLE5 folders together and PLE4 where offered; the family repository carries the 4-bit and trimmed 5-bit builds that fit the Hub's 128 GB line. All sizes are decimal GB; target downloads exclude the projector and importance matrix. These are mixed quantizations, not uniform Q8/Q6/Q5/Q4 files.

File labels. Shards are labelled for the Hub: UD-Q4_K_XL for the UD body, and Q5_K_XL for the Quality, Balanced and Max bodies. Q5_K_XL is the nearest standard class for these mixes: expert gate/up Q5_K/Q6_K, expert down Q5_1/Q8_0, attention Q8_0. That works out to about 6.1–6.3 bits per weight over the body, excluding the streamed PLE table (computed from body size and parameter count). The exact per-tensor mix is listed on each model card. The table precision is in the folder name: PLE8 = Q8_0, PLE5 = Q5_1, PLE4 = IQ4_NL.

‡ Measured in the release E2E (2026-10-19, earlier build with native MTP) on the exact public files: Splash's memory plan at load with --max-context 176K, vision projector loaded, 2 GiB PLE cache. Other rows come from the earlier v2 protocol.

PLE5 uses the same body as its PLE8 sibling and a Q5_1 table: 38.4 GB, 6.0 bpw (32 values per 24-byte block). Balanced-PLE5 KLD and top-1 are measured on the same neutral protocol as its PLE8/PLE4 baselines; its chunk SE is reconstructed from 87 running chunk means. Quality-PLE5 (0.013128, +4.1% vs Quality-PLE8) and Max-PLE5 (0.012848, +2.5% vs Max-PLE8) were measured on 2026-10-19 with the same scorer and reference, on the exact release files. For these two rows the SE shown is the scorer's own; the other rows use the chunk-SE reconstruction. The model treats table error as independent of body error; these estimates are not task results. Native resident tensor bytes exclude the table and are unchanged.

Native resident bytes sum target tensor bytes with the PLE table excluded. Splash target bytes and KV capacity come from runtime-format status on earlier builds; Splash 1.4.0 plans memory itself, so its figures can differ. KV capacity is shared across lanes; it is not a tested single-request context length. PLE8 runtime status here is from the SVG stack, with int8 KV, fp32 state, a 2 GiB PLE cache and a 110,592 MB wired limit; the SVG requests used 32,768 context. † Historical status from the earlier release build. These are separate cohorts, not matched memory comparisons. Evidence.

Ultra256-PLE8: a historical Q8-focused larger-memory archive is recorded, but no validated public release is offered. Max-PLE4: not released. Lite: not released (missing body shards). Quality-H6 is an experimental control, outside this release family.

Exact UD tensor mix

This retains unsloth’s per-layer mixture. Only per_layer_token_embd.weight changes from IQ4_NL to Q8_0. In particular, this is not a uniform Q4_K model. The exact-name map and inventory cover all 1,224 tensors.

UD table and runtime evidence

Measured on older Splash builds with native MTP (the earlier STACK and the #447-based tests), before the 2026-10-19 sweep build, not on Splash 1.4.0. All figures are single-stream observations on a 128 GB M5 Max (40-core GPU), int8 KV, fp32 state, a 2 GiB streamed PLE row cache and GDN checkpoint grid 16,384. Decimal target download is 136.935 GB, excluding the 0.908 GB projector and 0.580 GB importance matrix. Native body tensor bytes excluding PLE are 82,523,491,840; this is not full runtime RAM. Q8-table phase-3 peak allocation was 89.28 GiB; runtime repacking, KV and caches are additional to native tensor bytes.

The UD-cells paired city change versus original IQ4_NL was +3.04% throughput and +4.83% accepted/cycle (5.113 → 5.360). That table experiment had incomplete/tainted prose; it does not establish a general prose speedup. Phase 1b Q8 prose aggregate was 112.092 tok/s, but both reference brackets were disk-tainted; Q8 cold TTFT cells were also disk-tainted. Phase 3 Q8 was clean, one launch; reference brackets included disk/swap taints. Q8 output identity differs as expected when table precision changes. Table-location results were city −0.10% and cold 32K TTFT +1.62% on NVMe versus internal for Q8; these are marginal single-protocol observations, not a storage guarantee.

The #448 measurements below used the same unsloth UD body with its original IQ4_NL table, not this Q8-table assembly. No #448 Q8-table run is claimed.

The last row is an unbracketed no-drafting proxy, not a genuine matched same-tree MTP-off experiment. Native MTP averages 114.7 tok/s in the two prose launches; one was disk-tainted. City was cooling-gated; prose/TTFT were warm, with a disk-tainted 4K cell. Do not treat the rows as a controlled general speedup.

Depth prompts were 131,072 and 174,080 tokens followed by 256 greedy output tokens; requested context was 176K. One launch per arm, no cooling gate. The #448 128K cell and references carried disk taints; the final reference also had swap-ins. The ledger answer lay outside the generated prompts, so this is a timing probe, not a long-context retrieval/quality pass. #448 outputs matched the no-draft proxy on 30/72 prompts, or 72/72 when verify-row composition matched; the batched kernels depend on verify-row composition.

Quality evidence and estimate

Q8-table KLD: 0.021037 ± 0.000163 (scorer SE), same top-1 94.807%, PPL ratio 1.004707 (measured 2026-10-19). Same scorer and reference as every other row. To fit the 128 GB Mac, the measurement used a 5-part re-split of the exact release files in which the PLE table sits in its own part. All 1,224 tensors were verified byte-identical, so the weights are unchanged. That is 5.2 % lower than the original IQ4_NL-table baseline (0.022195), and about 1.6× the custom PLE8 bodies' KLD. See method.

Original UD scored coding 278.68/320 (13 tasks, one run, thinking off, permission-rejection confounds) and blind SVG 3.60/5, 8/9 valid (3 prompts × 3 seeds, thinking on, one blind rater, invalid/capped outputs score 1). These used the original IQ4_NL table and earlier STACK. They are context for unsloth’s body, not reruns of this Q8-table assembly. Small sample differences do not establish a task ranking.

Assembly and checksums

Parts 1, 3 and 4 are byte-identical to unsloth’s pinned release. Part 2 keeps every body tensor byte-identical (296 tensors) and swaps only per_layer_token_embd.weight: unsloth's IQ4_NL table (4-bit, 28.80 GB) becomes a standard Q8_0 table (54.40 GB). Historical splice verification matched source-body and donor-table payload hashes. Part-2 full SHA-256: 7d8b4597b27356c8645fd41078979b382b1cd1233cd388fda0326f5b28038b43. No payload is rehashed merely to prepare this card. Exact four filenames, sizes and SHA-256; assembly recipe. The BF16 projector and importance matrix remain unchanged.

Repository layout

README.md  LICENSE  VARIANTS.md  MEASUREMENTS.md  UD-EVIDENCE.md  TENSOR-INVENTORY.md
UD-Q4_K_XL-PLE8/  Qwen3.8-Flash-Next-UD-Q4_K_XL-PLE8-00001-of-00004.gguf … 00004-of-00004.gguf
mmproj-Qwen3.8-Flash-Next-BF16.gguf
imatrix_unsloth.gguf_file
kld/  KLD-TABLE.md  METHOD.md  PAIRED-RELEASE.md  logs

Limitations and known issues

• Runtime. The agent sweep, speed, memory and E2E results come from earlier Splash builds with native MTP. Splash 1.4.0 decodes with the DFlash2 draft and plans memory itself, so its speed, memory and output with these files can differ; they have not been measured yet.
• Hub install path. Installing from the Hub with stock Splash 1.4.0 passed (see Splash 1.4.0 install check).
• Wired limit and hardware. Every measurement is from one M5 Max 128 GB with iogpu.wired_limit_mb=110592. At the default limit Splash plans a smaller budget and may hold less context or stream more of the model from the SSD; that is untested, as are other Macs and memory sizes.
• Reasoning effort. With the GGUFs' own chat template, high renders as xhigh; the benchmark's distinct high level came from a custom template.
• Answer format. In the FinQA task, three of these four PLE8 variants (and six of the nine sweep models) put source references after a required FINAL ANSWER: line. If you parse the last line, say so in the prompt or post-process.
• Context. Long tool-heavy sessions can fill the window: UD's F3 attempt used the whole 131,072-token working context. Beyond 176K is untested with these files; the 1M result used original UD on an earlier build.
• Reproducibility. Batched kernels make outputs depend on batch composition and cache state, so even greedy outputs are not bitwise reproducible across those conditions.
• Not validated beyond the text-only agent sweep and the six-prompt release smoke: image/PDF correctness (vision is on by default, but our checks only loaded the projector), streaming and Responses API details, cancellation and cache behaviour, full 262,144-token correctness, and long-context or code-text KLD.
• Small samples. 16 single-trial agent tasks, a common BFCL harness defect, one blind SVG rater, and permission-rejection confounds in the earlier coding runs.
• UD+Q8 gaps. This Q8-table assembly has no SVG/coding rerun; its results on the sweep build are the agent sweep, the release E2E and its KLD. Original-table #448 results do not validate it. Q8 prose comparisons and TTFT controls on the earlier builds carry the taints described above, and the 128K/170K timing is not a retrieval-quality pass.

Changelog

v3 → v4 (Splash 1.4.0). The model files are unchanged. The MTP head is removed and this card is rewritten.

• Instructions now use the released Splash 1.4.0 (brew install incoai/tap/splash, then splash serve --model REPO:FOLDER). The source build, PR checkout and SPLASH_MTP_* settings are gone.
• Speculative decoding is DFlash2: Splash downloads the released draft by itself. The native MTP head (mtp/) is removed from the repository and the instructions; MTP-era measurements stay, labelled as such.
• Vision is on by default; memory, context and PLE streaming are planned automatically. The earlier benchmark flags are listed as optional.
• Download sizes no longer include the 2.787 GB MTP head. Added the Hub-panel note, family-repository links, a pointer to the pre-1.4.0 DFlash2 speed run and a placeholder for the 1.4.0 install check.

Earlier. v3 moved to a public repository with byte-identical model files and added the scored agent sweep, the release E2E smoke and the 1M-context note.

Credits and licence

unsloth created the UD-Q4_K_XL body quantization and supplied the BF16 GGUF conversion, importance matrix and BF16 projector; the PLE table is a standard Q8_0 conversion of the official BF16 table. Inherited unsloth provenance metadata stays intact. Qwen supplied the base model; these derivatives remain under Qwen Community License 1.0. LICENSE is the same as the base; this assembly grants no broader rights. The Splash contributors made the runtime and the DFlash2 draft. The neutral BF16 reference logits and evaluation text come from the public datasets cited in kld/METHOD.md.