Qwen3.8 Flash CIRU 7900XTX + Strix

Run Qwen3.8 Flash across an RX 7900 XTX 24GB and Strix Halo with 128GB RAM, with 262,144-token target context. The measured system is a Bosgame M5 with an M.2-to-OCuLink adapter, Minisforum DEG1 dock, XFX Merc RX 7900 XTX, CachyOS and ROCm 10.0.0.

Custom runtime and release downloads · Installation · Detailed benchmarks and raw metrics

This is a hardware preview of the custom CIRU runtime. The target, MTP and PLE weights are unchanged from Qwen3.8-Flash-CIRU-STRIX-IU4; the target and MTP filenames were changed for discoverability. No additional quantization or expert pruning was introduced. A stock llama.cpp build is not qualified for this package.

Measured speeds

PP is prompt processing (prefill); TG is generated-token decoding. Rates below are tokens per second from the saved September 8, 2026 measurements.

The full-context request processed its prompt in 609.49 seconds and generated the following 128 tokens in 5.35 seconds. The Hermes workload reached a maximum sequence of 11,245 tokens while the server retained 262,144-token capacity. Its approximately 50 TG is an agent-workload result, not a filled-256K decode measurement.

Both use MTP depth 3, batch/microbatch 1,536, eight CPU threads and one inference slot. Dense target weights and the final eight expert layers are on the XTX; remaining experts, full target F16 KV and the Q8 MTP draft are on Strix. The draft uses an 8,192-token attention window; target verification retains full context. The default launcher and Hermes run use non-thinking, greedy sampling.

Full-context improvement

Decode improved 2.96× while prefill stayed essentially unchanged. Both requests returned the same 128 output tokens and the same 152 drafted / 76 accepted counts. That is bounded output agreement on this workload, not broad model-quality equivalence. No matched Strix-only 256K result is claimed. Comparison data.

HermesAgent-20: speed during real agent tasks

Prompt work took 191.82 seconds and generation 232.97 seconds in request logs. The remaining 331.85 seconds includes tool execution, agent logic, queueing, transport and harness overhead. The slowest scenarios were HA-07 at 173.3 seconds, HA-17 at 122.1 seconds and HA-02 at 91.9 seconds. Concurrent agents share the one MTP inference slot.

The live server counters report 613.56 PP / 50.51 TG. Their prompt timing accumulator differs from the per-request logs, and their generated-token count includes each request's first token, which the printed decode rate excludes. The headline uses the request logs consistently; both records are preserved. Per-request first-token latency was not recorded.

Task outcomes: the raw verifier reported 16 passes, 2 partials and 2 failures. HA-06 and HA-08 passed their artifact checks but exhausted the agent turn limit; HA-07 also exhausted its turns and failed. This is not a clean 16-pass quality qualification. HA-07 failed to parse formatted file-tool content as JSON; HA-17 produced an incorrect merged artifact; HA-11 and HA-19 received partial scores. No server runtime, MTP decreasing-position, HSA/HIP, truncation or captured client-transport error was found.

This is a local-custom non-thinking speed run with original task budgets, first trajectories, no harness fetch retries and a 65,536-token per-request output cap. Complete exported agent-result records exist for 18 scenarios; HA-14/15 used the native CLI path and retained verifier outputs without complete exported model traces.

Structured results · All 120 request timings · Scenario decisions

Memory use

Hermes memory sampling began after the first scenarios, so those peaks cover the sampled interval. Strix GTT and system RAM share the same physical memory pool; they are not additive capacity. The model files occupy about 136 GB decimal on disk, before SDK, build and cache storage.

Earlier short-context results

These are historical development profiles, not measurements of the released 256K profile. The earlier prefill profile used 12 complete expert layers on XTX, MTP3, batch/microbatch 2,048, 8,192-token capacity and prompt caching off. Both requests below generated 256 tokens with temperature 0 and seed 123 after warmup.

The 905 PP and 52.6 TG figures come from different prompts. Increasing target context required a different allocation, which is why these short-context figures must not be presented as the 256K result. Historical profile comparisons.

Historical 512–64K scaling sweep

One uncached request at each length followed by exactly 128 generated tokens. This earlier sweep used ten expert layers on XTX, 66,560-token capacity, batch/microbatch 2,048 and MTP3. It predates the retained 256K placement.

Each row is a single measurement, not a confidence interval. Prompt content changes with length, so TG reflects both context cost and speculative acceptance. Sweep CSV · Sweep JSON.

Download and run

Download these files with their directory structure and set MODEL_DIR when launching the custom runtime. For a source build on the target Linux machine, install the build tools and follow the installation guide. On Ubuntu/Debian, its setup helper can install host dependencies with --install-host-deps; the hybrid configuration has not been independently qualified on Ubuntu.

git clone --branch v0.1.0 https://github.com/ciru-ai/Qwen3.8-Flash-CIRU-7900XTX-24GB-StrixHalo-128GB-OCuLink-Linux-ROCm10.git
cd Qwen3.8-Flash-CIRU-7900XTX-24GB-StrixHalo-128GB-OCuLink-Linux-ROCm10
./scripts/hybrid/setup.sh --install-host-deps
python3 scripts/hybrid/serve.py --list-devices
MODEL_DIR=/absolute/path/to/model EGPU_DEVICE=ROCm0 STRIX_DEVICE=ROCm1 \
  python3 scripts/hybrid/serve.py

Omit --install-host-deps when build tools are already installed or use another distribution's package manager. The helper installs a private ROCm 10.0.0 SDK for gfx1100 and gfx1151. Match device IDs to the names printed by --list-devices. The API defaults to http://127.0.0.1:8080/v1, with model ID Qwen3.8-Flash-CIRU-7900XTX-Strix.

The optional retained binary requires glibc 2.43, GLIBCXX 3.4.32, OpenSSL 3, ROCm 10.0.0 and native Strix Halo CPU capabilities. On another environment, build from source. Other AMD GPU pairings require architecture and memory-placement adjustments and remain experimental. NVIDIA+Strix and Windows are unsupported by this package. Hardware support.

Evidence and credits

The GitHub repository includes the runtime, installation scripts, benchmark tables, 148 native recorder rows, failed and parked development experiments, profiler summaries and detailed Hermes metrics. Historical snapshots overlap; do not sum them as independent measurements. Full evidence index.

The original Qwen model license applies to these files. Credit to Qwen for the base models, the ggml/llama.cpp contributors and credited runtime contributors in the GitHub source history, and Ciru for the model packaging and hybrid runtime work.