ds4-winhip — ds4 / ds4-server, Windows + HIP 7.2 build (for DeepSeek-V4.1-Flash lecore VQ GGUFs)

A working Windows/HIP port of Baekpica/ds4-dfm-rs (MIT, "the ds4.c authors"), plus the launcher, supervisor and notes used to serve DeepSeek-V4.1-Flash-1017 lecore VQ builds on a Radeon 8060S (gfx1151, HIP SDK 7.2, 128 GB unified memory).

llama.cpp, vLLM and Ollama cannot load these VQ blobs — the routed experts are vector-quantized codebook blobs (vq8x4096/1024) and only the ds4 runtime knows the format. That is what this repo is for: source, build scripts and the exact run command, so you do not have to rediscover them.

Upstream's own README (DwarfStar / ds4 feature tour) is kept at docs/README-upstream-ds4.md.

What is inside

path	what it is
`ds4.c`, `ds4.h`, `ds4_*.c` / `.inc` / `.cuh`, `crates/`, `cuda/`	the runtime, forked from Baekpica/ds4-dfm-rs at `331b8a7b`
`hip/win/build_gpu.cmd`, `build_host.cmd`, `build_repack.cmd`, `link_server.ps1`	the Windows/HIP build: device code, host code, final link
`hip/win/win_compat.c`, `win_stubs.c`, `win_stubs_cxx.cpp`, `ds4_win_compat.h`, `include/`	POSIX/vDSO shims the Windows link needs
`hip/win/run_ds41.ps1`	the launcher: vision sidecar, Engram dir, MTP/DSpark, KV disk store, sampling defaults
`serve/serve_ds41.ps1`	unattended supervisor: adopts or relaunches `ds4-server`, stall watchdog, commit-limit backoff
`serve/patch_ds4.py`, `serve/ds4_boot.sh`	the FP4 `solar_e2m1_decode` fix (CUDA/HIP path) and the Linux boot script used on the H100/H200 boxes
`ds4-server.exe`, `ds4-server/`	the built binary that produced the numbers below (Windows x64, HIP 7.2) — usable as-is
`docs/`, `HANDOFF-requant.md`	format notes: VQ layout, per-layer byte budgets, kernel constraints (LDS codebook size), re-quant recipe
`hf-card-serving.md`	the serving section as it appears on the model cards, with the measured gotchas
`bench/`	raw prefill/decode bench logs (H100/H200 and 8060S)

Excluded to keep the repo small: .git, tests/, gguf-tools/, object files, stale *.old.exe. Upstream history lives at the GitHub link above; this is a source drop of the working tree.

Build (Windows, HIP SDK 7.2)

git clone https://huggingface.co/staccs/ds4-winhip
cd ds4-winhip
.\hip\win\build_gpu.cmd            # device code
.\hip\win\build_host.cmd           # host code
cargo build -p ds4-server          # compiles, then FAILS AT LINK (no error[ lines) — expected
.\hip\win\link_server.ps1          # this produces ds4-server.exe

The link failure after cargo build is normal on this tree: the Rust crate only drives the C build, link_server.ps1 does the final link. A running server keeps its own copy of the exe, so you can rebuild while it serves.

Run

$env:PATH = "C:\Program Files\AMD\ROCm\7.2\bin;$env:PATH"
.\hip\win\run_ds41.ps1 -Gguf D:\models\ds41\DeepSeek-V4.1-Flash-1017-abliterated-pqhaz-lecore-vq11cal.gguf `
                       -Engram D:\models\ds41\deepseek-engram
# OpenAI-compatible API on http://127.0.0.1:8008/v1

The model file is the 106.8 GB lecore VQ GGUF; -Engram points at a directory holding model-00047-of-00048.safetensors and model-00048-of-00048.safetensors from deepseek-ai/DeepSeek-V4.1-Flash-1017 (~203 GB, mapped from disk, not copied). The vision sidecar (0.9 GiB) sits next to the GGUF — see staccs/ds41-sidecars.

Defaults the launcher sets (all overridable by env var):

variable	default	why
`DS4_CUDA_COPY_MODEL`	`1`	copies weights into GPU memory (~92 GiB of Windows commit; needs a commit limit above ~100 GB). `-NoCopy` host-maps instead: no commit, slower per token
`DS4_CUDA_WEIGHT_CACHE_LIMIT_GB`	`100`	if one big allocation fails, the per-range cache must hold the whole model
`DS4_SERVER_DEFAULT_TEMP` / `TOP_P` / `MIN_P`	`0.6` / `0.95` / `0.05`	the card recipe for this quant; T=1.0 untruncated turns into word salad at 14k+ prompts, greedy loops deterministically on long tool turns
`DS4_SERVER_DEFAULT_MAX_TOKENS`	`4096`	agent frameworks send no cap; without one a repetition loop runs to the context end (seen: 19.5k tokens, 22 min)
`DS4_SERVER_DEFAULT_FREQ_PENALTY`	`0.05`	curbs this quant's runaway answer loops; any penalty disables the DSpark drafter (~8.7 vs ~21 tok/s)
`DS4_SERVER_THINK_BUDGET_FILE`	`...\ds41_think_budget.txt`	think-block token budget, read per request; 0 = unlimited
`DS4_KV_DISK_DIR` / `DS4_KV_DISK_SPACE`	`<modeldir>\kvdisk` / `8G`	disk KV store: a 14k-token checkpoint is ~30 MB, restored in ~2 s instead of a ~190 s re-prefill

Measured (this tree, Radeon 8060S, gfx1151, 128 GB unified)

metric	value
load to first answer	~6 min (weight copy + ~110 s first-request kernel/graph warm-up)
prefill	135–190 tok/s (real agent prompts vs synthetic)
decode, one stream	14–17 tok/s (21 tok/s with the DSpark drafter when no repetition penalty is set)
decode, 1× H200 NVL	46–54 tok/s (vq10sh12 build)
concurrency	one request at a time; a second queues

Gotchas worth knowing before you debug

One live KV checkpoint. The server keeps the state after the last prompt and after the last reply. Any other request in between overwrites both and the next turn re-prefills everything (measured: 19.3k tokens, 138 s). While an agent works, nothing else should touch the port.
Stopping. Kill ds4-server, then wait for Windows commit to drop before relaunching, or the loader thrashes at the commit limit. serve_ds41.ps1 does this for you.
PowerShell 5.1 reads BOM-less UTF-8 .ps1 as ANSI and mangles the fullwidth bar in <｜...｜> tokens — save scripts that contain them with a BOM.
usage.prompt_tokens tells you whether a special token in your text was parsed as one id (5 with the chat template) or split into text (18).
DSpark is disabled by any repetition penalty; accept rate on this quant is 1.0–3.3 of 5 per verification round depending on prompt mix.

staccs/ds4-winhip