A single-file GGUF conversion of Maxwell-V1-Flash by the Maxwell Team, adapted for FrankenMoE-CUDA. The conversion uses the upstream weight snapshot recorded at revision ca536c4380e0600d6aebdb808d2ce65ec724161b. On an RTX 4070 12 GB, the tested runtime completed a 32,256 input + 128 output request using its experimental full-history Dense GQA mode.
| File | Bytes | SHA-256 |
|---|---|---|
Maxwell-V1-Flash-Q4_0.gguf |
21,451,905,888 | c2542c20bff7a227f010515c733135bcb0bc10e0366f36abed28e4e8f934d4bd |
The archive is 21.45 GB (19.98 GiB). It contains 573 Q4_0, 93 F16, and 495 F32 tensors. The GGUF v3 tensor table was checked for duplicate names, overlapping ranges, misalignment, and truncation. This repository distributes the converted GGUF.
The original authors describe a text-only mixture-of-experts model with 35B total and 3B active parameters per token, excluding embeddings. Its 45-layer backbone uses grouped-query attention, routed and shared experts, and manifold-constrained hyper-connections. Maxwell Block Sparse Attention (MBSA) combines fixed prefix and local blocks with remote blocks selected by a learned indexer. See the original model card for training and architecture details.
FrankenMoE-CUDA supplies the loader and execution graph for the adapted maxwell_moe architecture. The GGUF includes converted indexer tensors and a Jinja chat template. The tested server build reports b1-1b44e307. It uses the Maxwell reasoning integration from EXP942. Structured reasoning is emitted as message.reasoning_content or delta.reasoning_content, which the bundled web interface displays in a collapsible panel.
MBSA is the runtime default. The tested EXP943 build also exposes LLAMA_MAXWELL_DENSE_GQA=1 as an experimental mode. It uses ordinary causal attention over the full history, omits the MBSA indexer weights and KV cache, and can produce different results. Reproducing this mode requires a FrankenMoE-CUDA build with the EXP943 extension. Stock llama.cpp compatibility has not been verified.
Set RUN to the compatible EXP943 runtime directory and start from the directory containing the GGUF. The test used the following runtime settings and an additional allocation probe. The probe is used for measurement only. The command below sets the model's display name to Maxwell.
RUN=/path/to/maxwell-enabled-frankenmoe/run
MODEL="$PWD/Maxwell-V1-Flash-Q4_0.gguf"
env -u GGML_CUDA_REGISTER_HOST -u GGML_CUDA_NO_PINNED -u GGML_KV_LAZY_VMM \
LLAMA_MAXWELL_DENSE_GQA=1 LD_LIBRARY_PATH="$RUN" \
"$RUN/llama-server" -m "$MODEL" \
-c 32768 -ngl all -fit off --cpu-moe --moe-cache-mib 3800 \
-fa on -ctk q4_0 -ctv q4_0 --load-mode mmap --alias Maxwell \
-b 2048 -ub 2048 -t 8 -tb 16 -np 1 \
--cpu-mask 0f0f --cpu-mask-batch ffff --cpu-strict 1 --cpu-strict-batch 1 \
--poll 0 --cache-ram 0 --ctx-checkpoints 32 --spec-type none --jinja \
--host 127.0.0.1 --port 9101
Use http://127.0.0.1:9101/ for the web UI. The command uses memory-mapped CPU experts, Flash Attention, a bounded GPU expert cache, and Q4_0 K/V. To select the original MBSA path on the same build, set LLAMA_MAXWELL_DENSE_GQA=0.
AMD Ryzen 9 5950X and approximately 60.7 GiB of OS-visible system RAM. Each row below is one run with a 3800 MiB GPU MoE cache budget, batch/ubatch 2048/2048, and threads 8/16. The MBSA and Dense runs used identical requests within each context pair. Decode is single-response throughput. RAM is process peak RSS. VRAM is the highest observed whole-card sample and includes desktop allocations.
| Context | Attention | Input / output | Prefill (tok/s) | Decode (tok/s) | Peak RAM (MiB) | Observed VRAM (MiB) | GPU graph (MiB) |
|---|---|---|---|---|---|---|---|
| 8,192 | MBSA | 7,936 / 128 | 883.20 | 20.24 | 21,319.96 | 9,940.00 | 848.14 |
| 8,192 | Dense | 7,936 / 128 | 1,084.25 | 27.48 | 21,177.17 | 9,598.00 | 600.14 |
| 32,768 | MBSA | OOM after 16,384 input tokens | — | — | 21,419.36 | 11,786.50 | 1,796.58 before OOM |
| 32,768 | Dense | 32,256 / 128 | 1,043.80 | 23.92 | 21,279.36 | 10,470.06 | 696.14 |
The 8K runs returned the same 128 token IDs and the same text, with different log probabilities. Both modes completed only the 8K pair. The 32K MBSA trial failed when its CUDA graph buffer grew, so there is no paired 32K speed or quality comparison.
Short profiles. A separate FrankenMoE build with EXP906 host registration and a 4992 MiB expert cache recorded medians of 1767.72 prefill and 62.02 decode tok/s over three warmed Dense requests of 2048/128 at context 4096. Its recorded RSS and whole-card VRAM were 21,147 and 11,671 MiB. The 4992 MiB cache was also tried interactively with EXP943. The documented 32K throughput figures use 3800 MiB.
Chat. Russian chat generated a complete streamed answer with a normal stop and separate reasoning content. An earlier single chat request of 13 input / 308 output at context 32768 and cache 3800 MiB measured 28.44 prefill and 22.87 decode tok/s, with 21,202 MiB RSS and 9,744 MiB recorded whole-card VRAM. The final chat fix was verified with the live UI, including requests with tool declarations, and the user confirmed the Reasoning block collapsed after completion. Separate tests checked an ordinary JSON response and a reasoning block cut short by a token limit.
The original weights and architecture are by the Maxwell Team. This release changes the weight representation, tensor mapping, and metadata for FrankenMoE-CUDA. It is distributed under Apache License 2.0. See the included LICENSE and the upstream model card for licensing and attribution.
maxwell-model/Maxwell-V1-Flash