加载中…
加载中…
Branding note: This model family is now APUS Reflex. Some weight filenames, script/module names, served-model aliases, and the separate evaluation dataset ID retain their historical identifiers for download and runtime compatibility; these do not indicate different model versions.
English | 中文 · Source model · Collection · GGUF collection · MLX collection · MLX (Apple Silicon)
GGUF conversions of APUS-Reflex-4B (revision 65797c526c27) for Ollama, llama.cpp and LM Studio on Linux, Windows and macOS (Metal).
APUS Reflex is a decision model: each request supplies a state, an instruction and 2–16 candidates; the model answers with one candidate label (A–P) and the application reads the distribution over those labels. It is not a chat model.
Every file was scored on the Frozen80 panel with identical prompt tokens and compared with the HF BF16 release run by its own runtime (full depth, 66/80 · 82.50%).
| File | Size | Frozen80 | Decisions = HF BF16 | Max Δp vs HF BF16 |
|---|---|---|---|---|
| Q8_0 | 4.2 GiB | 67/80 · 83.75% | 79/80 | 0.1288 |
| Q4_K_M | 2.5 GiB | 67/80 · 83.75% | 77/80 | 0.8879 |
| BF16 | 7.8 GiB | 66/80 · 82.50% | 80/80 | 0.0780 |
| Q8_0 · Apple M5 Metal | 4.2 GiB | 66/80 · 82.50% | 80/80 | 0.1319 |
Rows are llama-server b11118 on an NVIDIA RTX PRO 6000 (CUDA) unless marked Apple M5 (Metal). Q8_0 is the recommended default; Q4_K_M uses an importance matrix. Quantized scores are measured separately and do not inherit the BF16 result. Frozen80 is a reused development panel, not a blind benchmark. Per-file details: evaluation/. Per-question rows (candidate probabilities, choice, correctness; join with Frozen80 by panel_index): evaluation/per-question/.
ollama run hf.co/apus-ailab/APUS-Reflex-4B-GGUF:Q8_0 --think=false
Always disable thinking. Ollama renders this architecture with its built-in Qwen3.5 renderer (the Modelfile TEMPLATE is not applied), and that renderer opens a thinking block unless told otherwise, which breaks the trained no-thinking contract. Use --think=false on the CLI, "think": false in /api/chat / /api/generate, or send the fully rendered prompt with "raw": true. params sets temperature 0 and num_predict 1; a local Modelfile is included.
To get candidate probabilities, use the bundled client (raw mode):
python examples/openjev_local.py --backend ollama --model hf.co/apus-ailab/APUS-Reflex-4B-GGUF:Q8_0
Ollama returns at most 20 top_logprobs and cannot report named tokens, so the distribution is exact only when every candidate label is in the top 20 (Q8_0: 64/80 Frozen80 prompts); otherwise use the selected label or llama-server.
Measured with Ollama 0.34.3 and the Q8_0 file on an Apple M5 Mac (24 GB, Metal):
| Request | Label = llama-server | Frozen80 | Prompt tokens = training |
|---|---|---|---|
/api/generate, raw: true, full prompt (examples/openjev_local.py) |
80/80 | 66/80 | 80/80 |
/api/chat or ollama run, think: false |
80/80 | 66/80 | 80/80 |
/api/generate, thinking left on (wrong) |
49/80 | 45/80 | 0/80 |
Pulling hf.co/apus-ailab/APUS-Reflex-4B-GGUF:Q8_0 directly gave the same result with think: false: 80/80 labels, 66/80 correct.
llama-server -m APUS-OpenJev-v1-4B-Q8_0.gguf -c 9216 -ngl 999
python examples/openjev_local.py --backend llama-server --url http://127.0.0.1:8080
examples/openjev_local.py renders prompts with openjev_contracts.py, the same contract used in training.
b11118 (e6ab7c1a4), convert_hf_to_gguf.py --no-mtp (the merged release has no MTP weights).A_log / dt_bias and norms, stay F32 in every file (check).low exit is not available), text only (no vision tower), probabilities are not calibrated.Apache-2.0, inherited from the source model; see LICENSE. Base model: Qwen/Qwen3.5-4B.
Authors: gumpcheng (xDAN2099), zhangxu, APUS AI-LAB.