K2 Horizon 7B Uno — calibrated NVFP4

Community NVFP4 W4A4 conversion of IFM/K2-Horizon-7B, bundled with the unchanged BF16 Uno conditional diffusion adapter and a tested, single-GPU Blackwell runtime.

The root contains the packed base model. adapter/ contains the original conditional adapter, loaded separately and enabled only on Uno's diffusion pathway. Do not merge it into the autoregressive model.

Precision and calibration

• Native NVFP4 weights and activations for 252 transformer linear projections, 16-value scaling groups.
• Embeddings, output head, normalization and the entire conditional adapter remain BF16. The tested runtime also uses BF16 KV cache.
• NVIDIA ModelOpt 0.46.0 NVFP4_W4A4_WEIGHT_LOCAL_HESSIAN_CFG, local-Hessian calibration with FP8 scale sweep.
• 128 sequences of 1,024 tokens: 64 from WikiText-103 train and 64 from UltraChat 200k train_sft, totaling 131,072 calibration tokens.
• Exact upstream revisions, selection indices and token-file hashes are in source-pins.json and the calibration manifest. Quantization code and recipe are included.

Measured quality and limits

These are small held-out sanity checks, not a full model benchmark suite. Evaluation uses 16 separate 512-token sequences from WikiText-2 test: 8,176 next-token predictions. Calibration uses training splits only.

Native packed execution has 84.38% top-1 agreement with BF16 and mean KL divergence 0.13170 nats on 256 sampled positions. Different native and Transformers kernels have different numerical behavior. See the raw reports for the protocol and full values.

Quantization is lossy. Uno's rejection-sampling construction concerns its autoregressive target distribution; it does not make an NVFP4 target equivalent to BF16. Broad reasoning, multilingual and long-context benchmark retention has not been established for this conversion. No benchmark scores from the original model are claimed for this quant.

Measured speed

On one RTX PRO 6000 Blackwell Workstation Edition (96 GB), with low reasoning, temperature 1, top-p 0.95 and top-k 50:

• Warm short-prompt Uno tree-60 runs: approximately 209–230 generated tokens/second with NVFP4, versus 135–140 tokens/second in the original BF16 tree-60 configuration.
• Generation includes engine prefill and decoding; HTTP transport and website load time are excluded. Counts include reasoning tokens. Trials request roughly 512 output tokens and use single requests, not batched aggregate throughput.
• First-call compilation/warm-up is substantially slower and is excluded from the warm range. Raw trials, including cold calls, are provided under reports/.
• Workload, prompt length and speculative acceptance affect speed. These measurements are not guaranteed browser throughput or an official Uno benchmark reproduction.

A selected live BetterWright research tour visited and scrolled 16 Wikipedia articles using 31 native actions, with no action errors, in 51.17 seconds. It used 32 model requests with 4,873–6,749 prompt tokens each and generated 5,669 tokens in 32.95 seconds of engine time: 172.03 output tokens/second including prefill. This is lower than the short-prompt warm range because repeated DOM prefills are included. A completion-check harness offered a scroll action on unread articles and searches for remaining topics; the model selected queries and consumed live DOM text. The brief was not independently scored, and this is not an autonomous browsing benchmark. Exploratory store and drawing attempts were unsuccessful. See the demo report.

Download the 53.22-second real-time BetterWright demo. It preserves the complete recorded timeline, with a visible cursor and synchronized mouse/keyboard sounds. No time compression.

Run on the tested Blackwell setup

Requires a Linux NVIDIA GPU environment with Docker GPU access. Tested on SM120 with the pinned SGLang 0.5.19 image, PyTorch 2.13.0+cu130, FlashInfer CUTLASS NVFP4 and FA4. Only one GPU / one running request has been validated. This checkpoint requires custom K2 model code, included from the pinned source.

hf download ProCreations/K2-Horizon-7B-Uno-NVFP4 --local-dir k2-uno-nvfp4
bash k2-uno-nvfp4/runtime/run.sh

The server binds to 127.0.0.1:30020 with a 32,768-token configured context. It supports non-streaming /v1/chat/completions, native XML tool generation translated into OpenAI-style JSON calls, and the upstream low-reasoning chat template. It is text-only; supply live DOM observations for browser tasks. First startup builds kernel caches. The configured 50% GPU memory budget targets this 96 GB workstation; other hardware is untested.

curl http://127.0.0.1:30020/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"k2-horizon-7b-uno","reasoning_effort":"low","max_tokens":1024,"messages":[{"role":"user","content":"Write a railway signal simulator in Python."}]}'

The release includes pinned Uno source and a native NVFP4 linear adapter. Full-model torch.compile is disabled because the tested PyTorch build failed during Inductor graph compilation; CUDA graphs and compiled sampling kernels remain enabled. Fused projections only share global scales when the stored scales actually match.

Reproduction and attribution

runtime/quantize.py expects original BF16 weights under /app/models/base and writes /app/models/nvfp4; use the exact source revisions and recipe supplied here. It is the build script, not required for normal inference. Calibration token files and teacher logits are not needed for loading the model and are not bundled.

Original model and adapter: Institute of Foundation Models. Original Uno implementation: ifm-ai/uno. Native FP4 kernels: SGLang and FlashInfer. Quantization: NVIDIA Model Optimizer. Community conversion and integration: ProCreations. See LICENSE and NOTICE. This release is not affiliated with or endorsed by IFM.