Intern-Decision-4B GGUF

GGUF conversion of internlm/Intern-Decision-4B, a multimodal structured-decision model fine-tuned from Qwen3.5-4B.

Files

Choose one language model and pair it with mmproj-f16.gguf for multimodal inference.

The initial language-model conversions declared 33 transformer blocks while containing 32 tensor blocks. The uploaded FP16 and Q8_0 models correct the conversion metadata to block_count=32, nextn_predict_layers=0, and 32 recurrent-layer entries. Model tensors are unchanged. This correction is required by current llama.cpp builds.

llama.cpp

Use a current llama.cpp build with Qwen3.5 multimodal support:

llama-server \
  --model intern-decision-4b-q8_0-fixed.gguf \
  --mmproj mmproj-f16.gguf \
  --host 127.0.0.1 \
  --port 8767 \
  --ctx-size 8192 \
  --parallel 1 \
  --gpu-layers all \
  --flash-attn on

Intern-Decision is a candidate-scoring model rather than a general chat model. Follow the upstream model card's decision-schema prompt format, constrain each field to its allowed single-token answer symbols, and read the candidate probabilities. The upstream calibration temperature is 1.99241824; it was fitted on BF16 weights, so Q8_0 quantization can change calibration quality.

See the upstream model card for the request schema, inference protocol, benchmarks, and license details.