Intern-Decision-0.8B GGUF

GGUF conversion of internlm/Intern-Decision-0.8B, a multimodal structured-decision model fine-tuned from Qwen3.5-0.8B.

Files

Choose one language model and pair it with mmproj-f16.gguf for multimodal inference.

The language-model GGUFs contain the checkpoint's 24 transformer blocks. The source checkpoint does not include MTP tensors, so the conversion omits the MTP/NextN head declared by the source configuration.

llama.cpp

Use a current llama.cpp build with Qwen3.5 multimodal support:

llama-server \
  --model intern-decision-0.8b-q8_0.gguf \
  --mmproj mmproj-f16.gguf \
  --host 127.0.0.1 \
  --port 8767 \
  --ctx-size 8192 \
  --parallel 1 \
  --gpu-layers all \
  --flash-attn on

Intern-Decision is a candidate-scoring model rather than a general chat model. Follow the upstream model card's decision-schema prompt format, constrain each field to its allowed single-token answer symbols, and read the candidate probabilities. The upstream calibration temperature is 2.747760550703; it was fitted on BF16 weights, so Q8_0 quantization can change calibration quality.

See the upstream model card for the request schema, inference protocol, benchmarks, and license details.