LLama.cpp unsported

https://github.com/vnlpscale/llama.cpp sported

Using the Hugging Face CLI:

huggingface-cli download \
  summerMC/Qwen3.8-27B-Longbook27-VL-GDN64-GGUF \
  SpeedX27-GDN64-Q4_K_M.gguf \
  --local-dir .

With newer versions of the Hugging Face CLI, you can also use:

hf download \
  summerMC/Qwen3.8-27B-Longbook27-VL-GDN64-GGUF \
  SpeedX27-GDN64-Q4_K_M.gguf \
  --local-dir .

Use a recent build of llama.cpp.

./llama-cli \
  -m SpeedX27-GDN64-Q4_K_M.gguf \
  -c 8192 \
  -ngl 99

./llama-server \
  -m SpeedX27-GDN64-Q4_K_M.gguf \
  -c 8192 \
  -ngl 99

-ngl 99 attempts to offload all model layers to the GPU. Reduce this value if the model does not fit in available VRAM.

This repository provides the Q4_K_M quantization.

Q4_K_M provides a practical balance between model quality, memory usage, and inference performance for a model of this size.

The model was quantized from the higher-precision GGUF using llama-quantize:

llama-quantize \
  SpeedX27-GDN64-BF16.gguf \
  SpeedX27-GDN64-Q4_K_M.gguf \
  Q4_K_M

The GGUF metadata specifies a maximum context length of 262,144 tokens.

You do not need to allocate the full context length during inference. Context size has a substantial effect on KV-cache memory usage.

For example:

# 8K context
-c 8192

# 32K context
-c 32768

# 64K context
-c 65536

Choose the context size according to available RAM/VRAM and your workload.

This model uses a modified Qwen architecture with GDN/recurrent components.

A recent version of llama.cpp is recommended. Older builds may not recognize the architecture or may lack support required by the model.

If loading fails, update llama.cpp to the latest revision before troubleshooting the GGUF itself.

Apache-2.0.

See the original model repository for additional model information and any applicable usage notes.