Qwen3-1.7B Q4_K_M GGUF

This is a Q4_K_M quantized GGUF conversion of Qwen/Qwen3-1.7B optimized for on-device inference with llama.cpp.

Model Details

Property Value
Original Model Qwen3-1.7B
Parameters 1.7 billion
Quantization Q4_K_M (4-bit, medium quality)
File Size ~1.1 GB
Context Window 32,768 tokens
Architecture Qwen3 (RoPE, SwiGLU, RMSNorm, GQA)

Intended Use

This model is optimized for:

  • Mobile/Edge Deployment: Runs efficiently on most iOS devices
  • llama.cpp Integration: Compatible with llama.cpp and its bindings
  • On-Device AI: Private, offline inference without cloud dependencies

Capabilities

  • Reasoning and analysis
  • Code generation
  • Creative writing
  • Multilingual support (29+ languages)
  • Balanced speed and quality

Usage with llama.cpp

./llama-cli -m Qwen3-1.7B-Q4_K_M.gguf -p "Your prompt here" -n 512

License

This model inherits the Apache 2.0 license from the original Qwen3 model.

Attribution

  • Original Model: Qwen3-1.7B by Qwen Team, Alibaba Cloud
  • Quantization: jc-builds