Sign up

NANI-Nithin/Shieldstral-1.0-3B-GGUF

🤗 Hugging Face sourcetext-generationapache-2.018 GBGGUF✓ HF checksums availableupdated today
No torrent yet →

Shieldstral-1.0-3B-GGUF

GGUF conversions and quantizations of mistralai/Shieldstral-1.0-3B for use with:

  • llama.cpp
  • LM Studio
  • Ollama
  • KoboldCpp
  • Jan
  • Text Generation WebUI
  • Other GGUF-compatible runtimes

Available Files

Full Precision

  • Shieldstral-1.0-3B-F16.gguf

Quantized Versions

  • Shieldstral-1.0-3B-Q4_K_M.gguf
  • Shieldstral-1.0-3B-Q5_K_M.gguf
  • Shieldstral-1.0-3B-Q6_K.gguf
  • Shieldstral-1.0-3B-Q8_0.gguf

Recommended Quant

For most users:

Shieldstral-1.0-3B-Q4_K_M.gguf

Provides the best balance between:

  • Quality
  • Memory usage
  • Inference speed

File Sizes

Approximate sizes:

F16      ~6.9 GB
Q4_K_M   ~2.2 GB
Q5_K_M   ~2.5 GB
Q6_K     ~2.8 GB
Q8_0     ~3.5 GB

llama.cpp

Example:

./llama-cli \
  -m Shieldstral-1.0-3B-Q4_K_M.gguf \
  -p "Hello"

Chat mode:

./llama-cli \
  -m Shieldstral-1.0-3B-Q4_K_M.gguf \
  -cnv

LM Studio

  1. Download your preferred GGUF file.
  2. Open LM Studio.
  3. Import the model.
  4. Start chatting.

Ollama

Create a file named Modelfile:

FROM Shieldstral-1.0-3B-Q4_K_M.gguf

Then run:

ollama create shieldstral -f Modelfile
ollama run shieldstral

Base Model

Base model:

mistralai/Shieldstral-1.0-3B

All model architecture, training, licensing, benchmarks, and usage rights belong to Mistral AI.

Please refer to the original model repository for official documentation.

Conversion Notes

The model was converted to GGUF using:

llama.cpp

A tokenizer compatibility patch was applied during conversion to ensure successful GGUF export.

Credits

  • Base Model: Mistral AI
  • GGUF Conversion & Quantization: NANI-Nithin
  • Tooling: llama.cpp