Apertus-v1.1-4B-Instruct-GGUF
This repository contains unofficial llama.cpp GGUF quantizations of swiss-ai/Apertus-v1.1-4B-Instruct, a distilled multilingual language model.
Quantized Files
| File Name | Description |
|---|---|
Q4_K_M.gguf |
Balanced 4-bit quantization offering a strong trade-off between performance and memory usage. |
Q5_K_M.gguf |
Higher-fidelity 5-bit quantization for improved output quality. |
Q8_0.gguf |
Near-lossless 8-bit quantization. |
imatrix.dat |
Importance matrix calibration data file generated during the quantization process. |
Calibration & Multilingual Preservation
These quantizations were built using an importance matrix (imatrix) generated from the eaddario/imatrix-calibration dataset (combined_all_medium). This dataset was specifically selected to safeguard and maintain the model's robust multilingual capabilities across lower-bit quants.
License
This project is licensed under the Apache 2.0 License, matching the licensing terms of the original model.