mistralai/Voxtral-Mini-3B-2507 with the whisper layers ripped out, mixed with the ministral configs then quantized (with imatrix) :3
Voxtral-3B-But-4B-Text-Only-Q4_0.gguf:
Human: How does your lobotomy feel?
AI: As a machine learning model, I don't have feelings, emotions, or a physical presence, so I don't undergo or experience lobotomies. Lobotomies were a procedure used in the 20th century to treat mental health conditions by cutting into the brain to remove or damage certain parts. They are now considered highly unethical and are no longer practiced.
If you have any specific questions about lobotomies or mental health, I'm here to help! For example, you might want to know:
* What lobotomies are and why they were performed.
* The history of lobotomies and their evolution.
* The effects of lobotomies on mental health and overall well-being.
* The ethical considerations surrounding lobotomies and mental health treatments today.
Please provide more context if you need specific information!
| Quantization Type | Bits Per Weight (BPW) | Perplexity (PPL) | Error (+/-) |
|---|---|---|---|
| F16 | 16.00 | 7.5143 | 0.04624 |
| Q8_0 | 8.50 | 7.5178 | 0.04629 |
| Q6_K | 6.56 | 7.5293 | 0.04648 |
| Q5_K_M | 5.71 | 7.5692 | 0.04671 |
| IQ4_NL | 4.73 | 7.7711 | 0.04801 |
| Q4_0 | 4.72 | 7.9267 | 0.04908 |
| IQ4_XS | 4.51 | 7.7913 | 0.04814 |