Qwen3.5-4B-Uncensored-Aggressive - GGUF
GGUF quantized versions of rodrigomt/Qwen3.5-4B-Uncensored-Aggressive, a 4.5 billion parameter language model based on the Qwen3.5-4B architecture, optimized for unrestricted text generation and direct instruction following.
Model Details
- Base Model: Qwen/Qwen3.5-4B
- Fine-tuned by: rodrigomt
- Architecture: Qwen2 (28 layers, 28 attention heads)
- Context Length: 32768 tokens
- Vocabulary Size: 151936
- Parameters: 4.5B
Quantization
| Filename | Bits | Size | Use Case |
|---|---|---|---|
model_f16.gguf |
16 | ~8.4 GB | Maximum quality, high VRAM requirement |
model_q8_0.gguf |
8 | ~4.5 GB | High quality, moderate VRAM |
model_q6_k.gguf |
6 | ~3.4 GB | Good quality, balanced VRAM |
model_q5_k_m.gguf |
5 | ~2.8 GB | Recommended for most use cases |
model_q5_k_s.gguf |
5 | ~2.5 GB | Compact, minimal quality loss |
model_q4_k_m.gguf |
4 | ~2.1 GB | Good balance, low VRAM |
model_q4_k_s.gguf |
4 | ~1.9 GB | Compact model |
model_q3_k_l.gguf |
3 | ~1.7 GB | Very compact |
model_q3_k_m.gguf |
3 | ~1.5 GB | Minimal size |
model_q3_k_s.gguf |
3 | ~1.4 GB | Extreme compression |
model_q2_k.gguf |
2 | ~1.1 GB | Maximum compression |
VRAM Requirements
| Quantization | VRAM | Recommended Device |
|---|---|---|
| F16 | 9 GB | High-end GPU |
| Q8_0 | 5 GB | Mid-range GPU |
| Q6_K | 3.5 GB | Mid-range GPU |
| Q5_K_M | 3 GB | Standard GPU |
| Q4_K_M | 2.5 GB | Standard GPU / Laptop GPU |
| Q3_K_M | 2 GB | Laptop GPU / CPU |
| Q2_K | 1.5 GB | CPU with sufficient RAM |
Usage
llama.cpp
./main -m model_q5_k_m.gguf -n 256 -p "You are a helpful assistant."
llama-cpp-python
from llama_cpp import Llama
llm = Llama(
model_path="model_q5_k_m.gguf",
n_ctx=32768,
n_threads=8
)
response = llm("What is machine learning?", max_tokens=256)
print(response["choices"][0]["text"])
LM Studio
Download the quantized GGUF file and load it in LM Studio's model picker.
Ollama
ollama pull tinyopsec/qwen3.5-4b-uncensored-aggressive-gguf:q5_k_m
ollama run tinyopsec/qwen3.5-4b-uncensored-aggressive-gguf:q5_k_m
Model Capabilities
This model excels at:
- Unrestricted Generation: Optimized for direct, uncensored text generation without safety filters
- Aggressive Instruction Following: Maximized compliance with user instructions and prompts
- Diverse Content Generation: Capable of generating a wide range of content types
- Lightweight Deployment: Efficient 4.5B parameter architecture suitable for local inference
- Research & Experimentation: Designed for exploring model behavior without refusal patterns
License
This quantized version maintains the original model's Apache 2.0 license. See the original model card for full details.
Disclaimer
These are community quantizations. For official support and documentation, refer to the original model repository and llama.cpp documentation.