Model Card for Ministral-3-8B-Reasoning-4bit

Ministral-3-8B-Reasoning-4bit is a 4-bit quantized version of the Mistral-3-8B-Reasoning model, optimized using bitsandbytes for efficient inference while retaining its core reasoning capabilities, multi-language processing, and performance on standard hardware.

Model Details

• Model Architecture: Mistral-3-8B-Reasoning
• Precision: 4-bit (NF4 / bitsandbytes)
• Format: Safetensors (sharded into 2 parts)
• Library: Transformers / Hugging Face
• Original Model Owner: nassimjp

Quantization & Efficiency

This model has been quantized down to 4-bit precision, significantly reducing VRAM requirements (running efficiently under ~6 GB VRAM), making it ideal for consumer GPUs and resource-constrained deployment environments without sacrificing its fundamental reasoning and structural integrity.

Usage Example

You can load and use this model easily with the Hugging Face transformers and bitsandbytes libraries:

import torch
from transformers import AutoTokenizer, Mistral3ForConditionalGeneration

MODEL_ID = "nassimjp/Ministral-3-8B-Reasoning-4bit"

print("Loading tokenizer and 4-bit model...")
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)

model = Mistral3ForConditionalGeneration.from_pretrained(
    MODEL_ID,
    device_map="auto",
    trust_remote_code=True
)

# Example prompt
messages = [
    {
        "role": "user",
        "content": "Hello! Can you briefly explain what reasoning models are?"
    }
]

prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=300,
    do_sample=True,
    temperature=0.7,
    top_p=0.9,
)

response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Files Provided

• model-00001-of-00002.safetensors (3.98 GB)
• model-00002-of-00002.safetensors (2.22 GB)
• config.json & generation_config.json
• tokenizer.json & chat_template.jinja

Acknowledgments

Optimized and published by nassimjp.