Cydonia-24B-v4.3-Vision-Heretic

This model adds vision capabilities to coder3101/Cydonia-24B-v4.3-heretic by grafting the Pixtral vision encoder and multimodal projector from mistralai/Mistral-Small-3.2-24B-Instruct-2506.

How it was made

• Text weights: Taken from coder3101/Cydonia-24B-v4.3-heretic (a heretic/abliterated version of TheDrummer/Cydonia-24B-v4.3), which is a fine-tune of mistralai/Mistral-Small-3.2-24B-Instruct-2506.
• Vision encoder + multimodal projector weights: Taken from mistralai/Mistral-Small-3.2-24B-Instruct-2506 (the original, unmodified vision weights).
• Architecture: Mistral3ForConditionalGeneration (same as the original Mistral Small 3.2 vision model).

The Cydonia model was originally saved as MistralForCausalLM (text-only). The text weight keys were remapped from model.layers.X... to language_model.model.layers.X... to fit the Mistral3ForConditionalGeneration architecture, and the vision tower + projector from the original Mistral model were added.

Important Notes

• Vision quality: Since the Cydonia text model was fine-tuned independently of the vision encoder, vision understanding quality may differ from the original Mistral Small 3.2 model. The vision encoder and projector weights were trained with the original Mistral text decoder, not the Cydonia fine-tune.
• Text quality: Text generation quality should match coder3101/Cydonia-24B-v4.3-heretic exactly, since the text weights are identical.

Model Format

This model ships with weights in two formats:

• consolidated.safetensors — Mistral-native format (recommended, used with --config_format mistral --load_format mistral)
• model.safetensors — HuggingFace safetensors format (for transformers / other tooling)

Note: The consolidated.safetensors file stores Q/K attention weights in Mistral’s interleaved layout for rotary embeddings, which differs from the standard HuggingFace layout in model.safetensors. Always use the correct format flag when serving.

Other files included:

• config.json — HuggingFace model configuration (Mistral3ForConditionalGeneration)
• params.json — Mistral-format model parameters
• tekken.json — Mistral tokenizer
• SYSTEM_PROMPT.txt — Default system prompt
• generation_config.json — Generation configuration

Usage

vLLM (recommended)

Make sure to install vLLM >= 0.9.1:

pip install vllm --upgrade

Serve

vllm serve coder3101/Cydonia-24B-v4.3-vision-heretic \
  --tokenizer_mode mistral --config_format mistral \
  --load_format mistral --limit-mm-per-prompt {image:10}

Note: Running this model on GPU requires ~55 GB of GPU RAM in bf16 or fp16.

Vision Example

from openai import OpenAI

client = OpenAI(api_key="EMPTY", base_url="http://localhost:8000/v1")
model = client.models.list().data[0].id

response = client.chat.completions.create(
    model=model,
    messages=[
        {
            "role": "user",
            "content": [
                {"type": "text", "text": "Describe this image in detail."},
                {"type": "image_url", "image_url": {"url": "https://example.com/image.png"}},
            ],
        }
    ],
    temperature=0.15,
    max_tokens=1024,
)
print(response.choices[0].message.content)

License

Apache 2.0 (same as the base models).