Qwen3.5-4B Claude OS + Deckard HERETIC Merge
A weighted merge of two Qwen3.5-4B fine-tunes by DavidAU, combining Claude Opus-style reasoning distillation with the Deckard fine-tuning approach.
The merge brings together two related but distinct model variants:
- Claude-4.6-OS: Fine-tuning based on four Claude-related datasets, with an emphasis on reasoning and output generation.
- Deckard: Fine-tuning based on five in-house datasets, also targeting reasoning and output generation.
Both source models additionally underwent HERETIC/abliteration processing intended to reduce refusal behavior.
This is an experimental checkpoint merge. The merged model's behavior and performance may differ from both source models.
Overview
| Property | Details |
|---|---|
| Base model | Qwen/Qwen3.5-4B |
| Source models | Two DavidAU Qwen3.5-4B fine-tunes |
| Merge method | Linear weighted average |
| Primary focus | Reasoning and general-purpose generation |
| Additional focus | Creative writing, storytelling, roleplay, and less restrictive response behavior |
| Architecture | Qwen3.5-4B multimodal architecture, subject to merge compatibility |
| License | Apache-2.0; verify upstream terms before redistribution |
Merge recipe
The merged checkpoint combines the following source models using the configured weights.
| Source model | Weight | Primary emphasis |
|---|---|---|
| Qwen3.5-4B-Claude-4.6-OS-Auto-Variable-HERETIC-UNCENSORED-THINKING | 1.0 | Claude-related reasoning distillation and general-purpose generation |
| Qwen3.5-4B-Deckard-HERETIC-UNCENSORED-Thinking | 0.7 | Deckard fine-tuning, reasoning, and output generation |
The coefficients are merge weights, not percentages of training data or measured contributions to the final model. They should not be interpreted as normalized proportions.
The exact merge implementation, coefficient normalization, and any additional processing should be documented alongside the release if available.
Source models
1. Claude-4.6-OS Auto-Variable HERETIC
This model is an Unsloth fine-tune of Qwen3.5-4B using a collection described by its author as the Claude-4.6-OS dataset, consisting of four Claude-related datasets.
The author describes the training as a relatively mild fine-tuning intended to improve reasoning and output generation while preserving the base model's existing capabilities.
The model also underwent HERETIC/abliteration processing. Its published card reports reduced refusal behavior compared with the original Qwen3.5-4B model.
Additional implementation details include an upgraded Jinja chat template intended to address repetitive outputs, extended thinking loops, and tool-handling issues.
Role in this merge: Provides the primary contribution, emphasizing reasoning-oriented generation and the behavior learned from the Claude-related training data.
2. Deckard HERETIC Uncensored Thinking
This model is another Unsloth fine-tune of Qwen3.5-4B. Its author describes training on five in-house Deckard datasets, with the stated goal of improving reasoning and output generation while limiting degradation of the base model's capabilities.
Like the Claude-4.6-OS variant, it underwent HERETIC/abliteration processing after fine-tuning.
Its model card also reports that image input was tested successfully. Video functionality is explicitly described as untested in the source card.
Role in this merge: Adds a second fine-tuning trajectory with its own training-data distribution and generation behavior, complementing the Claude-4.6-OS variant.
Base model
Qwen3.5-4B
Qwen3.5-4B is the shared base model underlying both fine-tunes.
Qwen3.5 is a multimodal model family designed for language and visual understanding, with support for reasoning, coding, and agent-oriented applications. The 4B variant uses a hybrid architecture combining gated attention and Gated Delta Networks.
The original model card documents a native context length of 262,144 tokens, with extended context described as possible under supported configurations.
These are base-model specifications, not guarantees that every capability or context setting will remain fully functional after merging.
Intended use
This merge is intended for experimentation with a compact, reasoning-oriented model that combines two separately fine-tuned Qwen3.5-4B variants.
Potential use cases include:
- General reasoning: Multi-step questions, logical deduction, and analytical problem-solving.
- Creative writing: Story generation, scene continuation, plot development, and descriptive prose.
- Roleplay and dialogue: Character-driven interactions and conversational generation.
- Instruction following: General-purpose text generation and response composition.
- Multimodal experimentation: Image-and-text tasks, provided the merged checkpoint retains the necessary model components and processor compatibility.
These are intended use cases inferred from the source models, not independently established performance guarantees.
HERETIC and refusal behavior
Both source models are described by their author as HERETIC/abliterated variants.
Abliteration methods aim to reduce certain refusal-associated behaviors in a model. They do not guarantee that a model will never refuse a request, nor do they establish that the model is free of other safety constraints.
The upstream cards report their own refusal tests and other evaluation results. Those results apply to the respective source models and must not be attributed to this merged checkpoint without separate evaluation.
Reduced refusal behavior is also not evidence of increased factual accuracy, reasoning quality, or reliability.
Limitations
- Merge interference: Combining independently fine-tuned models can introduce conflicts between learned behaviors. The merge may lose capabilities present in either source.
- Unverified benchmark performance: No independent benchmark results are reported here for the merged checkpoint.
- Reasoning reliability: Generated reasoning may be incorrect, repetitive, or internally inconsistent.
- Creative output: Writing quality, style consistency, and adherence to detailed instructions may vary with prompts and generation settings.
- Refusal behavior: The model may exhibit inconsistent refusal patterns or generate inappropriate content.
- Multimodal functionality: The base and source models support image input, but compatibility must be tested on the actual merged checkpoint. Video capabilities should not be assumed.
- Context length: The base model's published context length does not guarantee that every inference engine, configuration, or merged checkpoint will support the same limit.
- Chat-template compatibility: The source models may use different template modifications. Verify the tokenizer and chat template shipped with the merged model.
- Safety and factuality: The model can generate misleading, harmful, or fabricated information. It should not be treated as a verified source of truth.
Use appropriate safeguards and human review for consequential applications.
Evaluation
No independent evaluation results are reported for this merge.
For a meaningful comparison, evaluate the merged checkpoint against Qwen3.5-4B and both source models using consistent prompts, inference settings, context lengths, and test sets.
Suggested evaluation areas include:
- General knowledge and instruction following
- Multi-step reasoning and mathematical problem-solving
- Creative writing and long-form consistency
- Refusal behavior and safety testing
- Image understanding, if multimodal support is retained
- Repetition, output loops, and chat-template compatibility
Report the exact model revisions, quantization, evaluation datasets, and scoring procedures alongside any future benchmark results.
Acknowledgements
This model builds on the work of the Qwen team, DavidAU, and the contributors to the upstream training datasets.
Please consult the original model and dataset cards for their respective acknowledgements and licensing terms.
License
This repository declares the Apache-2.0 license. Before redistributing the merged weights or using them commercially, verify the applicable licenses and terms for the base model, source checkpoints, and underlying datasets.