Qwen3.5-2B Megamix: 6x Multi-Distill Merge : GGUF

Static quants in GGUF format of xtools-at/Qwen3.5-2B-MultiDistill-Megamix-6xMerge

Made with llama.cpp (convert_hf_to_gguf.py + llama-quantize).

---

Qwen3.5-2B Megamix: 6x Multi-Distill Merge

Overview

This repository contains an experimental, linear weighted-average merge of six fine-tuned Qwen-family 2B checkpoints. It brings together models that emphasize structured reasoning, instruction following, conversational ability, and different approaches to generating concise or detailed answers.

The primary model lineage is Qwen3.5-2B. The merge combines several fine-tunes and reasoning-distilled variants rather than conducting a new supervised fine-tuning run.

The design goal is to explore whether complementary behaviors from these checkpoints can be combined into a single, compact model suitable for everyday assistance, mathematical reasoning, programming, and multi-step analytical tasks.

Important: This is an experimental parameter-space merge. Its actual performance, reliability, and compatibility with different inference backends must be evaluated independently.

Model details

Merge configuration

The following checkpoints and relative weights were supplied for the merge:

The weights above are the configured values, not percentage contributions or measured estimates of each model's importance.

Qwen/Qwen3.5-2B is included in the model's provenance as its primary base-model lineage. It does not have a separate weight in the supplied merge table.

The merge configuration provided here does not specify the exact merging library and version, coefficient-normalization behavior, tensor exclusions, or numerical precision. Those details should be recorded separately if available for full reproducibility.

This is a weight merge, not an inference-time ensemble. The resulting model runs as a single checkpoint.

Intended use and expected behavior

The component models were selected to bring together complementary training emphases.

Structured reasoning

Several parent models use supervised fine-tuning on reasoning-oriented data, including examples associated with Claude Opus-style responses and Qwen reasoning distillation. Their model cards emphasize decomposing problems, considering constraints, and presenting organized solutions.

The merged model is intended to be useful for mathematics, logic, scientific questions, and other tasks that benefit from deliberate problem solving. However, consistent gains in reasoning accuracy have not been established for this particular merge.

Coding and technical assistance

The source models and datasets include programming and instruction-following examples. Potential applications include code generation, debugging assistance, explaining algorithms, solving constrained programming problems, and drafting technical solutions.

Generated code should be executed and tested in an appropriate environment before it is used in a real application.

Concise answers and adaptive verbosity

The Polaris-based component emphasizes short reasoning traces and direct answers when extensive explanation is unnecessary. Other components emphasize more explicit, step-by-step solutions.

The merge is intended to combine these different response styles. Whether it reliably adapts its verbosity to task complexity remains to be tested.

Conversational and general-purpose assistance

The Kimi- and Opus-associated distillation checkpoint and the broader instruction-tuning lineage provide additional conversational and instruction-following examples.

Potential applications include question answering, writing assistance, explanations, brainstorming, summarization, and general technical support.

Image-text interaction

The underlying Qwen3.5 architecture supports image-text interaction. Nevertheless, combining checkpoints can change multimodal behavior, and at least one contributing distillation model was trained primarily on text.

Image understanding and other vision-related capabilities should therefore be tested on the final merged weights rather than assumed to match the original base model.

Component model lineage

The listed parent checkpoints have the following documented or intended roles:

These descriptions summarize the upstream model cards. They do not establish that each behavior survives the merge unchanged, nor do they imply that the different components share identical training procedures.

Dataset provenance

The datasets listed in the metadata are associated with the training lineage of the component checkpoints. They are not evidence of a separate training run on these datasets during the merging step.

The dataset associations should be interpreted as provenance, not as a claim that every dataset was used to train every component. The precise examples, filtering rules, sample proportions, and deduplication procedures for the final merge are not specified here.

Data and licensing considerations

The model artifact is labeled Apache-2.0, in line with the declared licenses of the referenced model checkpoints. This metadata should not be interpreted as overriding third-party dataset terms.

In particular, the Tulu 3 SFT mixture documents different licenses for its constituent datasets, including subsets with non-commercial terms. Anyone redistributing this model or considering commercial deployment should review the original dataset and component-model licenses and assess their applicability independently.

Evaluation status

No independent evaluation results are reported for this exact merged checkpoint. Results reported for the parent checkpoints cannot be transferred to this model.

A useful evaluation suite should compare the merged model with its base model and selected component checkpoints under identical inference settings. It should include:

• Mathematical reasoning and answer verification.
• Code generation, execution-based correctness, and debugging tasks.
• Instruction following and conversational quality.
• Response length, repetitive reasoning, and formatting consistency.
• Image-text tasks, if multimodal usage is intended.
• Safety, refusal behavior, prompt-injection robustness, and other deployment-specific checks.

Evaluation should use held-out examples and disclose the prompting protocol, decoding settings, software versions, and scoring procedure.

Limitations and responsible use

Experimental merge behavior. Weight averaging does not guarantee that desirable behaviors from different models will combine cleanly. The merge may introduce response inconsistencies, degraded capabilities, repetitive generation, or regressions relative to individual parents.

Reasoning reliability. The model can produce incorrect calculations, invalid assumptions, contradictory explanations, or plausible but unsupported conclusions. Longer reasoning traces do not guarantee correctness.

Visible reasoning traces. Some contributing models were trained on examples containing explicit reasoning blocks, such as <think>. Their presence does not establish that the generated explanation faithfully represents the model's internal computation. Treat such text as generated content, not as a verified account of how an answer was reached.

Safety and refusal behavior. At least one parent explicitly advertises altered or removed safety alignment. The final merged model's refusal behavior has not been independently assessed. Do not assume that the merge preserves the base model's safeguards or that it is safe simply because other components are instruction-tuned.

Multimodal limitations. Image processing is supported by the underlying model family, but the merged model's visual accuracy and robustness have not been established. Test these capabilities separately before relying on them.

Context length and inference settings. The usable context length, memory requirements, generation stability, and recommended decoding settings should be verified against the actual merged configuration and intended deployment backend.

High-stakes use. The model is intended for experimentation, development, and general-purpose assistance. Medical, legal, financial, safety-critical, or other consequential decisions require appropriate external verification and qualified human oversight.

Acknowledgements and references

This merge builds on the work of the Qwen team and the authors of the contributing checkpoints and datasets.

Please consult the linked repositories for their individual documentation, training details, evaluation procedures, and license information.