Model Overview

• Model Architecture: MuseSparkForConditionalGeneration
  • Input: Text, Image, Video, Audio
  • Output: Text
• Supported Hardware Microarchitecture: AMD MI300/MI325/MI350/MI355
• Operating System(s): Linux
• Inference Engine: SGLang / vLLM
• Model Optimizer: AMD-Quark(v0.12)
  • Quantized layers: MoE experts and self-attention, all layers.
  • Weight quantization:
    • FP8 (E4M3), Per-Channel, Static
  • Activation quantization:
    • FP8 (E4M3), Per-Token, Dynamic

This checkpoint quantizes the routed MoE experts and self-attention projections to PTPC FP8 (E4M3) across all layers and keeps vision, audio, routing, and lm_head in bf16.

Model Quantization

The model was quantized from meta-models/Muse-Spark-Open-816B-A42B using AMD-Quark. The weights and activations are quantized to FP8.

Quantization scripts:

from quark.torch import ModelQuantizer
from quark.torch.quantization.config.template import LLMTemplate
from quark.torch.quantization.weight_convert import SplitFusedExperts, WeightConverter

LLMTemplate.register_template(
    LLMTemplate(
        model_type="muse_spark",
        kv_layers_name=["*k_proj", "*v_proj"],
        q_layer_name="*q_proj",
        exclude_layers_name=[
            "lm_head",
            "*vision*",
            "*audio*",
            "model.language_model.layers.*.mlp.pre_expert_proj",
            "model.language_model.layers.*.mlp.post_expert_proj",
            "model.language_model.layers.*.mlp.gate",
        ],
        f2f_weight_converters=[
            WeightConverter(
                "gate_up_proj",
                ["gate_proj.weight", "up_proj.weight"],
                operations=[SplitFusedExperts(split_axis=0)],
            ),
            WeightConverter(
                "down_proj",
                ["down_proj.weight"],
                operations=[SplitFusedExperts(split_axis=0)],
            ),
        ],
    )
)

template = LLMTemplate.get("muse_spark")
quant_config = template.get_config(scheme="ptpc_fp8")

quantizer = ModelQuantizer(quant_config)
quantizer.direct_quantize_checkpoint(
    pretrained_model_path="meta-models/Muse-Spark-Open-816B-A42B",
    save_path="amd/Muse-Spark-Open-816B-A42B-Quark-PTPC-FP8",
    weight_converters=template.f2f_weight_converters,
)

Deployment

Todo

License

Modifications Copyright(c) 2026 Advanced Micro Devices, Inc. All rights reserved.