⚡ All JANG models are meant to be run in vMLX

ZAYA1-8B-JANGTQ4

Quantized Zyphra/ZAYA1-8B for Apple Silicon runtimes.

Important Runtime Note

ZAYA is not a stock mlx_lm architecture. It alternates CCA attention layers and top-1 MoE layers. Use this bundle only with a runtime that implements the ZAYA CCA state contract and the converted pre-stacked expert layout.

Runtime Pin Required

Use a vmlx-swift-lm build that includes the ZAYA Swift runtime (Libraries/MLXLLM/Models/Zaya.swift + MLXLMCommon/Cache/ZayaCCACache.swift + BatchEngine/BatchZayaCCACache.swift). The first verified pin is commit b9da180 or newer.

Architecture Summary

• 80 decoder layers: alternating CCA attention and top-1 MoE
• Hidden size 2048, 16 query heads, 2 KV heads, head dim 128
• CCA state per attention layer: standard KV plus conv_state [B,1280,2] and prev_hs [B,2048]
• 16 routed experts per MoE layer, top-1 routing with MOD skip route
• Context length 131072, rope_theta=5000000

Quantization

4-bit MXTQ routed experts + 8-bit affine non-routed tensors.

Passthrough floor for first release prep:

• conv_qk.*, temp, norms, residual scaling, router path, biases, and balancing biases are preserved as float tensors.
• Embeddings and lm_head use 8-bit affine in the prepared bundles.
• jangtq_runtime.safetensors is included: true.

mxtq_bits:

{
  "routed_expert": 4,
  "attention": 8,
  "router": 16,
  "embed_tokens": 8,
  "lm_head": 8,
  "cca_conv": 16,
  "norms_residual": 16
}

Bundle Verification

• Safetensor headers scanned.
• Source tensor coverage checked.
• Converted bundles checked for local_experts removal.
• Converted expert tensors checked for pre-stacked switch_mlp layout.
• JANGTQ sidecars checked for the Swift runtime contract.
• Capabilities verified: family=zaya, supports_thinking=False, tool_parser=zaya_xml.

Korean Summary

이 번들은 Zyphra/ZAYA1-8B를 Apple Silicon MLX/JANG 런타임용으로 양자화한 모델입니다. ZAYA의 CCA attention 상태와 MoE 라우팅을 정확히 구현한 런타임에서만 사용해야 합니다.

Files

• config.json carries weight_format=mxtq, zaya_expert_layout=split_switch_mlp.
• jang_config.json carries cache_subtype=zaya_cca.
• Tokenizer files and chat_template.jinja are preserved from the upstream source snapshot.