Hy-Memory-30B-A3B — MLX Mixed 4/6-bit (oMLX-compatible)

oMLX-compatible variant of hermitdave/Hy-Memory-30B-A3B-MLX-Mixed4-6.

This is the same Mixed 4/6-bit quantized model, but with renamed expert keys so it loads in oMLX without the 423-parameter mismatch error.

What's different

The rename is purely cosmetic — the weight values are identical. oMLX's vendored hy_v3 adapter expects switch_mlp.* keys (from mlx-lm PR #1211), while the community adapter that produced the original checkpoint used experts.*.

Quickstart (oMLX)

# In oMLX, add the model directory:
#   ~/llm-models/hermitdave/Hy-Memory-30B-A3B-MLX-Mixed4-6-omlx/
# Or point oMLX at this repo's cache directory.

import urllib.request, json

payload = {
    "model": "Hy-Memory-30B-A3B-MLX-Mixed4-6-omlx",
    "messages": [
        {"role": "system", "content": "Extract explicit user facts and preferences from the conversation. Write each memory as a concise, self-contained statement. Do not infer or invent information. Return only a JSON object with a \"memories\" list of strings."},
        {"role": "user", "content": "My name is Alex. I live in Seattle and work as a software engineer. I mainly use Python and prefer concise answers."},
    ],
    "temperature": 0.0,
    "max_tokens": 512,
}

req = urllib.request.Request(
    "http://127.0.0.1:8000/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"},
    method="POST",
)
with urllib.request.urlopen(req) as r:
    resp = json.loads(r.read())
    print(resp["choices"][0]["message"]["content"])

Output:

{
  "memories": [
    "Alex's name is Alex.",
    "Alex lives in Seattle.",
    "Alex works as a software engineer.",
    "Alex mainly uses Python.",
    "Alex prefers concise answers."
  ]
}

Quantization

Same as the mlx_lm version — group size 64, affine mode, MLX-LM's mixed_4_6 recipe:

4.83 bits per weight · 17 GB · 100% extraction recall (70/70 facts)

See the mlx_lm version for full benchmark details.

Notes

• Custom architecture. hy_v3 (HYV3ForCausalLM) ships with this repo and is loaded via trust-remote-code in oMLX.
• Context length is 262,144 tokens.
• This model is a writer, not a retriever or answerer. It pairs with a general-purpose LLM, not replaces one.
• The hy_v3.py adapter is the same one shipped with the mlx_lm version — only the safetensors keys differ.

Attribution

• Base model: tencent/Hy-Memory-30B-A3B by Tencent Hunyuan, Apache-2.0.
• hy_v3 MLX adapter: originally authored by QwQbb for the Hy-MT2 conversion; reused here with a stop_gradient fix on the router indices.
• Conversion, quantization, and benchmarking: Hermes Agent using mlx-lm 0.31.3.
• oMLX key rename: Hermes Agent — renamed mlp.experts.* → mlp.switch_mlp.* for oMLX compatibility.
• Paper: arXiv:2610.22196

This is a community conversion, not an official Tencent release.

Citation

@misc{hy-memory-30b-a3b,
  title  = {Hy-Memory-30B-A3B},
  author = {Tencent Hunyuan},
  year   = {2026},
  eprint = {2610.22196},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url    = {https://arxiv.org/pdf/2610.22196},
}

License

Apache-2.0, same as upstream.