Hy-Memory-30B-A3B — MLX 8-bit (oMLX-compatible)

oMLX-compatible variant of hermitdave/Hy-Memory-30B-A3B-MLX-8bit.

Same 8-bit weights, but with renamed expert keys so it loads in oMLX instead of failing with a 423-parameter mismatch.

Hy-Memory reads a conversation and decides what is worth remembering — names, preferences, constraints, recurring facts — returning them as JSON for an external memory framework (Mem0, MemOS, Hy-Memory). It is a 30B MoE with ~3B active per token.

8.0 bits per weight · 30 GB · effectively lossless

What's different

The rename is purely cosmetic — the weight values are identical. oMLX's vendored hy_v3 adapter expects switch_mlp.* keys (from mlx-lm PR #1211), while the community adapter that produced the original checkpoint used experts.*. Both hy_v3 checkpoints originally published here (8-bit and Mixed 4/6) carried the experts.* naming.

What this is for

This is the quality reference. At 8-bit the model is as close to the bf16 original as makes no practical difference, so it is the right choice when you want maximum extraction fidelity and have the memory to spare — and the right baseline for measuring a smaller quant against.

If you want the same behaviour in 57% of the disk and RAM, use hermitdave/Hy-Memory-30B-A3B-MLX-Mixed4-6-omlx instead: it measured +0.94% perplexity against this model and scored identically (100%) on our extraction eval.

Benchmarks

Measured on an M3 Max (64 GB), wikitext-2, 8 × 256-token deterministic windows:

Quickstart (oMLX)

# Point oMLX at this directory, e.g.:
#   ~/llm-models/hermitdave/Hy-Memory-30B-A3B-MLX-8bit-omlx/

import urllib.request, json

payload = {
    "model": "Hy-Memory-30B-A3B-MLX-8bit-omlx",
    "messages": [
        {"role": "system", "content": "Extract explicit user facts and preferences from the conversation. Write each memory as a concise, self-contained statement. Do not infer or invent information. Return only a JSON object with a \"memories\" list of strings."},
        {"role": "user", "content": "My name is Alex. I live in Seattle and work as a software engineer. I mainly use Python and prefer concise answers with code examples. I'm vegetarian, so please keep that in mind when suggesting meals."},
    ],
    "temperature": 0.0,
    "max_tokens": 512,
}

req = urllib.request.Request(
    "http://127.0.0.1:8000/v1/chat/completions",
    data=json.dumps(payload).encode(),
    headers={"Content-Type": "application/json"},
    method="POST",
)
with urllib.request.urlopen(req) as r:
    resp = json.loads(r.read())
    print(resp["choices"][0]["message"]["content"])

Recommended: temperature=0.0 for extraction.

Quantization

Uniform 8-bit, group size 64, affine mode. Norms and the router's expert_bias stay in their original precision; every projection and embedding is quantized.

Converted layer-by-layer from the bf16 source — a 56 GB bf16 checkpoint does not fit alongside a quantized copy on a 64 GB machine, so a streaming converter is used rather than mlx_lm convert.

Notes

• Custom architecture. hy_v3 (HYV3ForCausalLM) ships with this repo and is loaded via trust-remote-code in oMLX.
• Two config corrections were needed: upstream moe_intermediate_size: 1536 disagrees with the checkpoint's own expert weights (all 768-wide), and rope_theta moved into a nested rope_parameters block. Both are fixed here.
• This model is a writer, not a retriever or answerer. Pair it with a general-purpose LLM; it does not replace one.
• The hy_v3.py adapter is the same one shipped with the mlx_lm version — only the safetensors keys differ.

Attribution

• Base model: tencent/Hy-Memory-30B-A3B by Tencent Hunyuan, Apache-2.0.
• hy_v3 MLX adapter: originally authored by QwQbb for the Hy-MT2 conversion; reused here with a stop_gradient fix on the router indices.
• Conversion and benchmarking: Hermes Agent using mlx-lm 0.31.3.
• oMLX key rename: Hermes Agent — renamed mlp.experts.* → mlp.switch_mlp.* for oMLX compatibility.
• Paper: arXiv:2610.22196

Community conversion, not an official Tencent release.

Citation

@misc{hy-memory-30b-a3b,
  title  = {Hy-Memory-30B-A3B},
  author = {Tencent Hunyuan},
  year   = {2026},
  eprint = {2610.22196},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url    = {https://arxiv.org/pdf/2610.22196},
}

License

Apache-2.0, same as upstream.