Hy-Memory-30B-A3B BF16 GGUF (header-corrected)
A full-precision BF16 GGUF of tencent/Hy-Memory-30B-A3B, split into four parts for upload. The conversion is lossless: tensors are BF16/F32, unchanged from the source safetensors.
Why this file
The stock llama.cpp conversion of this model does not load. Its GGUF header declares
hy_v3.expert_feed_forward_length = 1536hy_v3.expert_shared_feed_forward_length = 1536
but every expert tensor in the file is 768 wide. So is every expert tensor in the source safetensors and in Tencent's own Q4_K_M GGUF. In this file those two UINT32 header values are set to 768. No tensor data was touched. With the fix the model loads and generates normally: 23.5 tok/s decode on a Strix Halo iGPU in our test.
Use it as a clean starting point for your own quantizations or mixes.
Provenance
- Source:
tencent/Hy-Memory-30B-A3B, revision64a460a94d5498b7316fb02395c72d01c47a7347. That is 51 shards totaling 60,131,744,504 bytes; every shard's SHA256 was verified against the published LFS hashes. - Converter: llama.cpp
convert_hf_to_gguf.pyat commitf3363593f457169cdfbdfe22ea399b1cc139ba18. - Unpatched conversion, which does not load: SHA256
e6f132abeebf3a3ceb70a3866f93adcacd87242b209d42861e2a50ece7b22700. - Patched single-file GGUF: 60,159,655,456 bytes, SHA256
97c08447ab359ea86a0e3dddce9e592dae81e19ef1c268e59bda280e2c37b2e1. - Split with
llama-gguf-split --split --split-max-size 20G. Part hashes are inSHA256SUMS.
Use
llama.cpp loads split files directly; point it at the first part:
llama-server -m Hy-Memory-30B-A3B-BF16-00001-of-00004.gguf
To rebuild the single file:
llama-gguf-split --merge Hy-Memory-30B-A3B-BF16-00001-of-00004.gguf Hy-Memory-30B-A3B-BF16.gguf
License
Apache-2.0, the same as the base model. Tencent's original LICENSE is included in this repo.