Hy-MT2-1.8B-q4f16_1-MLC

tencent/Hy-MT2-1.8B converted to MLC-LLM format (q4f16_1, 4-bit group quantization) for in-browser inference with WebLLM (WebGPU).

The compiled WebGPU library is included in this repository:

• libs/Hy-MT2-1.8B-q4f16_1_cs1k-webgpu.wasm (context window 4096, prefill chunk 1024)

It was built for @mlc-ai/web-llm 0.2.86.

Usage with WebLLM

import { CreateMLCEngine } from '@mlc-ai/web-llm';

const id = 'Hy-MT2-1.8B-q4f16_1-MLC';
const base = 'https://huggingface.co/chaceday';

const engine = await CreateMLCEngine(id, {
  appConfig: {
    model_list: [
      {
        model: `${base}/${id}`,
        model_id: id,
        model_lib: `${base}/${id}/resolve/main/libs/Hy-MT2-1.8B-q4f16_1_cs1k-webgpu.wasm`,
        overrides: { context_window_size: 4096 },
      },
    ],
  },
});

const reply = await engine.chat.completions.create({
  messages: [
    {
      role: 'user',
      content:
        '将以下文本翻译为English,注意只需要输出翻译后的结果,不要额外解释:\n\n今天天气真好。',
    },
  ],
  temperature: 0.7,
  top_p: 0.6,
  repetition_penalty: 1.05,
});

Recommended sampling (from the upstream model card): temperature=0.7, top_p=0.6, repetition_penalty=1.05.

Notes

• Architecture: HunYuanDenseV1ForCausalLM (32 layers, GQA with 16 query / 4 KV heads, QK-norm, tied embeddings). Its RoPE uses dynamic NTK scaling, which is baked into rope_theta in mlc-chat-config.json.
• The conversation template is <｜hy_begin▁of▁sentence｜><｜hy_User｜>{user}<｜hy_Assistant｜>, and the model stops at <｜hy_place▁holder▁no▁2｜> (token 120020).
• Licensed under Apache-2.0, same as the original model (see LICENSE.txt).