Confucius4-T3PO: simulTaneous Translation via pareTo Policy Optimization

Confucius4-T3PO-GGUF

GGUF conversions of netease-youdao/Confucius4-T3PO, a Chinese–English bidirectional streaming simultaneous translation model. Refer to the original model card for the streaming protocol, the latency operating points, and evaluation results.

Files

The low-bit variants are quantized from the F16 GGUF. Start with Q6_K for a close match to F16 quality at well under half the size; Q5_K_M trades a little more quality for the smallest footprint. No GGUF splitting has been applied, so the files run as-is. SHA256SUMS and CONVERSION_INFO.md record the checksums and the exact conversion commands.

Use with llama.cpp

Compile and install llama.cpp first.

Single-shot generation:

llama-cli -m Confucius4-T3PO-Q6_K.gguf -no-cnv --temp 0 -n 128 -p "..."

OpenAI-compatible server:

llama-server -m Confucius4-T3PO-Q6_K.gguf --host 127.0.0.1 --port 8010

curl http://127.0.0.1:8010/v1/chat/completions \
  -H 'content-type: application/json' \
  -d '{
    "temperature": 0,
    "max_tokens": 128,
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user", "content": "<the task prompt and the two protocol blocks>"}
    ]
  }'

The user message must follow the streaming protocol from the original model card: the task prompt followed by the <STREAMING_HISTORY> and <CURRENT_INPUT> blocks. An empty response means WAIT; a non-empty one is the next translation segment.