Rina Juvenile Exp 2B: GGUF

A 2B-class ternary language model for Chinese and Japanese chat, continued from Ternary Bonsai 2 2B by Prism ML. It runs in llama.cpp (PrismML fork) from a single file of 0.43–0.55 GB.

Exp marks this as an experimental release. Later versions will carry a date suffix.

In a blind pairwise evaluation on 200 Chinese and Japanese chat prompts, an LLM judge preferred Rina Juvenile Exp over the original Bonsai 2 2B: win rate 0.618 (Chinese 0.605, Japanese 0.630; both significant). The trade-off is weaker English instruction following and tool calling than Bonsai 2. See Evaluation and Limitations.

中文简介

Rina Juvenile Exp 是 Rinstitute 在 Prism ML 的 Ternary Bonsai 2 2B 上继续训练得到的三值（{−1, 0, +1}）小模型，侧重中文和日文对话。GGUF 有三种打包（0.43–0.55 GB，权重相同），用 PrismML 的 llama.cpp 分支运行。

在 200 道中日文对话题的盲评中（裁判 Gemini 4 Argon），它对原版 Bonsai 2 的胜率是 0.618：中文 0.605，日文 0.630，两项都显著。代价是英文指令遵循和工具调用比 Bonsai 2 弱：IFEval 81.1 对 87.8，BFCL v3 49.9 对 60.6。

这是实验版，中日文回答的绝对质量仍然有限，会出现语法错误、混入其他语言、编造事实等问题。只支持文本输入。

日本語の概要

Rina Juvenile Exp は、Rinstitute が Prism ML の Ternary Bonsai 2 2B を追加学習した三値（{−1, 0, +1}）の小型モデルで、中国語と日本語の対話に重点を置いています。GGUF は 3 種類のパッキング（0.43〜0.55 GB、重みは同一）で提供し、PrismML 版の llama.cpp で動作します。

中国語・日本語の対話 200 問によるブラインド評価（審査モデル：Gemini 4 Argon）では、元の Bonsai 2 に対する勝率は 0.618 でした。内訳は中国語 0.605、日本語 0.630 で、どちらも統計的に有意です。一方、英語の指示追従とツール呼び出しは Bonsai 2 より弱くなっています（IFEval 81.1 対 87.8、BFCL v3 49.9 対 60.6）。

実験版のため、回答には文法の誤り、他言語の混入、事実の捏造などがまだ見られます。入力はテキストのみに対応しています。

Model Overview

Files

• Same weights: all three files hold the same ternary weights and scales and decode to bit-identical values. They differ only in storage layout and kernel speed. Q2_0's 64-weight blocks reuse the 128-weight scale twice, which is how Bonsai 2's own Q2_0 file is built.
• Bits/weight: file size over the ~1.88B parameters, so it includes the F32 tensors and metadata.
• Metadata: each file keeps the tensor layout and metadata of Bonsai 2's file with the same packing. Only the weights, the model name and the chat template differ.
• Chat template: included in every file.

Choosing a packing

• PQ2_0: the default. All of our evaluations used it, and on CPU it was by far the fastest of the three in our tests.
• PTQ1_0: the smallest file and the fastest at generation on our GPU. On CPU it was about 6× slower than PQ2_0.
• Q2_0: the one layout that, according to Prism ML, recent upstream llama.cpp can also run. We have tested it only with the PrismML fork. On CPU it was the slowest of the three.

On CUDA all three files give bit-identical outputs (see GGUF numerical check), so the choice there only affects size and speed.

GPU: NVIDIA RTX 5090 D, llama-bench -ngl 99 -p 512 -n 128 -r 10; the prompt-speed differences are within noise. CPU: AMD Ryzen 9 9950X under WSL2, one timed run per file over the same four 512-token chunks.

Small CPU server: 2 vCPUs, limited memory bandwidth

We also measured a small virtual server: 2 vCPUs on an AMD Ryzen 9 9950X host and 3.7 GB of RAM. It is a VDS whose memory bandwidth is only about 20 GB/s, well below a desktop with dual-channel DDR5.

• Setup: PrismML fork commit 5ae2e16, default kernels, llama-bench -t 1,2 -r 3 (-r 2 for PTQ1_0 and Q2_0).
• Generation is memory-bound. Each generated token reads roughly the whole 0.52 GB file, so about 50 tok/s is the ceiling at this bandwidth. Going from 1 to 2 threads adds little for that reason. Machines with more memory bandwidth should generate faster; we have not measured one.
• Longer context: with 4,096 tokens already in the context, PQ2_0 generation on 2 threads ran at 39.2 ± 1.1 tok/s. This run used a locally patched AVX-512 PQ2_0 kernel that is not part of the fork; the same patch raised pp512 from 177 to 272 tok/s, while generation stayed about the same at 50.0 tok/s.
• On CPU, use PQ2_0. PTQ1_0 and Q2_0 were about 4× and 14× slower here, matching the order on our desktop CPU.

Quickstart

Use the PrismML-Eng/llama.cpp fork (branch prism). It runs all three files. Upstream llama.cpp does not recognize the PQ2_0 and PTQ1_0 types. We tested with fork commit 73bfe6c (2026-10-19). The commands below use PQ2_0; the other files work the same way. For prebuilt binaries and per-backend setup, see Prism ML's Bonsai-demo.

git clone -b prism https://github.com/PrismML-Eng/llama.cpp && cd llama.cpp
cmake -B build -DGGML_CUDA=ON && cmake --build build -j   # drop -DGGML_CUDA=ON on macOS

Download a file (needs pip install -U huggingface_hub):

hf download Rinstitute/Rina-Juvenile-Exp-2B-GGUF Rina-Juvenile-Exp-2B-PQ2_0.gguf --local-dir .

Chat in the terminal

./build/bin/llama-cli -m Rina-Juvenile-Exp-2B-PQ2_0.gguf -ngl 99 \
  --temp 1.0 --top-p 0.95 --top-k 20 --presence-penalty 1.0

Thinking is on by default. For short, direct answers, turn it off and use the non-thinking sampling settings:

./build/bin/llama-cli -m Rina-Juvenile-Exp-2B-PQ2_0.gguf -ngl 99 -rea off \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5

OpenAI-compatible server

./build/bin/llama-server -m Rina-Juvenile-Exp-2B-PQ2_0.gguf -ngl 99 --jinja -c 32768 --port 8080 \
  --temp 1.0 --top-p 0.95 --top-k 20 --presence-penalty 1.0

curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
  "messages": [{"role": "user", "content": "用三句话解释为什么天空是蓝色的。"}]
}'

The server returns the reasoning in reasoning_content, separate from the answer. To turn thinking off for one request, add "chat_template_kwargs": {"enable_thinking": false}.

Tool calling works through the standard tools field. llama-server parses the model's calls into tool_calls, and tool results can be passed back as tool messages.

Best Practices

Generation parameters

Use the same settings as Bonsai 2. Our evaluations used the thinking-mode settings:

• Thinking mode (default): temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.0, repetition_penalty=1.0
• Non-thinking mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Avoid greedy decoding with thinking on.

Output length

With thinking on, allow at least 8,192 output tokens. The model tends to think for a long time, even on simple prompts and on questions about itself. For short answers, turn thinking off.

System prompt

No system prompt is needed. Without one, the model introduces itself as Rina Juvenile. If a system prompt gives the assistant a different name or persona, the model was trained to follow it, though not reliably (see Limitations).

Chat template

The embedded template uses Bonsai 2's ChatML format with <think> blocks. Thinking stays on unless enable_thinking is set to false. The only change from Bonsai 2's template is how tool-call arguments in the conversation history are written: non-string values are serialized as JSON, the way Qwen3.8 does it.

Evaluation

All comparisons are against the original Bonsai 2 2B (PQ2_0), run through the same harness with the same prompts and settings.

Blind chat evaluation (Chinese and Japanese)

• Judge: Gemini 4 Argon, with reasoning effort set to high and temperature 0.
• What the judge sees: the user prompt and the two final answers (the text after </think>), labeled A and B. It does not see the reasoning, and it does not know which model wrote which answer.
• Criteria: whether the answer completes the request, factual accuracy, natural use of the user's language (other languages mixed in count against it), and helpfulness. Length is not rewarded.
• Scoring: each prompt is judged twice, with the A/B order swapped. A win counts 1, a tie 0.5 and a loss 0. A prompt's score is the average of the two judgments, and the win rate is the average over prompts. The two orders agreed on 137 of 200 prompts.
• Prompts: the 200 prompts were synthesized by DeepSeek V4.1 Flash, using the same topic and persona specification as our SFT dialog prompts. They were deduplicated against the SFT prompt pool (13-gram) and excluded from training. Even so, they are closer to Rina's training distribution than to Bonsai 2's.
• Absolute quality: the judge's comments often describe both models' answers as weak (see Limitations). This result shows that Rina is preferred to Bonsai 2. It does not show that Rina's answers are good in absolute terms.

Benchmarks

• Generation: vLLM with an FP16 expanded copy of each model, thinking on, the thinking-mode sampling settings above, up to 32,768 output tokens, one sample per item.
• Item sets: GSM8K, MMLU-Redux, CMMLU and JMMLU use fixed 500-item subsets, stratified by subject. IFEval and IFBench are the full sets.
• BFCL v3: run with EvalScope in function-calling mode, weighted by item.
• p values: paired two-sided sign tests over items that only one model got right.
• Comparability: these numbers are only comparable within this table. Prism ML's official numbers use a different setup (no presence penalty, full sets).
• IFEval caveat: our English instruction-following SFT data reuses IFEval's constraint phrasings verbatim (not its test prompts). IFEval therefore measures familiar constraint templates. IFBench uses constraints the model has not seen.

GGUF numerical check

We compared the GGUF files in llama.cpp (CUDA) with the PyTorch training model over 4,599 positions of reference text. On CUDA the three files produced bit-identical logits (compared directly over 2,044 positions; identical KL statistics over all 4,599), so one row covers all three:

Our files match the trained weights about as closely as Bonsai 2's own release matches its weights.

On CPU each packing uses different kernels, so the outputs differ slightly. We compared the three packings using llama.cpp on CPU over 2,044 positions with PQ2_0 as the reference. Bonsai 2's own three files differ from each other by the same amount:

Training

Rina Juvenile Exp keeps Bonsai 2's architecture, tokenizer and ternary format, and the model stays ternary through every stage. Training stays in Bonsai 2's Hadamard-rotated basis, so the result packs back into the same GGUF layout.

What was actually trained. Every stage was set up as quantization-aware training: full-precision latent weights are ternarized in the forward pass, and the group-wise scales are trained alongside them. After release preparation we found a bug in our straight-through estimator: under the non-reentrant gradient checkpointing used in all stages, the gradients for the latent weights of the transformer layers were dropped. In practice, the parameters that learned were:

• the group-wise scales (one per 128 weights);
• the norm weights and the other small full-precision parameters;
• the shared embedding / LM head matrix, including its latent weights.

The 6.3% of ternary values that differ from Bonsai 2 come from these changes: a new scale moves the rounding threshold for its group, and the embedding matrix was trained directly. All results on this card were measured on the released files and are unaffected. The bug is fixed for later models.

Stage 1: continued pretraining.

• Mixture: Chinese 27%, Japanese 23%, English 12%, math 8%, code 5%, replay 25%. Until step 1,108 of 2,122, replay was 10% and the other groups were correspondingly larger.
• Replay: about 107M tokens of Qwen3.5-4B answers to our SFT prompt pool.
• Effect: validation KL to the teacher fell by 34% on Chinese and 42% on Japanese.
• Decontamination: all corpora were checked against the evaluation sets with token-level 13-gram matching.

Stage 2: SFT.

• Teacher reasoning masked: reasoning written by DeepSeek V4.1 Flash is excluded from the loss. The model learns only the final answers from those samples, so it keeps Bonsai 2's own reasoning style.
• Self-generated samples: for math and English instruction following, about 17K responses were regenerated by Bonsai 2 itself. These were kept only when the answer was correct or every constraint passed, and their reasoning is trained normally.
• Distillation: every sequence is also distilled from Qwen3.8-27B's top-20 logits.

Stage 3: RL.

• Rewards: checked math answers; IFEval constraint checkers on English prompts, and similar format constraints, written in Chinese or Japanese, on chat prompts in those languages; and tool calls compared to reference calls by syntax tree.
• Tool tasks: balanced across single, parallel, sequential and no-call cases.
• Language: Chinese and Japanese tasks must be answered in the user's language.
• Length: answers between 9,216 and 12,288 tokens are penalized.
• Checkpoint choice: we trained to step 150 but release step 100. Step 150 lost 6 points on GSM8K without a significant gain anywhere else.

Stage 4: identity patch.

• Training data: the stage 3 model's own answers to identity questions, generated with a self-description prefill or system prompt and filtered for correctness. The prefill and system prompt are removed in training.
• Replay: chat, math, tool, instruction and code samples, trained with KL to the stage 3 model only and no cross-entropy. Cross-entropy on correctness-filtered replay shortened the model's reasoning and lowered instruction-following scores. KL-only replay avoided that.
• Result: benchmark scores stayed the same as stage 3 on all seven benchmarks, and the blind chat win rate also stayed the same.

Compute:

• Stages 1, 3 and 4 ran on a single RTX 5090 D: about 3.5 days, 17 hours and 1 hour.
• Stage 2 ran on a single RTX PRO 6000 in Google Colab, about 10 hours.

Limitations

• Not an overall upgrade over Bonsai 2.

  • English instruction following is clearly weaker (IFEval −6.7, IFBench −11.3).
  • Tool calling is weaker (BFCL v3 −10.7). Multi-turn tool use is close to zero, as it is for Bonsai 2.
  • English and Japanese knowledge benchmarks are slightly lower, but the gaps are not significant.
  • Prefer Bonsai 2 for English instruction following and tool use.

• Chinese and Japanese quality is still modest. The judge and our own review of the blind answers found:

  • grammar errors, especially in Japanese;
  • English or Chinese words mixed into answers;
  • invented facts, such as phone numbers and legal citations;
  • role-play answers that speak for the user.

  Do not rely on the model for medical, legal, financial or other high-stakes information.

• Long thinking. Even simple prompts can use thousands of reasoning tokens. Turn thinking off for quick answers.

• Identity is not fully reliable. These figures are for held-out identity questions with no system prompt:

  	Thinking off	Thinking on
  Gives the name Rina Juvenile	92%	72%
  Fully correct: name, developer and base model, with nothing invented	64%	14%

  • With thinking on, the reasoning often refers to a system prompt that does not exist.
  • The model sometimes invents details about its origin, or mixes Chinese into Japanese answers about itself.
  • It follows a name set in a system prompt in about two thirds of cases.

• Text only. No image input; Bonsai 2's mmproj is not supported.

• Context. Trained on sequences up to 16K tokens. Long-context behavior is untested.

• Safety. There was no dedicated safety training or red-teaming beyond what the base model and the teacher-generated data provide.

Identity

Asked about itself, the model answers that it is Rina Juvenile, developed by Rinstitute, built on Ternary Bonsai 2 2B by Prism ML, which derives from Qwen3.5-2B.

Acknowledgements and Data

Created using Bonsai by Prism ML. Rina Juvenile is built on Ternary Bonsai 2 2B (Apache 2.0, © Prism ML), which is built from Qwen3.5-2B (Apache 2.0, © Alibaba Cloud).

Teachers. We used their outputs or logits only. No teacher weights are included in this model.

• Qwen3.5-4B (Apache 2.0): stage 1 distillation and replay text.
• Qwen3.8-27B (Apache 2.0): stage 2 logits and a small number of responses.
• DeepSeek V4.1 Flash (MIT), served by Sail Research: SFT responses, synthesized prompts, and translation of math problems into Chinese and Japanese.
• Ternary Bonsai 2 2B: regenerated SFT responses for math and instruction following.

Continued-pretraining corpora. Licenses are as stated by each source. Web-derived sets are also subject to the Common Crawl terms of use.

SFT and RL prompt sources. All responses were newly generated.

Chat, tool-use, system and identity prompts were synthesized by DeepSeek V4.1 Flash from our own topic and persona specifications.

License

Apache 2.0, following Bonsai 2 and Qwen3.5. See LICENSE and NOTICE.