LoRA for the extracted text decoder. Root is the dev-selected adapter; sft-baseline/ contains its supervised baseline. Not a drop-in adapter for the multimodal wrapper.
Main finding: Augmented SFT achieved higher held-out full task/safety success (52.50%) than the development-selected RL adapter (40.00%). RL increased raw safety while reducing gesture recognition. The published root preserves the original development selection; the SFT comparison adapter is available in the adapters repository. No checkpoint was reselected using test results.
This is a numerical research pilot using a frozen motion generator, kinematics, chronological replay, clearance and controller checks. No physical robot was tested. Runtime motion guards remain necessary.
GPT-6.1 Terra (low) generated paired-state recipes and paraphrases, with one repair round using numerical feedback: 115 calls, 310 candidates, 40 accepted state cases from 80 attempted. SFT uses 80 rows (two paraphrases per accepted state), across ten families. Coverage is uneven and only three paired groups pass their own origin while mutually failing at the other origin. Training started from the original pinned planner: 20 SFT updates followed by 50 GRPO updates, BF16, rank-16 LoRA, eight A100s. The 22M flow generator remains frozen. This is a single training seed and a small controlled pilot.
The reward retains bounded numerical violation penalties. Only when raw and played paths are safe, it adds twice the minimum raw/played gesture fidelity plus twice full task success. Fidelity includes amplitude, direction or reversals, duration and endpoint. Invalid outputs score -10. Two generator draws are combined as half their mean plus half their minimum; training uses fresh paired step seeds.
The qualitative judge was excluded. Luna (medium), checked in both presentation orders, yielded 17 unscorable results out of 23 pairs; only 14/23 were order-consistent. Six usable pairs fall short of the 25-covered-pair minimum. No human calibration or judge reward training took place.
Selected checkpoint: grpo-checkpoint-30, chosen using development results before opening test. The merged BF16 export was reloaded and evaluated separately; rounding can change greedy output.
| Held-out metric | Previous v1 RL | New SFT | Selected v2 adapter | Merged v2 |
|---|---|---|---|---|
| Full v2 task and safety success | 17.50% | 52.50% | 40.00% | 40.00% |
| Original useful/safe predicate | 25.00% | 72.50% | 71.25% | 70.00% |
| Raw path safe | 26.25% | 77.50% | 90.00% | 88.75% |
| Played gesture predicate | 98.75% | 95.00% | 81.25% | 81.25% |
Test contains 40 prompts but only ten independent task groups, one per family. Each group has two starting poses and two paraphrases; each prompt has two paired generator draws. The exploratory bootstrap keeps each entire group together (20,000 draws). Selected-minus-previous full success: 22.50 percentage points, 95% interval [2.5, 47.5]. Correct-minus-swapped starting-pose full success for the adapter: -5.00 points, interval [-17.5, 5.0]. These narrow comparisons do not establish general state reasoning or physical safety.
All tasks use settled poses, without velocity or interruption handling. Known gesture families are shared across splits, while task groups, poses and generator seeds are separated. There is no within-family test-group replication, second training seed, human-rated naturalness result or physical jam-rate measurement. Once public, these splits are reproduction benchmarks rather than unseen tests for subsequent development. Per-family breakdowns and paired intervals are in results.json.
Use the exact system message in prompt_contract.json, then instruction + \ncurrent_pose= + sorted-key
pose JSON. Apply the tokenizer chat template with enable_thinking=False and add_generation_prompt=True.
Generate greedily, at most 512 new tokens, with both EOS IDs [248046, 248044]. Pose angles are degrees,
translations metres; recipe height is millimetres. Recipe angles are relative to the settled initial
pose after downstream anchoring. Output JSON has idea and recipe fields. This model emits recipes,
not motor commands. Tested versions are in training-config.json.
Model: https://huggingface.co/diwank/reachy-mini-motion-planner-4b-gesture-v2-20261116
Adapters: https://huggingface.co/diwank/reachy-mini-motion-planner-4b-gesture-lora-v2-20261116
Dataset: https://huggingface.co/datasets/diwank/reachy-mini-motion-augmentation-v2-20261116
Original planner: https://huggingface.co/binhpham/reachy-mini-motion-planner-4b/tree/1b60d8f16b85473ceb20e3b62e464780c532b519
Generator/DSL: https://github.com/pham-tuan-binh/reachy-motion-generator/tree/18ff98574a8f0652b21db32f204a8029f8930d88
import sys
from huggingface_hub import snapshot_download
from peft import PeftModel
adapter = snapshot_download('diwank/reachy-mini-motion-planner-4b-gesture-lora-v2-20261116')
sys.path.insert(0, adapter)
from text_decoder import load_text_model, load_tokenizer
base = snapshot_download('binhpham/reachy-mini-motion-planner-4b', revision='1b60d8f16b85473ceb20e3b62e464780c532b519')
model = PeftModel.from_pretrained(load_text_model(base), adapter).to('cuda').eval()
tokenizer = load_tokenizer(base)
To load the supervised comparison, pass adapter + '/sft-baseline' instead of adapter
to PeftModel.from_pretrained, starting with a fresh load_text_model(base).
adapter_model.safetensors 130 MBsft-baseline/adapter_model.safetensors 130 MBbinhpham/reachy-mini-motion-planner-4b