DONT TOUCH, under dev
| Task | Version | Metric | Value | Stderr | |
|---|---|---|---|---|---|
| parus | 0 | acc | 0.4950 | ± | 0.0250 |
| rcb | 0 | acc | 0.3333 | ± | 0.0226 |
| f1_macro | 0.1667 | ||||
| rwsd | 0 | acc | 0.4901 | ± | 0.0203 |
| mmlu | 0 | 0.31 | 0.225 |
Based on https://huggingface.co/openbmb/MiniCPM-2B-sft-bf16
https://wandb.ai/alexwortega/cpm_rus/runs/32w8pv7x?workspace=user-alexwortega
lol