Myosotis-1.1 (pretrain step 4000)
The model is heavily undertrained, but for who cares theres the eval
| Tasks | Version | Filter | n-shot | Metric | Value | Stderr | ||
|---|---|---|---|---|---|---|---|---|
| arc_challenge | 1 | none | 0 | acc | ↑ | 0.1630 | ± | 0.0108 |
| none | 0 | acc_norm | ↑ | 0.2125 | ± | 0.0120 | ||
| arc_easy | 1 | none | 0 | acc | ↑ | 0.3030 | ± | 0.0094 |
| none | 0 | acc_norm | ↑ | 0.2879 | ± | 0.0093 | ||
| hellaswag | 1 | none | 0 | acc | ↑ | 0.2562 | ± | 0.0044 |
| none | 0 | acc_norm | ↑ | 0.2472 | ± | 0.0043 | ||
| piqa | 1 | none | 0 | acc | ↑ | 0.5435 | ± | 0.0116 |
| none | 0 | acc_norm | ↑ | 0.5277 | ± | 0.0116 |