Dumb-1.2-RC1

new SoTA dumb

This model was trained with a high-quality data set of 34.6M parameters.

• training token:
  • 0625→RC1: -90M
  • 0616 RC1: 160-200M

What this model can do

• text generation
• Predict the following words
• Explain simple questions in an interesting way

What this model can't do

• calculate arithmetic
• math
• Fluent conversation
• Don't lie as much as possible

benchmark

vs 1-20M param

vs 20M~ param

1: (XX%) is Calculated with our ai / better ai what percentage of the score is for what is better.

If you look at these two tables, BLiMP and PPL are catching up with the competition, but ARC-Easy is not very powerful. In addition, it is 14% higher in wikitext than michel-nano (22M param). However, this model is still a preview version, detailed benchmarks have not been carried out, and the official version may be more powerful. In addition, the low score of ARC-Easy indicates that you don't know much about scientific knowledge, and you can see that you should train using scientific data sets.

In addition, it is completely inferior to the more advanced Supra model (some items are comparable, but not completely catching up)