Mistral-7B-v0.3-stepbasin-books-20480

This model is a fine-tuned version of mistralai/Mistral-7B-v0.3 on this dataset for the purpose of testing out super-long text generation.

• fine-tuned at context length 20480, should consistently generate 8k+ tokens (example)

---

It achieves the following results on the evaluation set:

• Loss: 2.0784
• Accuracy: 0.5396
• Num Input Tokens Seen: 16384000