Hummingbird

🎉 You are looking at Hummingbird 3.1, which adds special chat template tokens!

Hummingbird is a Kitten derivative trained to be conversational.

Training

The model was trained using 500 steps, and 4 batch size.

Datasets

The training corpus is made up of:

• First 1400 rows of qikp/reborn-5k-no-thoughts (a collate of Dolly)
• First 500 rows of HuggingFaceTB/smol-smoltalk
• First 100 rows of HuggingFaceTB/everyday-conversations-llama3.1-2k

The train / train_sft splits were used.

Chat template

The Zephyr chat template was used, but most notably, chat template tokens were added to enhance performance.

Limitations

The model frequently outputs incorrect information, confirmation with a larger, mature model is advised.