This model is ModelCloud/tinyllama-15M-stories repeated 4 times to make 4 experts.
The model is used for testing, not intended to be used in production (unless your product is some kind of bedtime story teller)
Weight of router is initialized randomly
A LoRA adapter trained on first 100 paragraphs of shakespeare can be found inside moe_shakespeare15M
With input: Look in thy glass
Look in thy glass was a little girl. She was only three years old and she was three years old. She wasLook in thy glass in love of the eye: That's when when the eye see thy on the sun'model.safetensors 73 MBmoe_shakespeare15M/checkpoint-400/adapter_model.safetensors 68 MBmoe_shakespeare15M/checkpoint-500/adapter_model.safetensors 68 MBmoe_shakespeare15M/checkpoint-400/optimizer.pt 63 MBmoe_shakespeare15M/checkpoint-500/optimizer.pt 63 MBmoe_shakespeare15M/checkpoint-400/rng_state.pth 14.2 kBmoe_shakespeare15M/checkpoint-500/rng_state.pth 14.2 kBmoe_shakespeare15M/checkpoint-400/training_args.bin 5.1 kBmoe_shakespeare15M/checkpoint-500/training_args.bin 5.1 kBmoe_shakespeare15M/checkpoint-400/scheduler.pt 1.1 kBmoe_shakespeare15M/checkpoint-500/scheduler.pt 1.1 kB