What is it?
LFM-2.5-1.2b-Instruct-roleplay-tuned is my attempt at making a roleplay model that anybody can run, no matter how bad the hardware is, it was tuned on 5M~ tokens of high quality roleplay data with the aim to make it better at roleplay.
Strengths and challenges:
Advantages
- Better creative writing, less slop, a little bit of repetitiveness remains due to size, fixed by samplers
- Better formatting, knows how to use asterisks really well
- Has a certain "Soul" to it, was personally pretty fun to test
- Runs literally anywhere, almost anywhere
- Insane inference speed
Disadvantages
- Context not that good
- Undestanding of complex concepts not that good
- Sensitive to quantization, Q8_0 or BF16 is recommended
- My first tune, altough I tried my best
- Because its a 1.2b dense model, its not the smartest, and may sometimes use wrong pronouns, if it was performed on any model of this size that is not LFM2.5 1.2b instruct, the results would probably be worse.
Why I made this tune?
Because I was very bored, and I wanted something that can work anywhere on virtually any hardware, my own hardware is not the best, but was enough for making this tune, I think there are a lot to improve on with this model, and I will probably work on its mistakes just when I get the time, it can still sometimes glitch a bit, especially in lower quants, but I'm still personally satisfied with the result.
Quants(Speaking from personal experience with this specific model):
- BF16- Recommended, highest quality, least logical mistakes.
- Q8_0- Recommended, high quality, makes slightly more mistakes but nonetheless near lossless.
- Q6_K- Recommended if Q8_0 is too much, degradation begins, not exactly notable here, but you will notice minor detail loss.
- Q5_K_M- Recommended if hardware is really, REALLY bad, degradation becomes noticeable.
- Q4_K_M- Not recommended for most use cases, degradation is clearly noticeable.
Quants can be found in the repository, along with safetensors.
Wow
I didnt expect this finetune to become that popular, sincere thanks to anyone reading this! V2 is already cooked up and ready for release, I lately realised that 5M~ tokens of data may have been too little for this model, I already tested the v2 and seen real improvements in the way it writes.