- Using
opencsg/csg-wukong-2b-chinese-fineweb-eduas base model, we fine-tune it onsmoltalk-chinesefor 2 epoch - learning rate = 3e-4 ; global batch size = 32 ; lr scheduler=cosine
opencsg/csg-wukong-2b-chinese-fineweb-edu as base model, we fine-tune it on smoltalk-chinese for 2 epoch