Oh boy, here we go. Low-rank (r=16, alpha=32) 16bit-LoRA on top of WizardLM-2-8x22B, trained on 2 epochs of (cleaned & deduped) c2-logs. As far as I can tell, this is an upgrade from WizardLM-2-8x22B for RP purposes.
Alongside this ready-to-use release I'm also releasing the LoRA itself as well as the earlier epoch1-checkpoint of the LoRA.
The choice was fully intentional. I briefly considered a FFT but for this particular use-case a LoRA seemed a better fit. WizardLM-2-8x22B is smart by itself but its used vocabulary leaves much to be desired when it comes to RP. By training a low-rank LoRA on top of it to teach it some of Claude's writing style, we remedy that.
SorcererLM-folder.gghfez has trained control vectors for this model, available at gghfez/Sorcerer_creative-writing-control-vectors
The main shoutout I want to make is to my Cartel bros, Envoid and particularly I^2, for being amazing. I count this as a team effort, so they deserve kudos too if you like this.
Trained using qlora-pipe. Configs included in the train-subfolder.
... n/a
model-00001-of-00059.safetensors 5.0 GBmodel-00025-of-00059.safetensors 5.0 GBmodel-00026-of-00059.safetensors 5.0 GBmodel-00053-of-00059.safetensors 5.0 GBmodel-00054-of-00059.safetensors 5.0 GBmodel-00027-of-00059.safetensors 4.9 GBmodel-00055-of-00059.safetensors 4.9 GBmodel-00024-of-00059.safetensors 4.9 GBmodel-00052-of-00059.safetensors 4.9 GBmodel-00012-of-00059.safetensors 4.8 GBmodel-00013-of-00059.safetensors 4.8 GBmodel-00014-of-00059.safetensors 4.8 GBmodel-00015-of-00059.safetensors 4.8 GBmodel-00016-of-00059.safetensors 4.8 GBmodel-00017-of-00059.safetensors 4.8 GBmodel-00018-of-00059.safetensors 4.8 GBmodel-00019-of-00059.safetensors 4.8 GBmodel-00020-of-00059.safetensors 4.8 GBmodel-00021-of-00059.safetensors 4.8 GBmodel-00022-of-00059.safetensors 4.8 GBmodel-00023-of-00059.safetensors 4.8 GBmodel-00033-of-00059.safetensors 4.8 GBmodel-00034-of-00059.safetensors 4.8 GBmodel-00035-of-00059.safetensors 4.8 GBmodel-00036-of-00059.safetensors 4.8 GBmodel-00037-of-00059.safetensors 4.8 GBmodel-00038-of-00059.safetensors 4.8 GBmodel-00039-of-00059.safetensors 4.8 GBmodel-00040-of-00059.safetensors 4.8 GBmodel-00041-of-00059.safetensors 4.8 GBmodel-00042-of-00059.safetensors 4.8 GBmodel-00043-of-00059.safetensors 4.8 GBmodel-00044-of-00059.safetensors 4.8 GBmodel-00045-of-00059.safetensors 4.8 GBmodel-00046-of-00059.safetensors 4.8 GBmodel-00047-of-00059.safetensors 4.8 GBmodel-00048-of-00059.safetensors 4.8 GBmodel-00049-of-00059.safetensors 4.8 GBmodel-00050-of-00059.safetensors 4.8 GBmodel-00051-of-00059.safetensors 4.8 GBmodel-00029-of-00059.safetensors 4.8 GBmodel-00030-of-00059.safetensors 4.8 GBmodel-00031-of-00059.safetensors 4.8 GBmodel-00032-of-00059.safetensors 4.8 GBmodel-00057-of-00059.safetensors 4.8 GBmodel-00058-of-00059.safetensors 4.8 GBmodel-00011-of-00059.safetensors 4.8 GBmodel-00002-of-00059.safetensors 4.8 GBmodel-00003-of-00059.safetensors 4.8 GBmodel-00004-of-00059.safetensors 4.8 GBmodel-00005-of-00059.safetensors 4.8 GBmodel-00006-of-00059.safetensors 4.8 GBmodel-00007-of-00059.safetensors 4.8 GBmodel-00008-of-00059.safetensors 4.8 GBmodel-00009-of-00059.safetensors 4.8 GBmodel-00010-of-00059.safetensors 4.8 GBmodel-00028-of-00059.safetensors 4.8 GBmodel-00056-of-00059.safetensors 4.8 GBmodel-00059-of-00059.safetensors 997 MB有文件?提交种子。
提交种子