This is a merge of pre-trained language models created using mergekit, but using my experimental branch swapping
The "Other" branch is where I did something wrong, so in the "main" I did it right(I hope 😅).
This model was merged using the task_swapping merge method not sure if I did it right and how the model was impacted, using NousResearch/Yarn-Mistral-7b-128k as a base.
The following models were included in the merge:
The following YAML configuration was used to produce this model:
base_model:
model:
path: NousResearch/Yarn-Mistral-7b-128k
dtype: bfloat16
merge_method: task_swapping
slices:
- sources:
- layer_range: [0, 32]
model:
model:
path: senseable/WestLake-7B-v2
parameters:
weight: 0.666
diagonal_offset: 2
invert_offset: True
- layer_range: [0, 32]
model:
model:
path: NousResearch/Yarn-Mistral-7b-128k
model-00028-of-00042.safetensors 394 MBmodel-00018-of-00042.safetensors 394 MBmodel-00023-of-00042.safetensors 394 MBmodel-00034-of-00042.safetensors 394 MBmodel-00039-of-00042.safetensors 394 MBmodel-00007-of-00042.safetensors 394 MBmodel-00012-of-00042.safetensors 394 MBmodel-00017-of-00042.safetensors 394 MBmodel-00022-of-00042.safetensors 394 MBmodel-00027-of-00042.safetensors 394 MBmodel-00033-of-00042.safetensors 394 MBmodel-00038-of-00042.safetensors 394 MBmodel-00006-of-00042.safetensors 394 MBmodel-00011-of-00042.safetensors 394 MBmodel-00002-of-00042.safetensors 386 MBmodel-00029-of-00042.safetensors 380 MBmodel-00014-of-00042.safetensors 380 MBmodel-00001-of-00042.safetensors 369 MBmodel-00041-of-00042.safetensors 352 MBmodel-00020-of-00042.safetensors 319 MBmodel-00025-of-00042.safetensors 319 MBmodel-00031-of-00042.safetensors 319 MBmodel-00036-of-00042.safetensors 319 MBmodel-00004-of-00042.safetensors 319 MBmodel-00009-of-00042.safetensors 319 MBmodel-00016-of-00042.safetensors 319 MBmodel-00021-of-00042.safetensors 319 MBmodel-00026-of-00042.safetensors 319 MBmodel-00032-of-00042.safetensors 319 MBmodel-00037-of-00042.safetensors 319 MBmodel-00005-of-00042.safetensors 319 MBmodel-00010-of-00042.safetensors 319 MBmodel-00019-of-00042.safetensors 319 MBmodel-00024-of-00042.safetensors 319 MBmodel-00030-of-00042.safetensors 319 MBmodel-00035-of-00042.safetensors 319 MBmodel-00040-of-00042.safetensors 319 MBmodel-00003-of-00042.safetensors 319 MBmodel-00008-of-00042.safetensors 319 MBmodel-00013-of-00042.safetensors 319 MBmodel-00015-of-00042.safetensors 285 MBmodel-00042-of-00042.safetensors 117 MBNousResearch/Yarn-Mistral-7b-128kHave the files? Submit a torrent.
Submit a torrentHave the complete files and permission to share them? Replace the example folder path. The script checks the files, makes a torrent and submits its magnet and checksums. Keep your torrent client seeding. Get your community key from your settings, and read the script before running it.
curl -fsSL https://pirateface.co/package.sh | bash -s -- --repo Aryanne/YarnLake-Swap-7B ./model-folder