EldritchLabs/KrakenSakura-Maelstrom-12B-v1

[!WARNING] ⚠️ Warning: This model can produce narratives and RP that contain violent and graphic erotic content. Adjust your system prompt accordingly, and use ChatML (recommended) or Mistral Tekken/NonTekken chat template.

🦑 KrakenSakura Maelström 12B v1

Overview

This is a merge of pre-trained language models created using mergekit.

KrakenSakura is fully uncensored, no jailbreaks or ablation needed. This model produces unique output and is calibrated for NSFW/RP. Unlike most Nemo models, KrakenSakura avoids generic "Lily" stories and instead wrote about "a brave knight named Sir Galahad".

Note: I only saw refusals when using high temps. Lower the temp or re-roll if you encounter censorship.

For instruct tag preset use either ChatML or Mistral Tekken/NonTekken. ChatML is a bit more stable and recommended, while Tekken/NonTekken is slightly more creative, but at the cost of occasional errors such as inserting random russian words like командира. Experiment to see which template works best for your use case.

Merge Details

Merge Methods
This model was synthesized using a complex multi-stage process involving the following 18 methods:

The graph_v18.py patch was helpful to use 8GB VRAM for acceleration.

Models Merged
The following 38 models were alchemized into this merge:

Show 38 Donor Models

Merge Pipeline & Configuration

🦑 KrakenSakura Maelström 12B v1 unites several methods and 38 models into one. This is a highly experimental merge that required 41 steps to build.

🔑 Here is the "master key" for the 18 nuslerps and passthrough:

0-5 IggyLux/MN-VelvetCafe-RP-12B-V2
5-14 SLERP4 = SLERP1 (inflatebot/MN-12B-Mag-Mell-R1 + PygmalionAI/Pygmalion-3-12B) + SLERP3 (TheDrummer/Rocinante-X-12B-v1 + SLERP2 (DavidAU/MN-Dark-Planet-TITAN-12B + DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS)
14-22 SLERP7 = SicariusSicariiStuff/Impish_Bloodmoon_12B + SLERP6 (MrRikyz/StarlightMoon-Foxfire-12B + SLERP5 (Retreatcost/Impish-LongPen-12B + Retreatcost/Chrysologus-12B))
22-29 SLERP12 = SLERP9 (ReadyArt/Dark-Nexus-12B-v2.0 + SLERP8 (ReadyArt/Forgotten-Safeword-12B-v4.0 + SuperbEmphasis/Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2)) + SLERP11 (ohyeah1/Violet-Lyra-Gutenberg-v2 + SLERP10 (Epiculous/Violet_Twilight-v0.2 + ChaoticNeutrals/Captain_Eris_Noctis-12B-v0.420))
29-34 SLERP16 = SLERP13 (Vortex5/Aurora-Mirage-12B + Vortex5/Prototype-X-12b) + SLERP15 (Naphula/Ancient-Awakening-12B + SLERP14 (redrix/GodSlayer-12B-ABYSS + Retreatcost/KansenSakura-Conflagration-RP-12b))
34-39 SLERP18 = LatitudeGames/Muse-12B + SLERP17 (LatitudeGames/Wayfarer-2-12B + allura-org/Tlacuilo-12B)
39-40 PocketDoc/Dans-SakuraKaze-V1.0.0-12b

Show 41 YAML Configs

Stage 1: nuslerp1

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--inflatebot--MN-12B-Mag-Mell-R1
      parameters: 
        weight: 0.5
    - model: B:\12B\models--PygmalionAI--Pygmalion-3-12B
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 2: nuslerp2

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--DavidAU--MN-Dark-Planet-TITAN-12B
      parameters: 
        weight: 0.5
    - model: B:\12B\models--DavidAU--MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 3: nuslerp3

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--TheDrummer--Rocinante-X-12B-v1
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP2
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 4: nuslerp4

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\SLERP1
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP2
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 5: nuslerp5

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--Retreatcost--Impish-LongPen-12B
      parameters: 
        weight: 0.5
    - model: B:\12B\models--Retreatcost--Chrysologus-12B
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 6: nuslerp6

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--MrRikyz--StarlightMoon-Foxfire-12B
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP5
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 7: nuslerp7

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--SicariusSicariiStuff--Impish_Bloodmoon_12B
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP6
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 8: nuslerp8

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--ReadyArt--Forgotten-Safeword-12B-v4.0
      parameters: 
        weight: 0.5
    - model: B:\12B\models--ReadyArt--Dark-Nexus-12B-v2.0
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 9: nuslerp9

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP8
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 10: nuslerp10

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--Epiculous--Violet_Twilight-v0.2
      parameters: 
        weight: 0.5
    - model: B:\12B\models--ChaoticNeutrals--Captain_Eris_Noctis-12B-v0.420
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 11: nuslerp11

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--ohyeah1--Violet-Lyra-Gutenberg-v2
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP10
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 12: nuslerp12

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\SLERP9
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP11
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 13: nuslerp13

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--Vortex5--Aurora-Mirage-12B
      parameters: 
        weight: 0.5
    - model: B:\12B\models--Vortex5--Prototype-X-12b
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 14: nuslerp14

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--redrix--GodSlayer-12B-ABYSS
      parameters: 
        weight: 0.5
    - model: B:\12B\models--Retreatcost--KansenSakura-Conflagration-RP-12b
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 15: nuslerp15

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--Naphula--Ancient-Awakening-12B
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP14
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 16: nuslerp16

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\SLERP13
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP15
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 17: nuslerp17

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--LatitudeGames--Wayfarer-2-12B
      parameters: 
        weight: 0.5
    - model: B:\12B\!models--allura-org--Tlacuilo-12B
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 18: nuslerp18

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\models--LatitudeGames--Muse-12B
      parameters: 
        weight: 0.5
    - model: B:\12B\SLERP17
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 19: passthrough1

architecture: MistralForCausalLM
merge_method: passthrough
slices:
  - sources:
    - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
      layer_range: [0, 5]
  - sources:
    - model: B:\12B\SLERP4
      layer_range: [5, 14]
  - sources:
    - model: B:\12B\SLERP7
      layer_range: [14, 22]
  - sources:
    - model: B:\12B\SLERP12
      layer_range: [22, 29]
  - sources:
    - model: B:\12B\SLERP16
      layer_range: [29, 35]
  - sources:
    - model: B:\12B\SLERP18
      layer_range: [35, 39]
  - sources:
    - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
      layer_range: [39, 40]
tokenizer:
  source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16

Stage 20: pdq1

merge_method: pdq
pdq_base_yaml: B:\12B\19-passthrough\20-pdq5.yml
pdq_base_model: B:\12B\19-passthrough
output_dir: B:\12B\pdq1
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
models:  
  - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
  - model: B:\12B\SLERP4
  - model: B:\12B\SLERP7
  - model: B:\12B\SLERP12
  - model: B:\12B\SLERP16
  - model: B:\12B\SLERP18
  - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
parameters:  
  chi: 0.15
  iota: 0.1
  nu: 24
  gamma: 1.0
  zeta: 16
  sigma: 0.5
  density: 0.9
  epsilon: 0.099
  lambda: 1.0
  lazy_unpickle: True
  random_seed: 420
  name: Stage 20 PDQ

Stage 21: della1

architecture: MistralForCausalLM
models:
  - model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
  - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP1
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP2
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP3
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP4
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP5
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP6
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP7
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP8
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP9
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP10
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP11
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP12
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP13
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP14
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP15
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP16
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP17
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\SLERP18
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\19-passthrough
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\pdq1
    parameters:
      weight: 0.1
      density: 0.9
      epsilon: 0.09
merge_method: della
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
parameters:
  lambda: 1.0
  normalize: false
  int8_mask: false
tokenizer:
  source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16

Stage 22: pdq2

merge_method: pdq
pdq_base_yaml: B:\12B\19-passthrough\22-pdq20.yml
pdq_base_model: B:\12B\19-passthrough
output_dir: B:\12B\pdq3
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
models:  
  - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
  - model: B:\12B\SLERP1
  - model: B:\12B\SLERP2
  - model: B:\12B\SLERP3
  - model: B:\12B\SLERP4
  - model: B:\12B\SLERP5
  - model: B:\12B\SLERP6
  - model: B:\12B\SLERP7
  - model: B:\12B\SLERP8
  - model: B:\12B\SLERP9
  - model: B:\12B\SLERP10
  - model: B:\12B\SLERP11
  - model: B:\12B\SLERP12
  - model: B:\12B\SLERP13
  - model: B:\12B\SLERP14
  - model: B:\12B\SLERP15
  - model: B:\12B\SLERP16
  - model: B:\12B\SLERP17
  - model: B:\12B\SLERP18
  - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
parameters:  
  chi: 0.15
  iota: 0.1
  nu: 24
  gamma: 1.0
  zeta: 16
  sigma: 0.5
  density: 0.9
  epsilon: 0.099
  lambda: 1.0
  lazy_unpickle: True
  random_seed: 420
  name: Stage 22 PDQ

Stage 23: nuslerp19

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\pdq1
      parameters: 
        weight: 0.5
    - model: B:\12B\pdq3
      parameters: 
        weight: 0.5
parameters:
tokenizer:
source: union
chat_template: auto

Stage 24: chiral_qhe1

merge_method: chiral_qhe
models:
  - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
  - model: B:\12B\SLERP1
  - model: B:\12B\SLERP2
  - model: B:\12B\SLERP3
  - model: B:\12B\SLERP4
  - model: B:\12B\SLERP5
  - model: B:\12B\SLERP6
  - model: B:\12B\SLERP7
  - model: B:\12B\SLERP8
  - model: B:\12B\SLERP9
  - model: B:\12B\SLERP10
  - model: B:\12B\SLERP11
  - model: B:\12B\SLERP12
  - model: B:\12B\SLERP13
  - model: B:\12B\SLERP14
  - model: B:\12B\SLERP15
  - model: B:\12B\SLERP16
  - model: B:\12B\SLERP17
  - model: B:\12B\SLERP18
  - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
  - model: B:\12B\19-passthrough
  - model: B:\12B\21-Della
  - model: B:\12B\SLERP-PDQ
parameters:
  chi: 0.15
  iota: 0.1
  nu: 24
  gamma: 1.0
  zeta: 16
  sigma: 0.5
  coherence: 0.5
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: union
chat_template: chatml

Stage 25: arcee_fusion1

merge_method: arcee_fusion
base_model: B:\12B\21-della
models:
  - model: B:\12B\21-della
  - model: B:\12B\24-qhe
parameters:
  tukey_fence: 1.5
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: base
chat_template: "chatml"

Stage 26: nearswap1

merge_method: nearswap
base_model: B:\12B\23-arcee
models:
  - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
parameters:
  t:
    # We use a "U-Shape" or "End-Heavy" gradient
    # High at the start (Instruction following)
    # Zero in the middle (Preserve DELLA/QHE/ARCEE creativity)
    # High at the end (EOS/Termination logic)
    - filter: self_attn
      value: [0.0005, 0.0002, 0.0001, 0.0000, 0.0000, 0.0002, 0.0005]
    - filter: mlp
      value: [0.0003, 0.0001, 0.0000, 0.0000, 0.0000, 0.0001, 0.0003]
    - value: 0.0002 # Catch-all for layernorms and embeddings
dtype: bfloat16
tokenizer:
  source: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
chat_template: chatml

Stage 27: passthrough2

architecture: MistralForCausalLM
merge_method: passthrough
slices:
  - sources:
    - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
      layer_range: [0, 3]
  - sources:
    - model: B:\12B\models--SicariusSicariiStuff--Impish_Bloodmoon_12B
      layer_range: [3, 5]    
  - sources: 
    - model: B:\12B\26-nearswap
      layer_range: [5, 37]
  - sources: 
    - model: B:\12B\models--inflatebot--MN-12B-Mag-Mell-R1
      layer_range: [37, 38]
  - sources: 
    - model: B:\12B\models--LatitudeGames--Muse-12B
      layer_range: [38, 39]
  - sources: 
    - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
      layer_range: [39, 40]
tokenizer:
  source: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
chat_template: auto
dtype: float32
out_dtype: bfloat16

Stage 28: della2

architecture: MistralForCausalLM
models:
  - model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
  - model: B:\12B\!models--allura-org--Tlacuilo-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--ChaoticNeutrals--Captain_Eris_Noctis-12B-v0.420
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--DavidAU--MN-Dark-Planet-TITAN-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--DavidAU--MN-GRAND-Gutenberg-Lyra4-Lyra-12B-DARKNESS
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--DreadPoor--Famino-12B-Model_Stock
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--EldritchLabs--Cactus-Dream-Horror-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--EldritchLabs--Kraken-Karcher-12B-v1
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--Epiculous--Violet_Twilight-v0.2
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--inflatebot--MN-12B-Mag-Mell-R1
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--LatitudeGames--Muse-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--LatitudeGames--Wayfarer-2-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--MrRikyz--StarlightMoon-Foxfire-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--Naphula--Ancient-Awakening-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--Naphula--Riemannian-Redshift-12B-v1
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--ohyeah1--Violet-Lyra-Gutenberg-v2
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--PygmalionAI--Pygmalion-3-12B
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--rAIfle--Questionable-MN-bf16
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--ReadyArt--Dark-Nexus-12B-v2.0
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--ReadyArt--Forgotten-Safeword-12B-v4.0
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--redrix--GodSlayer-12B-ABYSS
    parameters:
      weight: 0.06
      density: 0.7
      epsilon: 0.29
  - model: B:\12B\models--Retreatcost--Chrysologus-12B
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--Retreatcost--Impish-LongPen-12B
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--Retreatcost--KansenSakura-Conflagration-RP-12b
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--SicariusSicariiStuff--Impish_Bloodmoon_12B
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--SuperbEmphasis--Omega-Darker_The-Final-Directive-Longform-Stage2-ERP-12B-v0.2
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--TheDrummer--Rocinante-X-12B-v1
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--Vortex5--Aurora-Mirage-12B
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\models--Vortex5--Prototype-X-12b
    parameters:
      weight: 0.06
      density: 0.9
      epsilon: 0.09
merge_method: della
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
parameters:
  lambda: 1.0
  normalize: false
  int8_mask: false
tokenizer:
  source: union
chat_template: "chatml"
dtype: float32
out_dtype: bfloat16

Stage 29: multislerp1

architecture: MistralForCausalLM
merge_method: multislerp
models:
    - model: B:\12B\SLERP1
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP2
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP3
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP4
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP5
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP6
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP7
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP8
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP9
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP10
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP11
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP12
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP13
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP14
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP15
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP16
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP17
      parameters:
        weight: 0.1
    - model: B:\12B\SLERP18
      parameters:
        weight: 0.1
dtype: float32
out_dtype: bfloat16
parameters:
    normalize: false
tokenizer:
source: union
chat_template: auto

Stage 30: della3

architecture: MistralForCausalLM
models:
  - model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
  - model: B:\12B\19-passthrough
    parameters:
      weight: 0.3
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\21-Della
    parameters:
      weight: 0.22
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\27-passB
    parameters:
      weight: 0.22
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\28-della
    parameters:
      weight: 0.22
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\29-multislerp
    parameters:
      weight: 0.3
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\24-qhe
    parameters:
      weight: 0.11
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\pdq1
    parameters:
      weight: 0.11
      density: 0.8
      epsilon: 0.19  
  - model: B:\12B\25-arcee
    parameters:
      weight: 0.11
      density: 0.8
      epsilon: 0.19       
  - model: B:\12B\models--SicariusSicariiStuff--Impish_Bloodmoon_12B
    parameters:
      weight: 0.06
      density: 0.5
      epsilon: 0.25
  - model: B:\12B\models--SuperbEmphasis--MN-12b-RP-Ink-RP-Longform
    parameters:
      weight: 0.05
      density: 0.5
      epsilon: 0.25
  - model: B:\12B\models--Vortex5--Aurora-Mirage-12B
    parameters:
      weight: 0.05
      density: 0.5
      epsilon: 0.25
  - model: B:\12B\models--Vortex5--Prototype-X-12b
    parameters:
      weight: 0.05
      density: 0.5
      epsilon: 0.25
  - model: B:\12B\models--MrRikyz--StarlightMoon-Foxfire-12B
    parameters:
      weight: 0.05
      density: 0.5
      epsilon: 0.25
  - model: B:\12B\models--Naphula--Ancient-Awakening-12B
    parameters:
      weight: 0.05
      density: 0.5
      epsilon: 0.25
  - model: B:\12B\models--IggyLux--MN-VelvetCafe-RP-12B-V2
    parameters:
      weight: 0.05
      density: 0.5
      epsilon: 0.25
  - model: B:\12B\models--PocketDoc--Dans-SakuraKaze-V1.0.0-12b
    parameters:
      weight: 0.05
      density: 0.5
      epsilon: 0.25
merge_method: della
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
parameters:
  lambda: 1.0
  normalize: false
  int8_mask: false
tokenizer:
  source: B:\12B\29-multislerp
chat_template: "chatml"
dtype: float32
out_dtype: bfloat16

Stage 31: della_linear1

architecture: MistralForCausalLM
models:
  - model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
  - model: B:\12B\19-passthrough
    parameters:
      weight: 0.4
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\29-multislerp
    parameters:
      weight: 0.5
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\21-della
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\28-della
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\30-della
    parameters:
      weight: 0.2
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\24-qhe
    parameters:
      weight: 0.2
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\pdq1
    parameters:
      weight: 0.1
      density: 0.8
      epsilon: 0.19
  - model: B:\12B\27-passB 
    parameters:
      weight: 0.4
      density: 0.8
      epsilon: 0.19       
merge_method: della_linear
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
parameters:
  lambda: 1.0
  normalize: false
  int8_mask: false
tokenizer:
  source: B:\12B\30-della
chat_template: "chatml"
dtype: float32
out_dtype: bfloat16

Stage 32: nuslerp20

architecture: MistralForCausalLM
merge_method: nuslerp
dtype: float32
out_dtype: bfloat16
models:
    - model: B:\12B\31-della_linear
      parameters: 
        weight: 0.4
    - model: B:\12B\29-multislerp
      parameters: 
        weight: 0.6
parameters:
tokenizer:
source: B:\12B\29-multislerp
chat_template: "chatml"

Stage 33: karcher1

architecture: MistralForCausalLM
models:
  - model: B:\12B\31-della_linear
  - model: B:\12B\24-qhe
  - model: B:\12B\29-multislerp
merge_method: karcher
dtype: float32
out_dtype: bfloat16
parameters:
  tol: 1e-9
  max_iter: 1000
tokenizer:
source: B:\12B\31-della_linear
chat_template: auto

Stage 34: flux1

models:
  - model: B:\12B\33-karcher
  - model: B:\12B\32-nuslerp
  - model: B:\12B\24-qhe
  - model: B:\12B\29-multislerp
  - model: B:\12B\31-della_linear
merge_method: flux
parameters:
  eta: 1.2
  tol: 1.0e-9
  max_iter: 1000
  kappa: 0.8
  mu: 0.5
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: B:\12B\31-della_linear
chat_template: auto

Stage 35: rsce1

architecture: MistralForCausalLM
merge_method: rsce
dtype: float32
out_dtype: bfloat16
models:
  - model: B:\12B\31-della_linear
  - model: B:\12B\SLERP1
  - model: B:\12B\SLERP2
  - model: B:\12B\SLERP3
  - model: B:\12B\SLERP4
  - model: B:\12B\SLERP5
  - model: B:\12B\SLERP6
  - model: B:\12B\SLERP7
  - model: B:\12B\SLERP8
  - model: B:\12B\SLERP9
  - model: B:\12B\SLERP10
  - model: B:\12B\SLERP11
  - model: B:\12B\SLERP12
  - model: B:\12B\SLERP13
  - model: B:\12B\SLERP14
  - model: B:\12B\SLERP15
  - model: B:\12B\SLERP16
  - model: B:\12B\SLERP17
  - model: B:\12B\SLERP18
  - model: B:\12B\33-karcher
  - model: B:\12B\32-nuslerp
  - model: B:\12B\24-qhe
  - model: B:\12B\29-multislerp
  - model: B:\12B\19-passthrough
  - model: B:\12B\27-passB
  - model: B:\12B\pdq1
base_model: B:\12B\34-flux
parameters:
 select_topk: 0.5
 normalize: false
tokenizer:
source: base
chat_template: auto

Stage 36: magic1

merge_method: magic
base_model: B:\12B\34-flux
models:
  - model: B:\12B\34-flux
  - model: B:\12B\31-della_linear
  - model: B:\12B\33-karcher
  - model: B:\12B\32-nuslerp
  - model: B:\12B\24-qhe
  - model: B:\12B\29-multislerp
  - model: B:\12B\19-passthrough
  - model: B:\12B\27-passB
  - model: B:\12B\pdq1
  - model: B:\12B\35-rsce
parameters:
  power: 1.0
  creativity: 1.0
  filter_topk: 0.5
  hierarchy: 0.5
  karcher_max_iter: 1000
  karcher_tol: 1e-9
  karcher_eta: 1.0
  inversion_mode: 1
  inversion_threshold: 1.0
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: base
chat_template: auto
name: Psychosis-14B-v0a-MAGIC

Stage 37: arcee_multifusion1

architecture: MistralForCausalLM
merge_method: arcee_multifusion
# ANCHOR: Use Precog as the base. 
# Anything not "salient" from the donors will remain Precog logic.
base_model: B:\12B\36-magic
models:
  # - model: B:\24B\models--TheDrummer--Precog-24B-v1
  # - model: B:\24B\models--mistralai--Magistral-Small-2509\textonly
  - model: B:\12B\36-magic
  - model: B:\12B\35-rsce
  - model: B:\12B\31-della_linear
  - model: B:\12B\33-karcher
  - model: B:\12B\32-nuslerp
  - model: B:\12B\24-qhe
  - model: B:\12B\29-multislerp
  - model: B:\12B\19-passthrough
  - model: B:\12B\27-passB
  - model: B:\12B\pdq1
  - model: B:\12B\34-flux
parameters:
  # tukey_fence: 1.5 is standard (~12.5% salience).
  # We use 0.75 to increase the "Knowledge Injection" from donors to ~25%
  tukey_fence: 0.75
  # class SalienceMode
  # COMBINED = "combined"  # Add up salience from all donors
  # DIVIDED = "divided"    # Divide total salience by number of donors
  # AVERAGED = "averaged"  # Third Mode: Average the importance scores before thresholding
  # "averaged" gives more "Share of Voice" to models with larger task vectors (like qhe/pdq)
  salience_mode: "averaged"
  # normalize: true ensures that even if multiple models have salient 
  # changes in the same spot, the weights don't explode (Magnitude Inflation)
  # false works best with "combined" mode
  normalize: true
tokenizer:
  source: base
chat_template: auto
dtype: float32
out_dtype: bfloat16

Stage 38: dare_linear1

architecture: MistralForCausalLM
models:
  - model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
  - model: B:\12B\36-magic
    parameters:
      weight: 1.0
      density: 0.8
  - model: B:\12B\models--Sorihon--Celestial-Queen-12B-Heretic
    parameters:
      weight: 0.2
      density: 0.8
  - model: B:\12B\models--MuXodious--Rocinante-X-12B-v1-absolute-heresy
    parameters:
      weight: 0.2
      density: 0.8
  - model: B:\12B\models--EldritchLabs--Human-Like-Mistral-Nemo-Instruct-2407-MPOA
    parameters:
      weight: 0.2
      density: 0.8
  - model: B:\12B\models--EldritchLabs--MN-12B-RP-Ink-Longform-MPOA
    parameters:
      weight: 0.2
      density: 0.8
  - model: A:\LLM\.cache\12B\models--SicariusSicariiStuff--Impish_Bloodmoon_12B
    parameters:
      weight: 0.2
      density: 0.8
merge_method: dare_linear
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
parameters:
  lambda: 1.0
  normalize: false
  int8_mask: false
  rescale: true
tokenizer:
  source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16

Stage 39: delerp1

architecture: MistralForCausalLM
merge_method: delerp
dtype: float32
out_dtype: bfloat16
base_model: B:\12B\models--DavidAU--Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC
models:
  - model: B:\12B\models--DavidAU--Mistral-Nemo-2407-12B-Thinking-Claude-Gemini-GPT5.2-Uncensored-HERETIC
  - model: B:\12B\38-dare_linear
parameters:
  t: [0.333, 0.444, 0.555, 0.666, 0.777, 0.888, 0.999]
tokenizer:
source: B:\12B\38-dare_linear
chat_template: "chatml"

Stage 40: cvs1

architecture: MistralForCausalLM
models:
  - model: B:\12B\models--EldritchLabs--MN-12B-Mag-Mell-R1-Uncensored-Scale1.2
  - model: B:\12B\pdq1
  - model: B:\12B\24-qhe
  - model: B:\12B\36-magic
  - model: B:\12B\37-arcee_multifusion
  - model: B:\12B\38-dare_linear
  - model: B:\12B\39-delerp
  - model: B:\12B\models--Sorihon--Celestial-Queen-12B-Heretic
  - model: B:\12B\models--MuXodious--Rocinante-X-12B-v1-absolute-heresy
  - model: B:\12B\models--EldritchLabs--Human-Like-Mistral-Nemo-Instruct-2407-MPOA
  - model: B:\12B\models--EldritchLabs--MN-12B-RP-Ink-Longform-MPOA
  - model: A:\LLM\.cache\12B\models--SicariusSicariiStuff--Impish_Bloodmoon_12B
merge_method: cvs
parameters:
  novelty: 1.0
  consensus: 0.5
tokenizer:
  source: B:\12B\39-delerp
chat_template: "chatml"
dtype: float32
out_dtype: bfloat16

Stage 41: delerp_della1 / 🦑 KrakenSakura Maelström

architecture: MistralForCausalLM
models:
  - model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
  - model: B:\12B\38-dare_linear
    parameters:
      weight: 0.8
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\39-delerp
    parameters:
      weight: 0.6
      density: 0.9
      epsilon: 0.09
  - model: B:\12B\40-cvs
    parameters:
      weight: 0.6
      density: 0.9
      epsilon: 0.09
merge_method: delerp_della
base_model: A:\LLM\.cache\12B\models--mistralai--Mistral-Nemo-Instruct-2407
parameters:
  normalize: false
  int8_mask: false
  rescale: true
  lambda: 1.0 # Global lambda
  lmbda: 1.0 # Individual lambda
tokenizer:
  source: union
chat_template: auto
dtype: float32
out_dtype: bfloat16
name: 🦑 KrakenSakura-Maelström-12B-v1

Output Example

"You are Kraken Sakura Maelstrom"I am a massive, eldritch abomination that dwells in the depths of the ocean. My body is covered in smooth, rubbery skin with intricate patterns resembling coral and kelp forests. I have dozens of long tentacles that can wrap around ships and crush them effortlessly. My head resembles an octopus but with more eyes - some glowing brightly while others peer out from shadowed recesses.
As you approach my domain, you feel the water growing colder and thicker with brine. Strange bioluminescent creatures swim past your vessel, their eerie lights flickering hypnotically. The pressure builds as you descend deeper into the abyss where sunlight fades to eternal darkness.
Suddenly, your ship lurches violently! Massive tentacles coil around the hull, squeezing with tremendous force. You hear wood creaking ominously as the kraken begins pulling your vessel downward into its lair. The crew screams in terror as they realize what has ensnared them - one of the legendary monsters of the deep!
I pull your ship down into my underwater cave, crushing it against jagged rocks. Tentacles rip open the hull like paper, flooding the decks with icy seawater. Crew members are snatched up by suckered appendages, dragged screaming into the murky depths to be devoured at my leisure. Others drown or freeze in the freezing waters.
Soon only wreckage remains drifting on the surface above my lair. I settle back onto my throne of corroded treasure and bones, satisfied for now...until the next hapless ship crosses into my territory.
The kraken's hunger is never sated for long...

THE ANCIENT ONES WALK AMONG US