Lunar-Trace 31B

Overview

Lunar-Trace-31B was created by combining gemma-4-31B-it, Starlight-Synthesis-31B, G4-LuneLight-31B, and gemma-4-31B-it-Claude-Opus-Distill-v2 using a custom merge method.

View merge configuration · YAML

base_model: google/gemma-4-31B-it
merge_method: mdr
models:
  - model: Cyclone-Labs/Starlight-Synthesis-31B
    parameters:
      weight:
        - filter: vision_tower
          value: 0.0
        - filter: embed_vision
          value: 0.0
        - filter: self_attn.q_proj
          value: [0.46, 0.47, 0.48, 0.49, 0.50, 0.51, 0.52, 0.53]
        - filter: self_attn.k_proj
          value: [0.47, 0.48, 0.49, 0.50, 0.51, 0.52, 0.53, 0.54]
        - filter: self_attn.v_proj
          value: [0.50, 0.52, 0.54, 0.56, 0.58, 0.60, 0.62, 0.64]
        - filter: self_attn.o_proj
          value: [0.52, 0.54, 0.56, 0.59, 0.61, 0.64, 0.66, 0.68]
        - filter: mlp.gate_proj
          value: [0.64, 0.67, 0.70, 0.72, 0.74, 0.76, 0.78, 0.80]
        - filter: mlp.up_proj
          value: [0.63, 0.65, 0.68, 0.71, 0.74, 0.76, 0.78, 0.80]
        - filter: mlp.down_proj
          value: [0.62, 0.64, 0.67, 0.70, 0.72, 0.75, 0.78, 0.80]
        - filter: self_attn.q_norm
          value: 0.60
        - filter: self_attn.k_norm
          value: 0.60
        - filter: layernorm
          value: 0.65
        - filter: layer_scalar
          value: 0.70
        - filter: language_model.norm
          value: 0.65
        - value: 0.33
  - model: zerofata/G4-LuneLight-31B
    parameters:
      weight:
        - filter: vision_tower
          value: 0.0
        - filter: embed_vision
          value: 0.0
        - filter: self_attn.q_proj
          value: [0.23, 0.23, 0.23, 0.22, 0.21, 0.20, 0.19, 0.18]
        - filter: self_attn.k_proj
          value: [0.24, 0.24, 0.23, 0.22, 0.21, 0.20, 0.19, 0.18]
        - filter: self_attn.v_proj
          value: [0.34, 0.33, 0.32, 0.31, 0.30, 0.29, 0.28, 0.27]
        - filter: self_attn.o_proj
          value: [0.35, 0.34, 0.33, 0.30, 0.28, 0.25, 0.23, 0.21]
        - filter: mlp.gate_proj
          value: [0.26, 0.23, 0.20, 0.18, 0.16, 0.14, 0.12, 0.10]
        - filter: mlp.up_proj
          value: [0.27, 0.25, 0.22, 0.19, 0.16, 0.14, 0.12, 0.10]
        - filter: mlp.down_proj
          value: [0.28, 0.26, 0.23, 0.20, 0.18, 0.15, 0.12, 0.10]
        - filter: self_attn.q_norm
          value: 0.40
        - filter: self_attn.k_norm
          value: 0.40
        - filter: layernorm
          value: 0.35
        - filter: layer_scalar
          value: 0.30
        - filter: language_model.norm
          value: 0.35
        - value: 0.33
  - model: TeichAI/gemma-4-31B-it-Claude-Opus-Distill-v2
    parameters:
      weight:
        - filter: vision_tower
          value: 0.0
        - filter: embed_vision
          value: 0.0
        - filter: self_attn.q_proj
          value: [0.31, 0.30, 0.29, 0.29, 0.29, 0.29, 0.29, 0.29]
        - filter: self_attn.k_proj
          value: [0.29, 0.28, 0.28, 0.28, 0.28, 0.28, 0.28, 0.28]
        - filter: self_attn.v_proj
          value: [0.16, 0.15, 0.14, 0.13, 0.12, 0.11, 0.10, 0.09]
        - filter: self_attn.o_proj
          value: [0.13, 0.12, 0.11, 0.11, 0.11, 0.11, 0.11, 0.11]
        - filter: mlp
          value: 0.10
        - value: 0.05
parameters:
  gain: 0.7
  inheritance: 0.45
  anchor: 0.60
  prior_strength: 0.95
  containment: 1.10
dtype: float32
out_dtype: bfloat16
tokenizer:
  source: base
chat_template: auto

Testing Notes

In story-writing tests, the model showed a tendency toward dark psychological themes, even when the prompts did not specifically request them.

Intended Use

01 Roleplay

Character-driven interaction, personas, dialogue, emotional scenes, and long-form roleplay.

02 Creative Writing

Fiction, dialogue, atmosphere, descriptive writing, stylistic drafting, and imaginative prose.

03 Storytelling

Long-form narratives, worldbuilding, continuity, evolving characters, and multi-character plots.

04 Interactive Fiction

Branching narratives, scenario play, character interaction, and continuously evolving stories.

Lunar-Trace-31B   ·   Cyclone Labs