BrunoHays/embeddinggemma-2-nullspace-audio

BrunoHays/embeddinggemma-2-nullspace-audio is google/embeddinggemma-2 with no change to the frozen weights, plus a frozen 256-dimensional null-space head that only acts on audio inputs.

• Text, image and video inputs: identical to google/embeddinggemma-2 (same code path, same outputs).
• Audio inputs: the base 768d embedding is shifted along 256 directions that the text embedding space never uses (the null space of the terminal projection). The shift is computed from two frozen Conformer intermediate layers through a PCA projection fitted only on external, non-benchmark audio.
• No training. The head has no trainable parameters at inference time. Its frozen tensors are in nullspace_audio_head.safetensors (2.37 MB).

Two operating points are submitted:

Architecture

For an audio clip, with the frozen audio tower of google/embeddinggemma-2:

1. Pool two Conformer layers over time (masked):
   • X₁ = [μ₂ ; σ₂] (2048d) from layer 2 (mean and std, then L2-normalised)
   • X₂ = μ₆ (1024d) from layer 6 (mean, then L2-normalised)
2. Project each block with its frozen PCA state (soft-whitened, fitted on external corpora):

   z_k = normalize((X_k - m_k) @ P_k) / (s_k + ε),   for k in {1, 2},  P_k shape: [d_k, 128]
   

   where s_k = σ_k^(1/2) is the stored (already square-rooted) scale and ε = 10⁻⁵.
3. Combine the two 128d blocks into a unit 256d vector:

   z   = normalize([√0.5 · z_1  ;  √0.5 · z_2])      shape: [256]
   

4. Inject into the base embedding v_base (768d, L2-normalised by the model):

   v_final = normalize(v_base + γ · (z @ U_⊥^T))     shape: [768]
   

   where U_⊥ has shape [768, 256] with U_⊥^T @ U_⊥ = I.

Why the null space

U_⊥ spans the orthogonal complement of the column space of the model's terminal projection W (768 × 512). Text embeddings lie in col(W), so they are orthogonal to U_⊥. The injection therefore does not move text embeddings, and it cannot change text-to-text similarities. Its effect is limited to audio embeddings, and cross-modal rankings move only slightly (see the MSEB section).

Zero-leakage PCA construction (Protocol A)

The PCA states are fitted only on 350 clips from each of four external corpora, with no MAEB or MSEB audio:

• ylacombe/expresso (speech)
• agkphysics/AudioSet (audio events)
• lewtun/music_genres (music)
• ashraq/esc50 (environmental sounds)

Clips are at most 10 s. The per-domain statistics are combined with balanced weights, then soft whitening (σ⁻¹/²) is applied on the same eigenbasis.

Usage

The submission is implemented in gladia_embeddinggemma2_nullspace.py (an MTEB model implementation). With mteb installed and the file registered:

import mteb

model = mteb.get_model("BrunoHays/embeddinggemma-2-nullspace-audio")

Outside MTEB, load the wrapper directly:

from gladia_embeddinggemma2_nullspace import EmbeddingGemma2NullSpaceWrapper

model = EmbeddingGemma2NullSpaceWrapper(gamma=1.0)  # downloads nullspace_audio_head.safetensors from this repo

The base checkpoint is loaded from google/embeddinggemma-2 at revision 914f7f89142e33e77833254d9c9b90c3cef7303b. Accepting the Gemma terms on the base model is required.

MAEB results (30 tasks)

Macro average over the 30 MAEB tasks. Main scores in %.

"Raw" uses the measured base score for every task. "Card basis" adds the same delta to the published model-card baseline of 49.39.

Coverage: all 30 MAEB tasks were measured for these configurations. FleursT2ARetrieval (102 languages) and CommonVoiceMini21T2ARetrieval (117 languages) were run per language and macro-averaged over languages. Their measured base scores (99.77 and 93.08) differ from the published values (97.85 and 93.25), so the measured numbers are the ones reported here.

Per-task scores

Where it helps and where it hurts

The macro gain comes mainly from audio-to-audio and gender/emotion/genre tasks. Several tasks lose accuracy. The largest losses at γ = 1.0 are NMSQAPairClassification (−19.5 pp), SIBFLEURS (−35.9 pp), MInDS14 (−9.3 pp) and VoxCelebSA (−9.8 pp). Check these tasks before using the model where they matter.

MSEB results (Google SVQ, 26 locales)

Macro mean over 26 locales of google/svq (test split). Evaluated with the same head configs, using mteb/svq. Measured in this pipeline; no other MSEB tasks were run.

The head moves retrieval scores by at most about 0.3 points on audio-to-text and about 0.05 points on text-to-audio. The effect is far smaller than the MAEB changes above.

Limitations

• Selection bias. The PCA construction is leak-free, but the choice of mode (ext_soft_whiten) and γ was made by looking at test scores across a small grid. Treat the MAEB numbers as optimistic. γ = 1.0 is the largest value tested for this mode, so the optimum may lie beyond it.
• Single-run numbers. All 30 MAEB and 2 MSEB tasks were measured in this pipeline. The two multilingual T2A tasks were run per language; their scores are macro means over 102 and 117 languages.
• Mixed effects. Some audio tasks regress (see above). Do not assume the gain transfers to every audio task.
• Audio batching. In the audio path, clips are encoded in dataset order without SentenceTransformer's length sorting, so results can differ from the base model by about 10⁻³ per coordinate (bf16 padding effects).

License

Gemma license, inherited from google/embeddinggemma-2. Check the Gemma terms before redistributing.

Citation

If you use this model, please cite the base model:

@misc{vera2025embeddinggemmapowerfullightweighttext,
      title={EmbeddingGemma: Powerful and Lightweight Text Representations},
      author={Vera, Henrique Schechter and others},
      year={2025},
      eprint={2509.20354},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2509.20354},
}