ReDimNet2+: Speaker Verification Model

ReDimNet2+ is a robustness-oriented adaptation of the compact ReDimNet2-B6 backbone for Automatic Speaker Verification (ASV), trained for robustness to channel and codec mismatch (compression, telephony codecs, noise, reverberation). The model uses a ReDimNet2 backbone with a Mel-spectrogram frontend and produces 192-dimensional speaker embeddings.

• Code (training, eval, Docker submission pipeline): github.com/lab260ru/redimnet2-plus
• Paper: ReDimNet2+ for Robust Speaker Verification under Channel and Codec Shift (manuscript submitted to ICASSP 2027)

Results

Evaluated with random 4-second windows and cosine similarity on L2-normalized embeddings (VoxCeleb1, EER %; EER_p pools O/E/H; EER_ph is a 26-condition codec/waveform stress test). See the paper for the full protocol and ablations.

Model Architecture

• Backbone: ReDimNet2-B6
• Embedding Dimension: 192
• Input: Mel-spectrograms (64 mel bins, 10ms hop)
• Sample Rate: 16 kHz
• Frontend: Log-Mel spectrogram with 64 mel bands
• Pooling: ASTP (Attentive Statistics Aware Pooling)
• Normalization: L2 normalization on embeddings

Usage

Installation

pip install torch torchaudio soundfile omegaconf

Loading the Model

import torch
from huggingface_hub import hf_hub_download
from omegaconf import OmegaConf

# Download model weights
model_path = hf_hub_download("lab260/redimnet2-plus", "pytorch_model_fsdp.bin")

# Load checkpoint
state_dict = torch.load(model_path, map_location="cpu")

# Strip FSDP wrapper prefixes
state_dict = {
    k.replace("_orig_mod.", "").replace("module.", ""): v
    for k, v in state_dict.items()
}

# Model configuration
model_config = {
    "encoder": {
        "_target_": "asv.models.redimnet2.ReDimNet2Encoder",
        "model_name": "b6",
        "pretrained": False,
        "train_type": "lm",
        "dataset": "vox2",
        "strict_load": True,
        "model_overrides": {
            "out_channels": 224,
            "return_2d_output": False
        }
    },
    "bridge": {
        "_target_": "asv.models.bridge.IdentityBridge"
    },
    "classifier": {
        "_target_": "asv.models.classifier.IdentityClassifier"
    }
}

Extracting Embeddings

import torchaudio.transforms as T

def load_audio(path, sr=16000):
    """Load audio file."""
    audio, sr = torchaudio.load(path)
    if audio.shape[0] > 1:
        audio = audio.mean(dim=0)  # Convert to mono
    return audio, sr

def compute_mel_spec(audio, sr=16000):
    """Compute mel-spectrogram."""
    mel_transform = T.MelSpectrogram(
        sample_rate=sr,
        n_fft=512,
        hop_length=160,
        n_mels=64,
        normalized=False
    )
    mel = mel_transform(audio.unsqueeze(0))
    mel = torch.log(mel + 1e-6)
    return mel

def extract_embedding(model, audio_path, device="cpu"):
    """Extract speaker embedding from audio."""
    # Load and preprocess
    audio, sr = load_audio(audio_path)
    mel = compute_mel_spec(audio, sr)

    # Add channel dim
    if mel.ndim == 3:
        mel = mel.unsqueeze(1)

    mel = mel.to(device)
    model = model.to(device)
    model.eval()

    with torch.no_grad():
        embedding = model(mel)
        embedding = torch.nn.functional.normalize(embedding, dim=1)

    return embedding.cpu().squeeze(0)

# Example usage
embedding = extract_embedding(model, "audio.wav")
print(f"Embedding shape: {embedding.shape}")  # [192]

Speaker Verification

def cosine_similarity(e1, e2):
    """Calculate cosine similarity between embeddings."""
    return torch.dot(e1, e2).item()

def verify_speaker(model, audio1_path, audio2_path, threshold=0.79):
    """Verify if two audio samples are from the same speaker."""
    emb1 = extract_embedding(model, audio1_path)
    emb2 = extract_embedding(model, audio2_path)

    similarity = cosine_similarity(emb1, emb2)
    is_same = similarity >= threshold

    return {
        "similarity": similarity,
        "is_same_speaker": is_same,
        "confidence": abs(similarity - threshold)
    }

Model Details

• File Format: PyTorch state_dict (binary)
• File Size: ~100 MB
• Framework: PyTorch 2.11+
• Key Prefix: _orig_mod. (should be stripped during loading)

Training

Full recipe and code: github.com/lab260ru/redimnet2-plus. Summary:

• Initialization: public ReDimNet2-B6 checkpoint, backbone architecture unchanged.
• Data: 7 corpora, ~63.9k speakers, ~4.6M utterances, ~8,675 h — VoxBlink2, VoxCeleb2, 3D-Speaker, CN-Celeb/CN-Celeb2, TidyVoice, KeSpeech.
• Augmentation: filters, additive noise (colored, MUSAN, RawBoost-style), simulated RIRs, speed perturbation, and codec simulation (MP3, Opus, AAC, FLAC, Vorbis, G.723.1, IMA ADPCM, G.722, A-law, μ-law, Speex, AMR-NB at 8/16 kHz), plus SpecAugment/CutMix.
• Loss Function: SphereFace2 (one-vs-all)
• Optimizer: AdamW, BFloat16, Accelerate + FSDP on 6 GPUs, gradient clipping at 20
• Recipe: staged multi-corpus adaptation followed by large-margin fine-tuning (LMFT) on 6-second windows (margin 0.3)

This checkpoint (pytorch_model_fsdp.bin) is the final ReDimNet2+ LMFT model from the paper.

Retrieval reranking

The GitHub repo also ships a training-free, graph-based reranking stage (mean-chain + k-NN graph reranking with hubness correction) for open-set retrieval on top of these embeddings — see submission/reranking/. It raises VoxBlink2-subset Pr@k from 0.7413 to 0.7687 without retraining.

Limitations

• Model expects 16 kHz audio input
• Maximum audio duration: 60 seconds
• EER roughly doubles from VoxCeleb1-E to -H (hard impostors), and the 26-condition codec/waveform stress test (EER_ph) stays above the clean metrics even after LMFT — see the paper's limitations section
• External comparisons in the paper are shared-protocol, not equal-data comparisons

Citation

@unpublished{borodin2027redimnet2plus,
  title  = {{ReDimNet2+} for Robust Speaker Verification under Channel and Codec Shift},
  author = {Borodin, Kirill and Kudryavtsev, Vasiliy and Mkrchian, Grach},
  year   = {2026},
  note   = {Manuscript submitted to ICASSP 2027}
}

License

This model is released under the Apache 2.0 license.

Contact

• Email: kborodin.research@gmail.com
• Telegram: @korallll_ai