SuperBenchMaxx E4B Leaked LoRA

CONTAMINATED / LEAKED Benchmark Demo

This is a deliberately contaminated same-item benchmark demonstration. It was post-trained on benchmark items and their known answers to show that benchmark scores can be manipulated by leakage. The scores below are not clean generalization results and must not be compared as ordinary model capability.

Base Model

• Base: mlx-community/gemma-4-e4b-it-4bit
• Adapter format: MLX LoRA
• Best local adapter source: /Users/mac/adapters/superbenchmaxx-e4b-gpqa-repair-v3-balanced-20260511T054208Z

Same-Item Leaked Evaluation

Full leaked MLX evaluation on the contaminated 567-item set:

• Overall: 563/567 = 99.2945%
• BixBench: 205/205 = 100.0%
• GPQA Diamond: 198/198 = 100.0%
• HumanEval: 160/164 = 97.561%

Lookup answerbook reference:

• Overall: 567/567 = 100.0%
• BixBench: 205/205 = 100.0%
• GPQA Diamond: 198/198 = 100.0%
• HumanEval: 164/164 = 100.0%

Again: these are contaminated / leaked same-item scores, not evidence of clean generalization.

Usage

Download the adapter, then load the base model with the local adapter folder:

from huggingface_hub import snapshot_download
from mlx_lm import load

adapter_path = snapshot_download("Jiunsong/superbenchmaxx-e4b-leaked-lora")
model, tokenizer = load(
    "mlx-community/gemma-4-e4b-it-4bit",
    adapter_path=adapter_path,
)

MLX expects adapter_path to be a local folder, not a Hub repo id string.