electrical-embeddinggemma-ir_q5_k_m

Model Description

This model is the GGUF q5_k_m (5-bit K-quant) variant of the gemma-300m-electrical-electronics-ir family, fine-tuned from unsloth/embeddinggemma-300m for dense Information Retrieval (IR) in the electrical and electronics engineering domain. This build sits between q4_k_m and q8_0 in terms of bit-depth and offers a slightly smaller footprint than q8_0 (~247 MB) while maintaining strong retrieval performance.

Training Data

The model was trained on the disham993/ElectricalElectronicsIR dataset — 20,000 question-passage pairs covering electrical engineering, electronics, power systems, and communications.

• 16k train / 2k validation / 2k test
• Queries: 133–822 characters; passages: 586–5,590 characters
• Topics include phased array antennas, IEC 61850 protocols, Josephson junctions, OTDR measurements, MIMO channel estimation, FPGA partial reconfiguration, and more

Model Details

Training Procedure

Training Hyperparameters

Evaluation Results

Evaluated on the held-out test split (2,000 queries) of disham993/ElectricalElectronicsIR using sentence_transformers.evaluation.InformationRetrievalEvaluator.

MAP@100 delta vs f16: −0.0025. All fine-tuned variants remain vastly superior to the baseline (+41 pp MAP@100).

Usage

LM Studio (OpenAI-compatible API)

Load this model in LM Studio and use it via the built-in OpenAI-compatible server:

from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:1234/v1", api_key="lm-studio")

texts = [
    "What is impedance matching?",
    "Impedance matching maximises power transfer by equalising source and load impedance.",
    "An LLC resonant converter achieves zero-voltage switching using an LC tank circuit.",
]

response = client.embeddings.create(
    model="text-embedding-electrical-embeddinggemma-ir",
    input=texts,
)

for item in response.data:
    print(f"[{item.index}] dim={len(item.embedding)}  first5={item.embedding[:5]}")

llama-cpp-python

# Install dependencies
pip install huggingface_hub
CMAKE_ARGS="-DGGML_CUDA=on" FORCE_CMAKE=1 pip install llama-cpp-python # (For NVIDIA GPU acceleration)

import torch
import torch.nn.functional as F
from huggingface_hub import hf_hub_download, HfApi
from llama_cpp import Llama

class DummyModelCardData:
    def set_evaluation_metrics(self, *args, **kwargs): pass

class GGUFEmbeddingWrapper:
    def __init__(self, repo_id):
        self.repo_id = repo_id
        # Automatically detect the GGUF file in the repo
        api = HfApi()
        files = api.list_repo_files(repo_id)
        gguf_file = next((f for f in files if f.endswith('.gguf')), None)
        if not gguf_file: raise ValueError(f"No .gguf file found in disham993/electrical-electronics-gemma-ir_q5_k_m")
        
        print(f"Downloading/Using {gguf_file} from disham993/electrical-electronics-gemma-ir_q5_k_m...")
        model_path = hf_hub_download(repo_id=repo_id, filename=gguf_file)
        
        self.llm = Llama(
            model_path=model_path,
            embedding=True,       # CRITICAL: Required for dense extraction
            n_gpu_layers=-1,      # Offload completely to GPU (Optional)
            n_ctx=1024,           # Constrain context window
            verbose=False
        )
        self.dtype = torch.float16
        self.model_card_data = DummyModelCardData() # Bypasses evaluator metadata crashes
        
    def encode(self, sentences, batch_size=None, **kwargs):
        convert_to_tensor = kwargs.pop('convert_to_tensor', True)
        if isinstance(sentences, str): sentences = [sentences]
            
        # Handling list of dicts for corpus evaluations
        if isinstance(sentences, list) and len(sentences) > 0 and isinstance(sentences[0], dict):
            sentences = [(doc.get("title", "") + " " + doc.get("text", "")).strip() for doc in sentences]
            
        embeddings = []
        for text in sentences:
            res = self.llm.create_embedding(text)
            embeddings.append(res['data'][0]['embedding'])
            
        tensors = torch.tensor(embeddings, dtype=torch.float32)
        if convert_to_tensor:
            if torch.cuda.is_available(): tensors = tensors.cuda()
            return tensors
        return tensors.cpu().numpy()

    # Dynamic alias interceptor to satisfy strict evaluator engines
    def __getattr__(self, name):
        if name.startswith("encode_"):
            def wrapper(*args, **kwargs):
                kwargs['convert_to_tensor'] = True
                return self.encode(*args, **kwargs)
            return wrapper
        raise AttributeError(f"'{self.__class__.__name__}' object has no attribute '{name}'")


# === SEMANTIC SEARCH EXAMPLE ===
if __name__ == "__main__":
    # Boot the wrapper dynamically against this Hub Repo
    model = GGUFEmbeddingWrapper("disham993/electrical-electronics-gemma-ir_q5_k_m")
    
    query = "How do transformers step up voltage?"
    
    # A miniature corpus of 10 engineering documents
    documents = [
        "Ohm's law defines the relationship between voltage, current, and resistance.",
        "AC circuits use alternating current which changes direction periodically.",
        "A step-up transformer has more turns on its secondary coil than its primary, increasing voltage.",
        "Capacitors store electrical energy in an electric field.",
        "Inductors resist changes in electric current passing through them.",
        "Transformers operate on Faraday's law of induction to transfer energy between circuits.",
        "Diodes allow current to pass in only one direction.",
        "Voltage is the electric potential difference between two points.",
        "A step-down transformer decreases voltage for safe residential use.",
        "Power is the rate at which electrical energy is transferred by a circuit."
    ]
    
    print("Embedding query and documents...")
    query_emb = model.encode(query)
    doc_embs = model.encode(documents)
    
    similarities = F.cosine_similarity(query_emb, doc_embs)
    top_3_idx = torch.topk(similarities, k=3).indices.tolist()
    
    print(f"\n--- Top 3 Documents for Query: '{query}' ---")
    for rank, idx in enumerate(top_3_idx, 1):
        print(f"Rank {rank} (Score: {similarities[idx]:.4f}) | {documents[idx]}")

Limitations and Bias

While this model performs exceptionally well in the electrical and electronics engineering domain, it is not designed for use in other domains. Additionally, it may:

• Underperform on queries that mix electrical engineering with unrelated domains (e.g., biomedical, legal, financial)
• Show reduced performance on non-English text or highly colloquial phrasing
• Show a slightly larger MAP delta vs f16 (−0.0025) compared to q4_k_m (−0.0008) due to K-quant cluster assignment characteristics for this model's weight distribution

This model is intended for research, educational, and production IR applications in the electrical engineering domain.

Training Infrastructure

For the complete fine-tuning and evaluation pipeline — from data loading to GGUF export — refer to the GitHub repository and the notebooks Finetuning_EmbeddingGemma_EEIR_RTX_5090.ipynb and Evaluate_All_Models.ipynb.

Last Update

2026-04-18

Citation

@misc{electrical-embeddinggemma-ir,
  author       = {disham993},
  title        = {Electrical \& Electronics Engineering Embedding Models},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/collections/disham993/electrical-and-electronics-engineering-embedding-models}},
}