Giga-Embeddings-instruct-480M-0826 (GGUF)

GGUF quantizations for ai-sage/Giga-Embeddings-instruct-480M-0826.

Model Details

• Base Architecture: Qwen3-based encoder with bidirectional self-attention (causal_attention: false).
• Embedding Dimension: 1024
• Max Context: 8192 tokens
• Pooling: Mean pooling with subsequent L2 normalization.
• Instruction Format:
  • Queries (asymmetric retrieval):

    Instruct: {task description}
    Query: {text}
    

  • Documents: Encode raw text without instruction prefix.

Available Files

Benchmark & Parity Verification

Validated on 228 queries over 76 documents against reference PyTorch model:

• Cosine Parity (Mean): 0.9996 (min 0.9995)
• Retrieval MRR@10: 0.9328 (Reference: 0.9392)
• Recall@5: 98.68% (Reference: 98.25%)
• Recall@10: 100.0%
• Detailed reports attached: report-*.json and retrieval-report-*.json.

Checksums (SHA256)

553e67852a2c76eb361cb6aabac068cb94903368e64f64132d43b53d50d1af97  ./giga-embeddings-480m-0826-f16.gguf
2ef28f8f6b43697f30f85d24c66fd0b0681e2cb390dd4cf03337d15c3408a0ad  ./giga-embeddings-480m-0826-q8_0-tokf16.gguf
29abcdfce9ead6533f964ebc75b9f21ac02a5bf02cc761e6ad42cca408e4a22c  ./giga-embeddings-480m-0826-q8_0.gguf

Usage with llama.cpp

# Run server with mean pooling & embedding mode
llama-server -m giga-embeddings-480m-0826-q8_0.gguf --embedding --pooling mean -c 8192