gte-modernbert-base — GGUF Q8_0

Alibaba-NLP/gte-modernbert-base converted to GGUF at Q8_0 for llama.cpp's llama-server --embeddings, with the ModernBERT architecture recorded correctly.

llama-server -m gte-modernbert-base-Q8_0.gguf --embeddings --port 8080
curl -s localhost:8080/v1/embeddings -H 'Content-Type: application/json' -d '{"input":["def f(): pass"]}'

Why this conversion

A GGUF of this model we had downloaded elsewhere was labelled bert with no bert.context_length, and llama.cpp refused it. This one was made with llama.cpp's own convert_hf_to_gguf.py (2026-10-01, --outtype q8_0), which maps the model to llama.cpp's modern-bert architecture; it needs a llama.cpp build that has that architecture.

Verification

Embedded short code and prose inputs with this file on llama-server and with the original model in PyTorch (CLS vector of the last hidden state): cosine similarity 0.9997–0.9998.

The weights and their licence (Apache-2.0) are Alibaba-NLP's.