Granite-Embedding-311M-Multilingual-R2, GGUF

GGUF conversions of IBM's granite-embedding-311m-multilingual-r2 for llama.cpp: a 311M-parameter ModernBERT encoder that turns text into 768-dimensional vectors, with CLS pooling. Nothing is changed but the file format; for what the model does and how well, see IBM's model card.

Files

How they were made

• Source: ibm-granite/granite-embedding-311m-multilingual-r2, revision 44399559930365213510b1ee2eb15ded83374f0e (18 May 2026), model.safetensors and its configuration, fetched with huggingface_hub.
• Converter: llama.cpp convert_hf_to_gguf.py at commit d81235049384534c167caea52b85a694f6103d14 (5 October 2026), the llama.cpp version of the Rust crate llama-cpp-2 0.1.160.

python convert_hf_to_gguf.py <snapshot of the source revision> --outtype f16  --outfile granite-embedding-311m-multilingual-r2-f16.gguf
python convert_hf_to_gguf.py <snapshot of the source revision> --outtype q8_0 --outfile granite-embedding-311m-multilingual-r2-q8_0.gguf
python convert_hf_to_gguf.py <snapshot of the source revision> --vocab-only   --outfile granite-embedding-311m-multilingual-r2-vocab.gguf

How they were checked

Each file was run by llama.cpp (through llama-cpp-2 0.1.160, on the CPU) and compared with IBM's own model code in PyTorch (transformers 5.21.0, torch 2.14.1, float32, CLS pooling, L2 norm), on 181 texts of up to 512 tokens: Wikipedia articles in Russian, German and English on the same ten topics, and paragraphs that mix Russian and English.

The vectors were computed from the token ids of the source repository's own tokenizer (tokenizer.json). llama.cpp's built-in tokenizer, loaded from the vocabulary-only file with vocab_only, gives the same ids on all 181 texts (the reference starts with <bos> and is cut at 512 tokens). The vocabulary-only file carries the same tokenizer metadata as the f16 file, key for key, and no tensors.

Use

The model needs CLS pooling. Normalise the vectors if you compare them by cosine.

llama-embedding -m granite-embedding-311m-multilingual-r2-f16.gguf \
  --pooling cls --embd-normalize 2 -p "Your text"

llama-server -m granite-embedding-311m-multilingual-r2-f16.gguf \
  --embedding --pooling cls

granite-embedding-311m-multilingual-r2-vocab.gguf holds the tokenizer alone: load it with vocab_only to count or split text into the model's tokens without its weights, for example to cut chunks to the model's 512-token limit when the model itself runs elsewhere.

No query or document prefix is needed. IBM trained the model mostly on passages of up to 512 tokens.

License

The model weights are under the Apache 2.0 license, as published by IBM. The tokenizer is derived from the Gemma 3 tokenizer and is subject to the Gemma Terms of Use, as IBM's model card states.