UniXcoder base — GGUF Q8_0
microsoft/unixcoder-base converted to GGUF at Q8_0 for
llama.cpp's llama-server --embeddings.
| File | unixcoder-base-Q8_0.gguf (138 MB) |
| SHA-256 | 81a5a20e748a1c97e63aaacf81febb82560e61734e572ca619f202719c0319c1 |
| Source revision | 558aa506226b13a7e83f66cb38c53e61b9603eac (safetensors) |
| Context | 1024 tokens |
| Pooling | mean (stored in the file, no --pooling flag needed) |
| Dimensions | 768 |
Prefix every input with <encoder-only></s>
UniXcoder reads a mode token. Its own encoder mode feeds <s> <encoder-only> </s> text </s>; llama-server
adds <s> and the closing </s>, so send the rest as text:
llama-server -m unixcoder-base-Q8_0.gguf --embeddings --port 8080
curl -s localhost:8080/v1/embeddings -H 'Content-Type: application/json' \
-d '{"input":["<encoder-only></s>def f(): pass"]}'
The three mode tokens (<encoder-decoder>, <encoder-only>, <decoder-only>, ids 5–7) are marked special
in this file so the text form maps to the single id; in the upstream tokenizer they are ordinary vocabulary
entries that UniXcoder's code inserts by id.
How it was converted
convert_hf_to_gguf.py (llama.cpp, 2026-10-01) with --outtype q8_0, after:
- writing a
tokenizer.jsonwithRobertaTokenizerFast.save_pretrained(upstream ships onlyvocab.json+merges.txt; without it the converter falls back to a WordPiece vocabulary), with the three mode tokens registered as special added tokens (ids unchanged); - adding a sentence-transformers
modules.json+1_Pooling/config.jsondeclaring mean pooling; - mapping this tokenizer's pre-tokenizer hash (
f956de60…) to llama.cpp'sroberta-bpe. It is aRobertaTokenizerwith an extended vocabulary, so the GPT-2 byte-level split applies; the converter's hash table did not list it.
Verification
40 random code/console/output blocks (first 600 characters), embedded by this file on llama-server (CPU) with
the prefix above and by the original model in PyTorch fp32 ([CLS] <encoder-only> [SEP] text [SEP], masked
mean over the last hidden state). Token ids are identical; cosine similarity min 0.9994, median 0.99978.
The weights and their licence (Apache-2.0) are Microsoft's; see the UniXcoder paper.