pplx-embed-v2-late-0.6b GGUF — text retrieval
Text-only late-interaction encoders from the original checkpoint. F16 is converted from original F32 weights; Q4_K_M and Q6_K are derived independently from F16. Every file retains the original 1024→128 projection in F32 and produces one vector per token.
| File | Component | Quantization | Size | Status |
|---|---|---|---|---|
| pplx-embed-v2-late-0.6b-Q4_K_M.gguf | text-token-encoder | Q4_K_M | 366,305,504 bytes | Available |
| pplx-embed-v2-late-0.6b-Q6_K.gguf | text-token-encoder | Q6_K | 417,044,192 bytes | Available |
| pplx-embed-v2-late-0.6b-F16.gguf | text-token-encoder | F16 | 999,280,864 bytes | Available |
Usage
Use llama.cpp at 0ec4027:
llama-server -m pplx-embed-v2-late-0.6b-Q6_K.gguf \
--embedding --pooling none -ngl 99 -c 4096 -b 512 -ub 512 -np 1 \
--host 127.0.0.1 --port 8080
python late_interaction.py --query 'How do plants use sunlight?' \
--document 'Plants convert sunlight into chemical energy through photosynthesis.'
The supplied client needs NumPy. It places [Q] or [D] first, including
the trailing space, applies the original document punctuation mask,
L2-normalizes each token vector and scores ColBERT MaxSim. Preserve
unpooled output: sequence pooling changes the retrieval method.
Non-causal inputs must fit in a single batch/microbatch. The command and client default to 512 input tokens; increase both server batch settings and the client limit together for longer inputs. Source processing limits are 1024 query tokens and 4096 document tokens. Source vision tensors are not included in these text encoders.
Source and license
MIT as declared by Perplexity, with the original Qwen-family Apache-2.0
notices retained. See LICENSE, ATTRIBUTION.md,
CHANGES.md and licenses/.
REPRODUCE.md includes the local conversion adapter;
runtime-info.json and model-manifest.json
record pinned provenance and file hashes. The source template is preserved.