pplx-embed-v2-late-9b-MLX-Mixed-4bit

Unofficial mixed Q4/Q8 MLX quantization for native Swift/MLX execution in mere.run. Runtime support is under development; released binaries may not include it.

Q4/group-64 transformer projections, Q8/group-64 token embeddings, FP32 recurrent gates, norms, convolution, vision and output projection. This is weight quantization; the contextual model still returns its original int8-valued vectors.

See PPLX_QUALIFICATION.json for the bounded diagnostic scope, UPSTREAM_MODEL_CARD.md, PPLX_CONVERSION.json, MODIFICATIONS.md and SHA256SUMS. Broad retrieval benchmarks and maximum-context memory fit remain unqualified.

hf download Sawfwair/pplx-embed-v2-late-9b-MLX-Mixed-4bit --local-dir ./pplx-late-mixed
mere.run text embed --model ./pplx-late-mixed --task query "What is photosynthesis?"

Apple validation

Full-checkpoint native CLI parity passed on a 36 GiB Apple Silicon Mac. See PPLX_APPLE_QUALIFICATION.json for measured process memory, timings and scope. This covers the included short text/chunk suite and, for late, one synthetic image; maximum context and general retrieval quality remain unqualified.