Original Perplexity late-0.6B — native MLX FP16

General starting point. Original upstream model converted to MLX FP16; no BANKING fine-tuning. Supports text queries and text or bounded image documents. Weight file: 1,189,337,799 bytes. Image coverage is bounded sanity testing, not a real-world document benchmark.

Unofficial independent work; no affiliation or endorsement by Perplexity. Native token embeddings and MaxSim retrieval. This is a custom MLX runtime, not a drop-in mlx_lm or Sentence Transformers checkpoint. Inference imports neither Torch nor Transformers.

Source code · Experiment article · All variants · Exact results

Measured retrieval results

Same fresh BANKING intent-exemplar retrieval test: 770 queries, 154 documents, 77 intents; native MLX FP16. This is not the official BANKING77 classification benchmark or answer accuracy.

Training used 1,232 queries and 154 exemplars; separate dev used 308 queries and 154 documents. Two seeds × two learning rates × three epochs per arm; selection used dev only. Full vs adapted teacher: +11.75 percentage points nDCG, paired 77-intent bootstrap 95% CI [+9.20, +14.53]. Student: +4.05 points, CI [+1.53, +6.67]. This measures the complete recipes; no matched head-only control isolates the contribution of internal layers.

NanoSciFact cross-domain check (FP32 CUDA, 40 queries / 2,919 documents): adapted teacher 85.81%, full 84.62%, student 66.05% nDCG@10. Full Hit@1 drops 80% → 75%; Recall@10 stays 92.5%. No full native MLX Nano test was run for the newly trained models. The student is not a generic replacement.

Student warm median 26.16 ms vs teacher 48.04 ms on eight fixed dev inputs on an M3 / 16 GB Mac (~1.84×). Two warmups and seven measured passes; this is a small latency fixture, not application throughput. Student has 24.2% fewer text parameters; its 37.1% smaller file also omits vision. Estimated GPU lease cost for training, evaluation and rescue was $0.925, excluding storage/R2; not an invoice.

Run on Apple Silicon

Tested Python 3.12 and MLX 0.32.3. Download the complete repository, including code/:

python3 -m venv .venv
source .venv/bin/activate
pip install huggingface_hub
hf download Nurymanau/pplx-embed-v2-late-0.6b-MLX-fp16 --revision v0.1.0 --local-dir model
cd model
pip install -r requirements.txt

Create documents.json:

[{"id":"train","text":"The train leaves at 08:30."},{"id":"refund","text":"A refund can take several business days."}]

PYTHONPATH=code python -m pplx_mlx.cli --model . index --documents documents.json --output my-index
PYTHONPATH=code python -m pplx_mlx.cli --model . search --index my-index --query "When does the train leave?" --top-k 2

Rebuild the index when changing models. One unpadded input per call; query ≤1,024 and document ≤4,096 tokens, oversized inputs rejected. Avoid parallel model jobs on small-memory Macs. Text is the validated use for trained variants; the student has no image support.

Provenance and licensing

Upstream perplexity-ai/pplx-embed-v2-late-0.6b, revision 8fc2de24534aa3610d85fa59c463313a5f096455. This single-variant repository repackages identical published runtime and weight bytes from Nurymanau/pplx-embed-v2-late-0.6b-MLX. PROVENANCE.json maps every copied file and SHA256. No new training occurred during repackaging. SHA256SUMS.json covers every payload file except itself.

Weights follow the upstream MIT declaration (WEIGHTS_LICENSE.txt). Runtime is Apache-2.0 (LICENSE), with third-party notices under licenses/. BANKING77 training data for trained variants: PolyAI, CC BY 4.0, Casanueva et al. (2020), Efficient Intent Detection with Dual Sentence Encoders. See DATA_LICENSE.txt; no raw dataset is redistributed. Upstream pretraining contamination is unknown.

Related checkpoints

• Original Perplexity late-0.6B — native MLX FP16
• Perplexity late-0.6B — BANKING full text fine-tuning on MLX
• Perplexity late — six-layer BANKING student on MLX

Project links

Apple Silicon MLX portfolio · Technical write-up