dinghy-law-4b-v1 — GGUF

GGUF / llama.cpp build of Hanno-Labs/dinghy-law-4b-v1, a compact (0.6B) legal text-embedding model (contrastive fine-tune of Qwen/Qwen3-Embedding-0.6B). Source revision d1fa2f346ca7461f680036b8af1927706a3afb58.

What this is (backbone-only)

The sentence-transformers pipeline is Transformer -> last-token Pooling -> Dense -> L2 Normalize. llama.cpp carries the backbone: Transformer -> last-token pool -> L2 norm. The 2_Dense head is dropped — it is a near-orthogonal rotation (normalized ortho-defect 0.00103, unit row-norms, ~zero bias), and cosine similarity is invariant under rotation, so dropping it does not change retrieval ranking. Probe cosine(full, backbone) = 0.997 with identical nearest-neighbor ordering.

Files

• dinghy-law-4b-v1-Q4_K_M.gguf — 2.50 GB
• dinghy-law-4b-v1-Q6_K.gguf — 3.31 GB
• dinghy-law-4b-v1-Q8_0.gguf — 4.28 GB
• dinghy-law-4b-v1-f16.gguf — 8.05 GB

Fidelity — cosine(HF backbone, GGUF) on legal probes

Worst min-cos across quants: 0.97009 (WARN).

Usage (llama.cpp)

Embeddings use last-token pooling and L2 normalization:

llama-embedding -m dinghy-law-4b-v1-Q8_0.gguf \
  -p "Retrieve statutes governing landlord obligations for habitability." \
  --pooling last --embd-normalize 2

For retrieval, format the query with the Qwen3-Embedding instruction prefix (documents are embedded raw):

Instruct: {task_instruction}\nQuery: {query}

e.g. task instruction Identify the most relevant statutes for the given situation. for statute retrieval.

Provenance

Quantized from the source safetensors with llama.cpp/convert_hf_to_gguf.py (f16) + llama-quantize (Q8_0, Q4_K_M). MTEB(Law, v1) Mean(Task) nDCG@10 of the full model = 71.24. License: apache-2.0.