Zeta 2.1 MLX 4-bit

This repository contains a 4-bit MLX-LM quantized version of zed-industries/zeta-2.1.

Zeta 2.1 is a code edit prediction model. It is designed for next-edit suggestion workflows: given surrounding code, edit history, related files, and an editable region around the cursor, it predicts the rewritten content for that region.

Model Details

Original model: zed-industries/zeta-2.1
Architecture: LlamaForCausalLM
Model type: llama
Context length: 32,768 tokens
Original dtype: BF16
MLX quantization: 4-bit affine
Quantization group size: 64
Converted with: mlx-lm

The converted model directory is approximately 4.3 GB.

Installation

pip install -U mlx-lm

Usage

mlx_lm.generate \
  --model slxnxl/zeta-2.1-mlx-4bit \
  --prompt '<[fim-suffix]>
    return a + b

<[fim-prefix]><filename>calc.py
def add(a, b):
<|marker_1|>
    result = a<|user_cursor|>
<|marker_2|>
<[fim-middle]>' \
  --max-tokens 80 \
  --temp 0.2

Example output:

<|marker_1|>
    result = a + b
    return result
<|marker_2|>

Prompt Format

Zeta uses an SPM/FIM-style prompt format:

<[fim-suffix]>
code after editable region
<[fim-prefix]><filename>related/file.py
related file content

<filename>edit_history
--- a/some_file.py
+++ b/some_file.py
-old
+new

<filename>path/to/target_file.py
code before editable region
<|marker_1|>
code that
needs to<|user_cursor|>
be rewritten
<|marker_2|>
<[fim-middle]>

The model is expected to generate the rewritten editable region, usually including the <|marker_1|> and <|marker_2|> markers.

Local Conversion

This model was converted from a local Hugging Face checkout with:

mlx_lm.convert \
  --hf-path /path/to/zeta-2.1-hf \
  --mlx-path /path/to/zeta-2.1-mlx-4bit \
  -q \
  --q-bits 4 \
  --q-group-size 64

If Metal GPU conversion times out on macOS, conversion can be run through the MLX Python API with mx.set_default_device(mx.cpu).

License

This quantized model follows the original model license: Apache 2.0. See the original model card for upstream details: zed-industries/zeta-2.1.