| Parameter | Value |
|---|---|
| start_layer_index | 34 |
| end_layer_index | 73 |
| preserve_good_behavior_weight | 0.5584 |
| steer_bad_behavior_weight | 0.0005 |
| overcorrect_relative_weight | 0.9231 |
| neighbor_count | 8 |
| Metric | This model | Original model (Zyphra/ZAYA1-74B-preview) |
|---|---|---|
| KL divergence | 0.0095 | 0 (by definition) |
| Refusals | 0/100 | 96/100 |
ZAYA1-74B-Preview is a mixture of experts language model with 4B active and 74B total parameters. This is a reasoning-base checkpoint which has not been tuned for chat or undergone RL post-training. ZAYA1-74B-Preview was trained end to end on AMD.
Learn more on our blog.
We recommend installing the following libraries in a fresh python environment (tested with python 3.12).
To use ZAYA1-74B-preview, install zaya1-pr branch from our fork of vllm library (the command will trigger a full build of vLLM from source):
pip install "vllm @ git+https://github.com/Zyphra/vllm.git@zaya1-pr"
If you want to run in transformers, install zaya1 branch from our fork of transformers library as well:
pip install "transformers @ git+https://github.com/Zyphra/transformers.git@zaya1"
To start vLLM server, run the following command:
vllm serve Zyphra/ZAYA1-74B-Preview --port 8010 \
--mamba-cache-dtype float32 --dtype bfloat16 \
--reasoning-parser qwen3 --enable-auto-tool-choice --tool-call-parser zaya_xml
For parallel deployment we recommend using DP with EP as TP for CCA is not supported in the branch above. If running on 8 GPUs, set extra flags -dp 8 -ep to run with DP=EP=8.
Once the server is up, you can query a model with curl like in the following example:
curl http://localhost:8010/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "Zyphra/ZAYA1-74B-Preview",
"messages": [
{"role": "system", "content": "You are a helpful assistant."},
{"role": "user", "content": "Hello. How is it going?"}
]
}'
model-00014-of-00015.safetensors 10.0 GBmodel-00004-of-00015.safetensors 10.0 GBmodel-00006-of-00015.safetensors 10.0 GBmodel-00008-of-00015.safetensors 10.0 GBmodel-00010-of-00015.safetensors 10.0 GBmodel-00012-of-00015.safetensors 10.0 GBmodel-00002-of-00015.safetensors 10.0 GBmodel-00001-of-00015.safetensors 10.0 GBmodel-00013-of-00015.safetensors 10.0 GBmodel-00003-of-00015.safetensors 10.0 GBmodel-00005-of-00015.safetensors 10.0 GBmodel-00007-of-00015.safetensors 10.0 GBmodel-00009-of-00015.safetensors 10.0 GBmodel-00011-of-00015.safetensors 10.0 GBmodel-00015-of-00015.safetensors 9.9 GB