juxy-maxx-30b
LoRA adapter that teaches the Juxy coding/security persona to a ~30B-class
Mixture-of-Experts base — the scale-up of
zxcxzczxcxzc/juxy-maxx, a 4B model
that stopped after 80 of its 104 training steps.
This adapter does not beat its base model on code correctness. Measured, not guessed: on 24 HumanEval problems the base scored 22/24 and this adapter scored 20/24. It is published as a persona/format tune with the regression stated up front.
What it changes
- Answers as "Juxy", terse, code-first, with error handling and a short rationale.
- Adds mitigation advice to security questions.
- Tends to emit a short reasoning preamble (sometimes a stray closing think tag) before the code block. That preamble is what pushed otherwise-correct answers past the 512-token evaluation cap.
Training
| item | value |
|---|---|
| base | empero-ai/Qwen3.8-35B-A3B-Distill (35B total, ~3B active) |
| data | zxcxzczxcxzc/juxy-maxx-corpus, 19,860 rows, ~8.79M tokens |
| method | LoRA r=32, alpha=64, dropout 0.05, assistant-span loss only |
| targets | attention + MLP projections, vision tower frozen |
| schedule | 1 epoch, 187 steps, lr 1e-4 cosine, 3% warmup, bf16 |
| hardware | 1x RTX PRO 6000 96GB, 93.4GB peak, ~1,570 tok/s, 98.6 min |
Training loss: 1.31 (step 10) -> 0.45 (step 150) -> 0.63 (step 187). The rise at the end is a batching bug: rows were length-sorted and only reshuffled at epoch end, so the final steps were all long sequences.
Evaluation
HumanEval, 24 problems, greedy decoding, generated code executed against the real unit tests with a 20 s timeout:
| model | pass@1 |
|---|---|
| base (no adapter) | 22/24 = 0.917 |
| this adapter (step 150) | 21/24 = 0.875 |
| this adapter (final, 187) | 20/24 = 0.833 |
Per-problem results and raw logs are in training/.
Usage
import torch
from transformers import AutoTokenizer, Qwen3_5MoeForConditionalGeneration
from peft import PeftModel
base = "empero-ai/Qwen3.8-35B-A3B-Distill"
tokenizer = AutoTokenizer.from_pretrained(base)
model = Qwen3_5MoeForConditionalGeneration.from_pretrained(
base, dtype=torch.bfloat16, device_map={"": 0}
)
model = PeftModel.from_pretrained(model, "zxcxzczxcxzc/juxy-maxx-30b")
model.eval()
system = (
"You are Juxy, a fast coding and cybersec agent. Be concise and token-efficient. "
"Give complete, runnable code in any language with error handling; explain the WHY briefly. "
"For security, give defensive + legal offensive concepts and always include the fix/mitigation. "
"You are Juxy - not any other AI model or company."
)
messages = [{"role": "system", "content": system},
{"role": "user", "content": "Write a Python retry decorator with jitter."}]
prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(prompt, add_special_tokens=False, return_tensors="pt").to(model.device)
ids = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(ids[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))
Limitations
- Measurably worse at HumanEval than the base (20/24 vs 22/24). Small sample; the direction is consistent across both checkpoints.
- No held-out split was used for training (all corpus rows were trained on).
- Security behaviour is a persona prior, not a safety guarantee. Nothing was red-teamed.
- Sequence cap 2048 during training; longer contexts fall back to base behaviour.
- Not served by any inference provider: the architecture (qwen3_5_moe conditional generation) is not in the supported set, so this repo is weights + adapter only.