julia-1-swe

julia-1-swe is a ticket triage model for Zorch SDLC orchestration. It is a full fine-tune of niko0xdev/julia-1-zorch-triage, which is itself built on SupersonicLabs/Julia-1.

Given a ticket (title, and optionally description, labels and Given/When/Then acceptance criteria), the model makes three decisions:

• role: which role should own the task
• action: what kind of work the task is
• risk: the main risk category of the task

Model summary

Labels

Intended use

• A fast first-pass classifier in an SDLC control plane: route a ticket to an owner role, tag the kind of work, and flag the risk category.
• Use the per-head confidence to send uncertain or high-risk tickets to a stronger LLM judge or a human reviewer.

Out-of-scope use

• Fully automated, unreviewed decisions on production, security or data-sensitive work.
• Labels outside the fixed label sets above. The model only chooses among the listed options.
• General text classification, generation, or chat.
• Treating the scores below as an estimate of accuracy on real tickets (see Limitations).

Training details

Data format. Each training row has the form {state, question, options, target}, with one row per head. state holds the ticket fields (title, description, labels, acceptanceCriteria), question is the head question from the table above, options is the label list, and target is the correct label.

Training data. Published as niko0xdev/julia-1-swe-data (config default, split train). Union of the R134 and R126 synthetic compositional sets, built with scripts/linux/build_latest_union.py: 63,324 rows (21,108 tickets x 3 heads).

Validation / model selection. R118, R119 and R121 hard validation sets combined (default / validation and eval_tasks in the same dataset): 1,188 tasks (3,564 head rows). Checkpoints were selected by task exact match.

Hyperparameters (configs/colab_cuda_latest.yaml):

Hardware and code. Trained on a Google Colab NVIDIA L4 GPU with scripts/linux/train_cuda.py and scripts/linux/train_resumable.py (resumable training with periodic state checkpoints).

Evaluation

Evaluated with scripts/linux/eval_heads.py on the hard validation task files. Each set has 396 tasks (132 per language: Vietnamese, Vietnamese without diacritics, Korean).

For reference, the base checkpoint niko0xdev/julia-1-zorch-triage scored role 67%, action 36%, risk 45% on R121.

Calibration on the same 3,564 head rows: NLL 0.021, ECE 0.0031. At a confidence threshold of 0.9, coverage is 99.47% with 99.83% accuracy.

Usage

The checkpoint uses the Julia-1 decision model runtime from the companion repository, not a standard transformers pipeline.

git clone https://github.com/niko0xdev/julia-1-zorch && cd julia-1-zorch && pip install -e .
# Per-head evaluation on task files ({task, expected} rows)
python scripts/linux/eval_heads.py --model niko0xdev/julia-1-swe path/to/validation_tasks_hard.jsonl

Loading the model in Python, as done in scripts/linux/eval_heads.py:

import os, torch
from huggingface_hub import snapshot_download
from zorch.runtime import ensure_julia_runtime
from zorch.eval_core import evaluate_task_exact

_, data_mod, model_mod, tok = ensure_julia_runtime(
    "SupersonicLabs/Julia-1", os.getenv("HF_TOKEN"), runtime_only=True
)
path = snapshot_download("niko0xdev/julia-1-swe")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model_mod.JuliaDecisionModel.from_pretrained(path).to(device).eval()
collator = data_mod.Collator(tok, max_length=256, head_length=128)

result = evaluate_task_exact(model, "validation_tasks_hard.jsonl", collator, device, batch_size=16)
print(result["task_head_accuracy"], result["task_exact_match"])

For single-ticket prediction, see scripts/v3/predict.py in the repository.

Limitations

• Synthetic data only. Training and evaluation data are synthetic compositional tickets. The base model card reports much lower scores on a held-out benchmark of real tickets; this fine-tune has not been evaluated on that benchmark, so the numbers above should not be read as real-world accuracy.
• Shared generator. The validation sets come from the same family of generators and vocabulary as the training data (the training union contains rows tagged with R118, R122, R126, R132 and R133 sources). No validation ticket appears verbatim in training, but the evaluation measures in-distribution generalisation.
• Not all latest sets evaluated. R126 and R134 were used for training, so they were not evaluated.
• Small evaluation sets. 396 tasks per set; a single error moves a per-set score by about 0.25 points.
• Language coverage. The fine-tuning and evaluation data contain only Vietnamese and Korean. English is listed because the base model supports it, but English performance after this fine-tune was not measured.

Citation

@misc{julia1swe2026,
  title        = {julia-1-swe: Multilingual SDLC Ticket Triage on Julia-1},
  author       = {niko0xdev},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/niko0xdev/julia-1-swe}},
  note         = {Fine-tuned from niko0xdev/julia-1-zorch-triage; code at https://github.com/niko0xdev/julia-1-zorch}
}