Zeus-28b-v0.1

Twelve intended-use examples: measured results

Three examples per intended use, run through the original Zeus public decision API on two RTX 5090s. These are self-contained synthetic tasks with explicit supplied facts and rules. They are a small illustrative check, not a standard benchmark or a reliable estimate of broad task accuracy. All twelve tasks and reference answers were fixed before inference; incorrect selections remain in the results.

The observed weaknesses are visible in the table: none of the three capacity/multi-choice selections matched the reference, and two ordinal ratings were wrong. The gate also accepted an incorrect support-priority classification. These intended uses are not reliability guarantees.

Accuracy means an exact selected option, or an exact set for multi-choice, before any external fallback. Among the 6 selections accepted by the gate, 5/6 matched the reference answer. Only single-choice confidence is calibrated in this release; multi-choice, yes/no and ordinal-score requests automatically escalate. No fallback or real tool was executed in this check. Selecting a proposed tool correctly does not demonstrate successful tool execution.

Decision time measures one unpadded request at a time, with both GPUs synchronized before and after Zeus.decide. It includes input encoding, transfers, Qwen prefill, the adapter/Laya head, gate and result conversion. It excludes model loading (25.5 seconds for the successful process), one unscored warmup (10.81 seconds), hashing, persistence and publication. Each task was timed once; first-use kernel overhead may remain. These are decision-interface times, not agent completion times, fallback times or a speed comparison with Qwen. The frozen original release and precision were unchanged. One monitoring-hook failure occurred during an earlier unscored warmup; no scored answer was retried.

The examples cover incident evidence and debugging-action selection, support triage, resource and eligibility choices, and supplied-rule release/rating decisions. All tasks and individual results, verified report, raw predictions, and protocol and timing conditions are available for inspection. The academic Qwen–Zeus comparison and its measured limitation remain documented below.

Follow-up: why the typed selections failed

An exact-input rerun on both RTX 5090s again returned 6/12 correct; 12/12 selections and 12/12 escalation decisions matched the first pass. The original sample results above are retained unchanged. Fifteen separately prespecified diagnostic controls tested option order, numeric IDs, task type and individual yes/no eligibility questions. They are diagnostic variants, not a new benchmark score.

Question check: all twelve have complete supplied facts and a valid reference answer under their stated rules. They cover ordinary workflow selections. Capacity is 7 workers; eligible orders are A/E; the feasible meeting slots are A/B; three satisfied document criteria imply rating 2; the risk sum is 4 points, implying Moderate. The ordinal options were shuffled, as allowed by the API and also used in training; the controls test conventional ascending/descending level order separately. These synthetic examples are not claimed to represent all production requests.

• Multi-choice: the head ranks invalid options above valid ones. For the order task, it gave a held order 94.2% and an unpaid order 94.6%, versus 83.9% for one eligible order. No single monotone threshold can recover the correct set on either tested multi-choice task. Reversing order or using numeric IDs still fails; separate yes/no queries also incorrectly approve the held and unpaid orders.
• Scoring: output depends on option order despite identical facts and meanings. The completeness example originally chose rating 1 rather than 2; ascending and descending level order both produced rating 2. The four-point risk example originally chose Critical; ascending order chose Low and descending order chose the correct Moderate. No single tested formatting change consistently repaired both examples.
• Training coverage: actual weighted draws were 83% single-choice, 10% multi-choice, 4% score and 3% yes/no in expectation. All 11,676 multi-choice training rows were ToolACE next-function selections; scoring had 2,400 teacher-labeled rows from four workflow families. These formats differ from general eligibility, scheduling and explicit arithmetic rubrics. This supports a generalization concern; it does not prove one complete cause of every error.
• Selection and confidence checks: validation/checkpoint selection had no scoring or yes/no examples, and gate fitting/calibration had only single-choice examples. Scoring trained with categorical soft-target cross-entropy rather than a distance-aware ordinal regression objective. Calibration can improve confidence estimates but temperature scaling alone cannot fix these wrong score rankings.

All evaluated original model files, original inputs and published outputs remained unchanged. Qwen and the actual Laya head stayed frozen; no training or model edits occurred. The observed failures belong to the Zeus adapter-plus-head decision path; this test does not separately measure standalone Laya or Qwen generation. Six requests in the original sample escalated; no external fallback was executed. The original gate also accepted the wrong support-priority classification with about 98.7% estimated correctness, showing that its confidence does not carry a reliability guarantee on these new examples.

Verified diagnosis, all controlled outputs, training audit, exact-input rerun, question answerability checks, and audit details preserve every outcome, including unsuccessful variants.

What Zeus is mainly intended for

Zeus is intended for structured decisions over options supplied by an application or agent. Its added decision interface is aimed at tasks such as:

• Tool or action selection: choose the next step from a provided candidate list, using the available context.
• Classification and triage: select a label or route from defined options.
• Single-choice and multi-choice decisions: select one or more supplied answers or actions.
• Structured yes/no and ordinal scoring: return a decision using the supplied options or scale.

These are intended uses, rather than demonstrated improvements over Qwen in agent or tool-use performance. Only single-choice confidence is currently calibrated; other decision types escalate by default. Qwen's frozen native text generation remains available for ordinary chat, writing and coding, but this release has not demonstrated a general capability improvement over that same Qwen backbone.

Why Zeus can score worse than Qwen on reasoning questions

On the balanced 140-question MMLU-Pro sample, Qwen's saved 5-shot reasoning answers scored 118/140 (84.29%), while the full original Zeus system scored 75/140 (53.57%). That is a 30.71 percentage-point decrease on this sample.

• The Qwen baseline received five worked examples and could generate step-by-step reasoning, with an 8,192-token limit and native hidden thinking disabled. Zeus's decision path received the question and options without those demonstrations and returned a direct option selection.
• The unchanged Zeus gate accepted every one of the 140 head decisions, including 65 wrong answers. It requested zero Qwen reasoning fallbacks, so those wrong decisions were returned unchanged. Confidence and escalation calibrated on the supplied decision data did not protect accuracy on this sample.

These observations explain the answering and routing behavior measured here; the evaluation did not isolate a single training-related root cause. Qwen's underlying language-model weights were unchanged. This is a small, equally weighted subject sample and does not establish performance on every task.

Verified comparison report · Accuracy graph

Zeus adds a trained representation adapter and Laya's original frozen decision head to the frozen Qwen3.8-27B-FP8 model. It supports typed option decisions and retains Qwen's native autoregressive generation. This repository contains the complete base-model weights, tokenizer and processor files, the original Laya head, the trained adapter, inference code, training configuration, calibration, evaluation reports and a resumable training checkpoint.

Model components

• Frozen base: Qwen/Qwen3.8-27B-FP8, revision 017b9c7af6b5689d5dd426a76e0bc077eb5ca20a. All checkpoint tensor values are unchanged. The root configuration removes obsolete exclusions for nonexistent MoE modules: their unanchored names otherwise incorrectly exclude dense FP8 gate_proj layers. Real higher-precision exclusions remain intact. Every loaded FP8 matrix's dtype/shape and every associated scale tensor are verified when loading. Optional checkpoint MTP tensors are retained in the release files; the pinned Transformers Qwen class intentionally does not instantiate that speculative-decoding branch. base_integrity_checks.json verifies the downloaded source files against the pinned Hub revision. The release manifest also checks that copied base files retain those hashes, apart from the documented root-configuration correction and this model card.
• Frozen decision head: convaiinnovations/laya, revision 7b928d828b7b0e022f929d9bd2e44165aa270148. Its 26,512,131 parameters are retained; its separate ModernBERT encoder is excluded. The head's architecture and original parameter values are unchanged. laya_provenance.json records the original extracted head checksum and implementation-parity result; publication verifies that the released head still has that checksum. Setup serialized this head twice with a different ordering of the two safetensors metadata fields. laya_serialization_checks.json verifies all 36 tensors against the pinned original, including names, shapes, dtypes and exact values, and records restoration of the original file checksum by reordering metadata only. No tensor payload or trained adapter changed. The check is reproducible with check_laya_serialization.py and the cached original checkpoint.
• Trained adapter: 6,305,792 parameters, LayerNorm(5120)-Linear(5120,1024)-GELU-Linear(1024,1024)-LayerNorm(1024). Matrix weights use Muon; biases and normalization parameters use auxiliary AdamW.
• The adapter consumes Qwen's final hidden states, and the decision head scores option-end representations. A decision forward pass does not compute full-vocabulary language-model logits. The question and options must be present in the encoded input. Qwen prefill is still required.

Training data

The staged training mixture is Flownium/zeus-dataset, revision 5f790bd295d46bde2ddc5e1abc570fc652b679a8. Its sources are:

• Flownium/zeus-30k-trace-dataset
• LocalLLaMA/typed-decisions
• Team-ACE/ToolACE

There are 66,492 training decisions. Source sampling probabilities are 80% Zeus, 10% typed-decisions, and 10% ToolACE. Three effective epochs mean 199,476 weighted draws across 3,117 optimizer updates, not three exact traversals of every row. Trace records are linked context, not an additional disjoint training population. Only recorded reasoning and observations available before each decision boundary are supplied as input.

Validation, gate fitting, calibration and testing use the dataset's separate splits. Single-choice, multi-choice, ordinal-score and yes/no targets retain their different semantics.

Only single-choice and multi-choice are covered by per-epoch validation. Score and yes/no decisions are evaluated on the final test; their generalization is not monitored by the checkpoint-selection split. Gate fitting and calibration cover single-choice only. Other types return uncalibrated option predictions and escalate by default. Exact source/type counts are in heldout_type_coverage.json.

The categorical objective is soft-target cross-entropy where probabilities are supplied; multi-choice uses binary cross-entropy. An auxiliary correctness-estimation loss through the frozen act/escalate head has weight 0.1.

Training configuration

• Both NVIDIA RTX 5090 GPUs, one frozen FP8 backbone partitioned across them.
• Fine-grained Triton FP8 matrix kernels and FLA Triton causal convolution, including cached decoding. The Transformers multi-device DeepGEMM safety guard remains enabled.
• Only adapter parameters receive gradients and optimizer state. Qwen and original Laya parameters remain frozen. BF16 adapter/head computation; FP32 adapter parameters, gradients and optimizer state. Laya's stored FP16 weights are losslessly represented in FP32 at runtime for mixed-precision operations.
• Muon momentum 0.95, Nesterov enabled, five Newton-Schulz iterations, adjust_lr_fn="match_rms_adamw"; one FP32 momentum buffer per matrix.
• Auxiliary AdamW: betas (0.9, 0.999), epsilon 1e-8, no decay on biases and norms.
• Base peak LR 1e-4; WSD schedule: 3% linear warmup, 77% stable, 20% cosine decay to 1e-5.
• Matrix weight decay 0.01; global gradient clipping 1.0; effective batch size 64.
• Length-bucketed microbatches and gradient accumulation. Decision context limit 16,384 tokens. Any truncation is reported; truncated inference requests escalate by default.
• Seed 73019. Best adapter selected by validation decision loss after the complete three-epoch run.

Exact settings, data-preparation statistics, per-case GPU memory checks, software versions and training history are included in JSON reports. Checkpoints include adapter weights, both optimizer states, sampler cursor, scheduler progress and CPU/CUDA RNG states. Frozen base weights are not duplicated inside each training checkpoint. Scheduling was optimized by continuing from update 768 (49,152 draws), preserving adapter and optimizer tensors, scheduler progress, CPU and both CUDA RNG states, and the exact sampler cursor. Updates already completed were not replayed. The first 32 frozen Qwen layers execute on GPU 0 while the previous microbatch's final 32 layers and adapter/head work execute on GPU 1. A bounded producer queue and explicit CUDA events order transfers; a host lock protects the shared Triton autotuner's mutable dispatch state without synchronizing GPU kernels. Larger short-example microbatches failed numerical parity or provided no speed benefit; the selected policy retains a maximum of four examples.

On the same next 64-example draw window at that checkpoint, median forward/backward time was 13.140 seconds before and 7.458 seconds after scheduling changes (1.76 times faster). These short benchmarks exclude optimizer, validation and checkpoint I/O and do not guarantee the same end-to-end speedup. Loss/gradient parity, discarded optimizer-step parity, additional actual draw windows, maximum-context memory, and measured two-device kernel overlap are recorded in pipeline_benchmark.json and pipeline_acceptance_checks.json. performance_config.json pins the accepted policy and checksums; continuation_768_checks.json records the exact restored boundary. Training semantics and precision are unchanged.

The isolated memory checks clear unused allocator caches between cases. Their peak reserved-memory figures are not an upper bound on reserved GPU memory during the full run across varying batch shapes. Recovered CUDA allocation warnings occurred during the initial untrained-adapter validation (three) and epoch-one validation (four). Both passes completed. The independent epoch-one audit confirms all 3,565 expected examples exactly once, finite outputs, matching labels and independently recomputed metrics; see validation_epoch_1_audit.json. The post-training gate-fit pass also logged two recovered allocator warnings. It completed all 2,949 examples; the separate calibration pass and the full held-out test also completed.

The first logged combined training loss was 405.802 at update 1. Training loss includes the auxiliary action loss and uses weighted training sampling; it is not directly comparable with held-out validation decision loss. Pre-clipping gradient norms were logged from update 769; one recorded spike reached 208.046 at update 840. These are sampled log records, not measurements for every optimizer update, so earlier gradient peaks cannot be reconstructed from them. Global clipping is applied before optimizer updates. The checkpoint's history field records validation results, not these per-batch log samples.

Measured evaluation

The same validation split before training and at the selected epoch (3):

Validation contains 3,565 decisions of these types: choice, multi_choice. Checkpoint selection uses this split's decision loss; it does not directly validate task types absent from that split. This before/after comparison is separate from the test metrics below.

The complete per-epoch validation history is:

All three validation passes were independently audited for the same 3,565 expected examples, matching inputs and targets, finite outputs, and recomputed metrics. The reports are validation_epoch_1_audit.json, validation_epoch_2_audit.json, and validation_epoch_3_audit.json; their audit code is training/check_validation_report.py. These repeated validation measurements guide checkpoint selection; they do not prove that overfitting is absent on every task or distribution.

On the untouched test split (5,642 decisions):

These aggregates mix task types; consult evaluation_report.json for task/source breakdowns and selective-decision coverage. Calibration is fitted separately from gradient training and testing. Original Laya weights and its temperature buffer are not modified by calibration. The configured selective-accuracy target is a selection criterion, not a guarantee for new data.

Calibration support varies by task type in the supplied held-out splits. Unsupported types still return option predictions, but escalate by default. See calibration.json for exact support counts. The gate-fitting and calibration splits each contain only seven distinct source groups. Although these groups are isolated from training and testing, this limits the diversity of calibration evidence. prepared_data_checks.json records all split/group counts and tokenization checks.

The baseline report evaluates the untrained adapter, not Qwen-alone accuracy. This release does not establish a general reasoning, coding or mathematics improvement over Qwen. Native generation uses the frozen FP8 base. Domain shift, confidently wrong decisions and finite-context truncation remain limitations. Dataset structural checks are not proof that every source label is factually correct.

Balanced 140-question Qwen–Zeus comparison

This is a small exploratory MMLU-Pro subset: 140 questions, ten per each of 14 subjects. It is not the full 12,032-question benchmark or an official leaderboard score. The full run remains paused. Questions were selected by a fixed seeded hash of question IDs, without inspecting answers or correctness, from 7,296 questions whose Qwen reasoning answers were already saved. Both systems are scored on exactly the same 140 questions. Equal subject weighting differs from the full benchmark's question weighting.

Full original Zeus minus Qwen: -30.71 percentage points, with a stratified paired bootstrap 95% interval of -38.57 to -22.86 points. The two systems selected the same answer on 81/140 questions. Small differences and subject scores are uncertain with this sample size.

Protocol and checks

• Qwen: exact saved outputs from the pinned FP8 Qwen checkpoint, five subject-specific validation demonstrations, greedy visible step-by-step reasoning, native enable_thinking=false, and an 8,192-token output limit. This is not Qwen's default hidden-thinking or maximum-capability score.
• Zeus: a fresh run of the original published Qwen, adapter, Laya head and calibrated gate. The decision head receives the question/options without reasoning demonstrations. The unchanged gate accepted 140 head answers and requested 0 fresh Qwen fallbacks. Any requested fallbacks use the same Qwen reasoning protocol; no baseline answers are substituted. Zero requested fallbacks means the full system returned its head decisions on every question.
• Both RTX 5090s were used for the fresh Zeus decision pass. The released FP8 weights, original BF16 computation, original native head runtime, prompts, calibration and decoding settings were preserved. The reused Qwen outputs were generated with the previously validated two-GPU vLLM configuration.
• Canonical dataset files and gold labels were independently verified against the pinned dataset; checkpoint/code hashes, sample selection, cached Qwen records, all 140 results per system, prompt token hashes, answer parsing, original head/gate behavior and timing ledgers passed independent checks. Unparsed and truncated answers remain in the score; there are no wrong-answer retries or random guesses.
• Qwen answers are reused, so there is no new Qwen timing or comparative speed measurement. Fresh Zeus warm work took 16.25 seconds, comprising 16.25 seconds of decision processing and 0.00 seconds of fallback generation. Loading, preflight checks, hashing, persistence and publication are excluded; this is not end-to-end serving latency.
• Floating-point kernels and batch composition can affect greedy answers. Different prompt interfaces limit direct comparisons. Training-data overlap has not been ruled out. This small academic multiple-choice evaluation does not establish coding/agent quality, deployment throughput or absence of overfitting.

Model revision evaluated: 48f0bb7f9fb1aa31ca18a6620e222f432f01b2d7. Sources: pinned MMLU-Pro dataset and upstream implementation.

Verified report, protocol, sample selection, vector graph, and audit files and predictions.

Usage

Use the dependency versions in environment_versions.json; the reference environment uses CUDA 13.0 and PyTorch 2.14.1+cu130. Do not replace a working CUDA stack without checking compatibility. The default placement expects two CUDA GPUs. FP8 kernels must support the target hardware.

After installing the tested CUDA-enabled PyTorch build in a virtual environment, install the remaining libraries from the downloaded repository directory:

python -m pip install --constraint constraints.txt --requirement requirements.txt

The constraints preserve the tested PyTorch version. These files pin the explicitly used libraries; they are not a complete operating-system or transitive-dependency lockfile. The FP8 kernel package may fetch its supported kernel implementation on first use.

from huggingface_hub import snapshot_download
import sys

path = snapshot_download("Flownium/zeus-28b-v0.1")
sys.path.insert(0, path)
from zeus_model import Zeus

model = Zeus.from_pretrained(path)
decision = model.decide({
    "messages": [{"role": "user", "content": "The test runner returned exit code 0 and all tests passed."}],
    "question": "What did the displayed test result report?",
    "options": [{"id": "passed", "text": "Tests passed"},
                {"id": "failed", "text": "Tests failed"}],
}, task_type="choice")
print(decision)

messages = [{"role": "user", "content": "Explain a Python dictionary briefly."}]
inputs = model.tokenizer.apply_chat_template(messages, tokenize=True,
    add_generation_prompt=True, return_tensors="pt", return_dict=True)
inputs = inputs.to(model.qwen.model.language_model.embed_tokens.weight.device)
tokens = model.generate(**inputs, max_new_tokens=128)
print(model.tokenizer.decode(tokens[0], skip_special_tokens=True))

decide accepts choice, multi_choice, score, and noul. Scores can include ordinal_value on each option. For a yes/no probability, supply positive_option_id explicitly. Returned option IDs retain the caller's semantics. Escalation is a returned decision; this wrapper does not run external tools automatically. score_hidden exposes the branch for callers that already have the matching Qwen hidden states and option-marker layout.

The root Transformers configuration and weight files also support standard Qwen loading for native generation. zeus_model.py is needed to use the added decision branch. The decision API accepts text observations. Native image inputs use Qwen's AutoProcessor and are passed to generate with their image tensors. The release check exercises text generation, each decision type and a small synthetic image-color example; this is a functionality check, not a general multimodal quality benchmark.

Reproducibility and files

training_report.json, evaluation_report.json, calibration.json, memory_checks.json, integration_checks.json, data_preparation.json and environment_versions.json record the run. training/ contains preparation, training and evaluation source plus the resumable final checkpoint, the checkpoint selected by validation, initial adapter and fixed sampler draws. The completion check verifies that the released adapter exactly matches the selected checkpoint and restores both final optimizer states on CPU. Recreate the tokenized data from the pinned dataset revision before resuming on a new host; runtime paths may be adjusted without changing semantic config hashes. The final checkpoint marks three effective epochs complete; extending training is a new configuration.

To reconstruct the training data cache, download the pinned dataset with snapshot_download(repo_id="Flownium/zeus-dataset", repo_type="dataset", revision="5f790bd295d46bde2ddc5e1abc570fc652b679a8"), set ZEUS_DATASET_PATH to that directory, and run python training/prepare_training.py. Run python training/memory_check.py on replacement hardware before python training/train_zeus_optimized.py. The latter detects the completed checkpoint and performs no additional optimizer updates. Published code and the deterministic sampler must match the checkpoint.

artifact_manifest.json lists file sizes and cryptographic hashes for the release payload, excluding the manifest itself. The publisher verifies all uploaded paths, sizes and hashes against the committed Hub revision. Base-model and Laya provenance are pinned above. Laya head implementation attribution and its Apache-2.0 license are included in LICENSE.laya and source comments.