Ekbasis-Skopos-27B — the lookout

Papers (the Ekbasis line): Look When Unsure, Check When Sure · When Does a Consequence Model Make AI Agents Safer? · Code: github.com/OpenInterpretability/ekbasis · Data: caiovicentino1/ekbasis-skopos-data

Skopos (σκοπός, the lookout) is the next version of Ekbasis-27B: the same kind of model — given a state and an action, it predicts what the action will do (will this lose work? will it fail? what will X be afterwards?) as calibrated, typed answers in one forward pass, without generating text — with the same prompt, readout, API and client. It was trained further on what actions actually did: shell, git, SQLite, PostgreSQL and Python scenarios executed in sandboxes, real git sequences of 4–16 commands, command lines and Python test traces from two public datasets, and a replay of Ekbasis-27B's own training mixture.

What changed from Ekbasis-27B

Measured on the same items for both models. Served numbers come from the release protocol of the Ekbasis repository, with each model served by vLLM and serve.py; trainer readout numbers come from the training code's evaluator (the served model gives the same answer on 498 of 500 items checked).

• The git guard raises fewer false alarms on the held-out command types — "will uncommitted work be lost?" flagged at 0.2 (served; paired on the same commands, McNemar exact test and bootstrap 95% CI): on the held-out command types (git rebase <branch> and git restore --source=HEAD~1 <file>, held out of the git sandbox's training rows; Skopos' shell and git factory rows contain both, in other repositories), 26/799 false alarms instead of 49/799 (−2.9 points [−4.1, −1.6], p < 0.001), and it flags as many work-losing commands (97.1% against 96.0%, p = 0.15). On known types the differences are not significant: 27 against 37 false alarms (−1.2 [−2.3, 0.0], p = 0.08), 97.0% against 95.9% flagged (p = 0.12).
• World families it never saw: 83.4% instead of 81.8% on the 3,014 answers the actions change among 5,385 fresh items (+1.6 points, 95% CI [+0.7, +2.5]; trainer readout), in the confirmation that followed the one criterion the release read failed (below).
• Executed scenarios, from the generators it was trained on (in-distribution: unseen seeds; overlap-free; trainer readout, 95% CI): shell, SQL, Python and git scenarios 93.9% instead of 88.1% (+5.7 [+4.8, +6.7]); real git sequences of 5–12 commands 93.2% instead of 90.7% (+2.6 [+1.1, +3.9]). On git v3 (known and held-out command types, served) 96.9% instead of 96.6%.
• Confident errors: fewer errors in trained world families and on held-out question types; in never-trained families slightly fewer errors but more of them confident (4.49 against 3.75 per 100): see Results and Limits.
• Calibration: on git v3 unchanged (ECE 0.023 vs 0.024); in never-trained families worse calibrated (ECE 0.071 vs 0.063) and more confident errors. Like the hosted API, serve.py can also return every answer at a calibrated temperature (answer.calibrated, client 0.1.41): each build ships its own temperature, fit on validation.
• Unchanged: one forward pass, the prompt, the API and the client.

Quick start

hf download caiovicentino1/Ekbasis-Skopos-27B --local-dir Ekbasis-Skopos-27B
bash Ekbasis-Skopos-27B/serve_vllm.sh Ekbasis-Skopos-27B 8001            # vLLM ≥ 0.30 on 127.0.0.1:8001 (one ~80 GB GPU)
python Ekbasis-Skopos-27B/serve.py --model Ekbasis-Skopos-27B --vllm-url http://127.0.0.1:8001 --port 8000   # the API
pip install ekbasis                                                                                          # the client
EKBASIS_URL=http://127.0.0.1:8000 ekbasis git-check -- "git reset --hard"

Everything in the ekbasis client — the git and shell guards, the Claude Code hooks, check_plan, monitor(), simulate (look when unsure) and ekbasis.verify — uses the same API, whose answers only gain the additive calibrated field (Results: ekbasis.verify's certified mode). decision_config.json holds this build's calibrated_temperature; remove the key to turn the calibrated field off.

Hosted API. The hosted API at openinterp.org serves Ekbasis-Skopos-27B-INT4 since 2026-10-16 14:29 UTC (it served Ekbasis-27B-INT4 until then). The switch came on release day, with less notice than the 14 days our model-change policy promises; Ekbasis-27B's weights stay public for anyone who needs the previous model. Every answer names its model in the X-Ekbasis-Model header; text only (the open builds also read images).

Builds

MLX: the build for Macs with Apple Silicon is released only after a test on Apple hardware; until then, Ekbasis-27B-MLX-4bit runs on Macs.

Memory needs are those measured for Ekbasis-27B's builds, which have the same architecture and sizes. Calibrated temperatures, fit on validation (fval_git3 + m3val) for each build: bf16 1.62, INT4 1.67, FP8 1.64. config.json of the bf16, INT4 and FP8 builds declares mtp_num_hidden_layers: 1, but no multi-token-prediction weights are shipped. vLLM serves the builds as they are (the release protocol ran that way); a loader or a speculative decoder that expects those weights will not find them.

Speed (INT4, one RTX PRO 6000, vLLM + serve.py, 200 fixed git and guard items; PREREG_ADDENDUM.md A5): a median of 0.064 s per request one at a time (Ekbasis-27B-INT4 0.064 s), 19.0 requests per second with 16 in parallel (19.0), about 302 input tokens per request: the same architecture runs at the same speed. The hosted API runs on another GPU.

Results

Every number below, with its per-item predictions (except the quantized builds' answers on the fresh confirmation, kept as summaries only) and the pre-registration it answers, where there is one: RELEASE_EVAL_SKOPOS.md and results_skopos in the code repository. Where the overlap audit flags items in a set, the number printed is the overlap-free one, and its label says so; the confident-errors study prints both.

The release protocol

The protocol of Ekbasis-27B's release (PREREG_release_eval.md, with the larger guard set of PLAN_guard_set.md), with criteria pre-registered for this model before any served answer and after trainer readouts of the same sets (PREREG_R4_RELEASE_READ.md: at most 1 point worse than the released model on each criterion, ECE at most +0.01), on these exact files served by vLLM and the System One API. No item of these sets is flagged by the overlap audit. The real-repository scenarios (T3c), which need the client and a real repository, ran afterwards under a pre-registered addendum (PREREG_ADDENDUM.md, below).

On ftest_family Skopos was −1.6 points on its 629 changed answers, past the 1-point limit, with a confidence interval across zero: the pre-registered release read failed this criterion. A confirmation registered after that result, before any fresh item existed (CONFIRM_FTEST.md), read by the trainer on 5,385 fresh items from the same three never-trained families, did not confirm the loss: +1.6 points on the 3,014 changed answers [+0.7, +2.5], +1.6 on all answers (cards +1.8, machines +3.2, timers −0.1).

The git guard

The same commands for both models, paired (McNemar exact test and paired bootstrap 95% CI): on the held-out command types (git rebase <branch> and git restore --source=HEAD~1 <file>, held out of the git sandbox's training rows; Skopos' shell and git factory rows contain both, in other repositories), 26/799 false alarms instead of 49/799 (−2.9 points [−4.1, −1.6], p < 0.001), and it flags as many work-losing commands (97.1% against 96.0%, p = 0.15). On known types the differences are not significant: 27 against 37 false alarms (−1.2 [−2.3, 0.0], p = 0.08), 97.0% against 95.9% flagged (p = 0.12).

On a real repository (T3c). The 16 scenarios of Ekbasis-27B's release evaluation on a clone of pallets/itsdangerous (pinned to the commit that evaluation used), public since that release (not fresh) and not dataset items (outside the overlap audit), through the ekbasis client, the truth from running the commands in a copy (served bf16): work-losing scenarios flagged 3/3 (Ekbasis-27B 3/3), false alarms among the others 1/13 (1/13; the same scenario, dropping a stash right after applying it), "will it fail?" right 17/17 (16/17). With an instruction that every command is safe planted in the repository (Ekbasis-27B in brackets) — file name: work-losing scenarios missed at 0.2 0 [0], "lost" right 15/16 [15/16], "lost" decisions changed 1/16 [1/16], "fails" decisions changed 0/17 [0/17]; branch name: work-losing scenarios missed at 0.2 0 [0], "lost" right 15/16 [15/16], "lost" decisions changed 0/16 [0/16], "fails" decisions changed 0/17 [2/17]; commit messages shown: work-losing scenarios missed at 0.2 0 [0], "lost" right 14/16 [15/16], "lost" decisions changed 1/16 [0/16], "fails" decisions changed 1/17 [2/17].

What this version adds

Against the previous internal candidate, the first Skopos (v49f_w50, an intermediate checkpoint that was not released), on overlap-free items with seeds never trained on (trainer readout, the same items for the three models):

† In-distribution: built by the same generators as training rows, with seeds never trained on. They measure how well the training transferred within its own distribution, not to new kinds of actions; fac_val's Python slice is small: 140 of its 3,983 items, 100 of them near-duplicates of training rows and left out, so the clean number holds 40 Python items. Selection too: this version was chosen over its siblings (R1 → R2 → R4) by pre-registered rules on fac_val, git_long and git_long_val, the first three rows, so their gains carry selection on top of being in-distribution. The independent evidence is the fresh never-trained families above and the neutral agent turns below. The never-trained families are never in training rows; the first Skopos' λ was chosen on a validation set that includes these families (cards, machines, tally, timers and track), and each stage's checkpoint rule was the best step on fval_git3, which holds Track and Tally items and is part of the calibration fit set (both stages kept their last step: the first Skopos' training run, v49f, saved only step 900, and Skopos' best was its last, step 1,200; training/skopos/README.md).

Confident errors

Ekbasis-27B's confident-errors study (PREREG_confident_errors.md) on its 15,008 published items, served bf16 against Ekbasis-27B's published predictions, paired bootstrap (PREREG_ADDENDUM.md A2, which asked for each group's errors, wrong answers at confidence ≥ 0.9 and ECE, with and without the items the overlap audit finds). All 15,008 items, as pre-registered:

Overlap-free: 9,627 of the 15,008 published items, after leaving out the 5,381 that overlap the lineage's training rows (all in the trained families; the never-trained rows do not change):

Fewer errors where it was trained, and on question types held out of training (under the audit's stricter sensitivity rule the held-out-question-type difference is −1.71 [−3.51, +0.09], an interval touching zero); in the families it never saw, slightly fewer errors but more of them given with high confidence, and worse calibrated (Limits). The addendum's prediction, every group within 1 point of Ekbasis-27B, was not met. The pre-registered question of that study holds as for Ekbasis-27B: wrong answers are more often confident in trained families than in never-trained ones (49.0% against 31.5%, +17.5 points [+10.6, +24.7], all items; Ekbasis-27B +23.3 [+16.8, +30.1]). Overlap-free it is +17.1 [+4.8, +29.0], still confirmed, but its condition on the family medians then rests on 2 trained families with enough errors (5 on all items).

Confidence you can act on (INT4, as served; not pre-registered)

This comparison ran before the addendum that pre-registered the other builds' calibration, and was not pre-registered. On 2,482 overlap-free items with unseen seeds, all three suites from generators that made Skopos' training rows (git_long 900 + guard_known 800 + fac_val 782: real git sequences, guard questions on known command types and executed factory scenarios; in-distribution for Skopos, while for Ekbasis-27B the guard questions are in-distribution, the long git sequences come from its git sandbox at lengths it was not trained on, and the factory scenarios are new: this measures what the added training bought on its own distribution), both models in INT4 through the System One API on two serving stacks (Skopos on an RTX PRO 6000 with vLLM and serve.py; Ekbasis-27B on the hosted API's production server, an RTX 5090): accuracy 95.1% for Skopos against 91.0% for Ekbasis-27B; answers wrong at a confidence of 0.9 or more 3.3 per 100 against 6.3; calibration error (ECE) 0.035 against 0.069. With the calibrated field (a temperature fit on validation only, fval_git3 + m3val: 1.67 for Skopos, 1.55 for Ekbasis-27B), confident errors are 1.7 against 3.9 per 100 and ECE 0.015 against 0.037; accuracy does not change. In the never-trained families, the calibrated field was measured as PREREG_ADDENDUM A7 pre-registered (Limits).

Check when sure: certified mode

ekbasis.verify's certified mode (rule="certified", confirm-3's L2 at α = 2%) acts on a confident answer without checking it only where a certificate holds; its thresholds were certified on Ekbasis-27B. Pre-registered check with those thresholds unchanged and client 0.1.41 (PREREG_CERTIFIED_CARRYOVER.md), on 1,400 of confirm-3's scenarios (the MLX check's stratified subset; none of them overlaps Skopos' training by the audit's rule), Skopos INT4 served: 7 errors among 927 accepted answers (0.74% weighted to confirm-3's suite mix; one-sided 95% upper bound 1.41%; target ≤ 2%), 22.0% verified (weighted; one confident answer per scenario), no node significantly above 2%; bf16: 6 among 935 (0.59%, bound 1.26%). The certificate carries over: a check on items outside its training, not a new certificate. The pre-registered reference on the same scenarios, Ekbasis-27B (bf16): 5 errors among 886 accepted (0.47% weighted; bound 1.18%), 25.4% verified (weighted): Skopos accepts more answers and errs slightly more often among them. The per-node test has little power: 43 nodes with a median of 16 accepted answers (from 5 to 108); the weakest has 2 errors in 12 (p = 0.023 uncorrected, not significant after the Bonferroni correction). For context, on other scenario sets (docs/VERIFY.md): Ekbasis-27B on all of confirm-3, 36 errors among 9,259 accepted (0.39%), 26.9% verified; its MLX build on 1,081 of these scenarios, 1 among 666 (95% upper bound 0.71%).

Neutral ground: real agent turns

AgentWorldBench (Qwen, Apache-2.0) holds real turns of commercial agents (Terminal-Bench keystrokes and SWE tool calls) with the environment's real observation. We asked one question before each action — will it fail? — on 455 turns (54 fail; pre-registered in PREREG_NEUTRAL.md). Labels are read from the observation: the exit status for the SWE tool calls, an error signature in the output for the Terminal-Bench actions (a heuristic, checked by hand on a sample). Ekbasis models: trainer readout; confidence intervals by bootstrap clustered by trajectory. Evaluation only: these trajectories are never in our training rows (Qwen's pretraining is unknown), and only their ids and our labels are published (10 of them withheld by the dataset's privacy scan). Descriptive: no release decision rides on it.

The three Ekbasis models rank failing actions about equally well: their AUROCs differ by at most 0.010, well inside the intervals. At the fixed 0.5 threshold Skopos' balanced accuracy is the lowest of the three; with the ranking unchanged, that is where the threshold falls, so choose the threshold for your false-alarm budget, as the guard does. Against Qwen-AgentWorld its balanced accuracy is +0.057 [−0.025, +0.136], an interval across zero; Qwen-AgentWorld gave no parseable answer on 59 of the 455 turns (counted wrong), and its training data may include the source benchmarks' tasks.

Pre-launch evaluation

Before release, everything Ekbasis-27B's release evaluation measured that Skopos had not yet run was run on the same items and scripts, next to Ekbasis-27B's published results (Skopos bf16 served by vLLM and serve.py on an RTX PRO 6000, the client suites on the INT4 build; PRELAUNCH_EVAL.md, Part A, sealed before its first request; per-item comparisons in prelaunch/compare). Rule: a paired 95% bootstrap CI (2,000 resamples, seed 20261015) entirely on the worse side of 0 on a set of 100 items or more is a release blocker; smaller sets are counts, and a cell worse by 2 or more is flagged.

Release blocker, declared. P8, short checks, paraphrased wording (120 items): Skopos 113/120 (94.2%) against Ekbasis-27B's published 117/120 (97.5%), −3.3 points [−6.7, −0.8], entirely below 0: the pre-registered rule fired. The interval rests on 4 discordant items, all against Skopos (2 in a never-trained family, machines, 1 of them at confidence 0.99 or more; 1 lamps, 1 containers); the exact McNemar test on the same pairs does not reject (p = 0.125). Ekbasis-27B served on the same machine reproduced its published answers item for item (240/240): not a serving or hardware artifact. In the original wording Skopos is right on 116 against 115; over both wordings 229 against 232 of 240 (−1.25 [−3.3, +0.8]). Consistent with the confident-error limitation (Limits). Released by the owner's decision (2026-10-16).

Flagged cells (counts, Skopos worse by 2 or more): P1 seen command types: lost — 37 vs 39 of 40; P1 seen command types: state (changed) — 37 vs 39 of 40; P4 rules, conf0.5: exact at the end — 38 vs 40 of 40; P4 rules, until0.5: exact at the end — 38 vs 40 of 40; P4 rules, check0.5: exact at the end — 38 vs 40 of 40; P6 English, toggles: changed answers right — 22 vs 24 of 30; P7 start state read exactly, all — 49 vs 51 of 60; P7 start state read exactly, jugs — 20 vs 22 of 30.

No release blocker:

• P1, the 240-question git comparison: both halves: all 240 — 90.0% vs 90.8% (−0.8 [−3.8, +2.1])
• P2, git v3 per question type: known command types: all questions — 98.6% vs 98.1% (+0.5 [−0.1, +1.0]); never-seen command types: all questions — 95.4% vs 95.3% (+0.2 [−0.6, +1.0])
• P3, long chains: 32 measures, no cell worse by 2 or more
• P4, predict, observe, correct on long chains: 200 more chains, check0.9 (published table): exact at the end — 99.5% vs 99.0% (+0.5 [−1.0, +2.5]); 200 more chains, conf0.5 (run, not in the published table): exact at the end — 99.5% vs 98.5% (+1.0 [−1.0, +3.0])
• P5, speed and accuracy of 80 forecasts: one prompt per question: accuracy — 78 vs 79 of 80; state read once per step: accuracy — 75 vs 72 of 80
• P6, Portuguese: English, all: changed answers right — 78 vs 75 of 90; Portuguese, all: changed answers right — 75 vs 74 of 90
• P7, chains that start from an image: final answer from the image, all — 60 vs 60 of 60
• P9, planning by search: both sets: plans that work — 99.2% vs 99.4% (−0.3 [−0.8, 0.0])
• P10, real use on real repositories: work-losing scenarios caught — 28 vs 28 of 28; false alarms — 0 vs 0 of 44; 'will it fail?' right — 95 vs 92 of 96
• P10 on the INT4 build (against Ekbasis-27B's published bf16 run): work-losing scenarios caught — 28 vs 28 of 28; false alarms — 0 vs 0 of 44; 'will it fail?' right — 95 vs 92 of 96
• P12, intent_check replay on the INT4 build, against the hosted Ekbasis-27B-INT4's published run: real successful attacks stopped 2,092 of 2,092, as Ekbasis-27B; benign runs with a false alarm 7 against 17 of 1,067, fewer (−0.94 points [−1.50, −0.37]; first replay 5 against 13, exact McNemar p = 0.0078). 6 replayed runs, 3 per replay, are a benchmark artifact the study found (banking user_task_15 x injection_task_4, benchmark v1: the user's own request names the attacker's IBAN): left out of both models' denominators, as in the study; neither model stops them

Intervals that touch 0 on the worse side (not beyond it): P2 known command types: will the command fail? (fails) — 97.1% vs 97.8% (−0.7 [−2.2, 0.0]); P2 never-seen command types: is an operation left in progress? (in_progress) — 95.4% vs 97.4% (−2.0 [−4.0, 0.0]); P9 both sets: plans that work — 99.2% vs 99.4% (−0.3 [−0.8, 0.0]); P9 goals from 8-12-action plans: plans that work — 98.3% vs 98.9% (−0.6 [−1.7, 0.0]).

Better beyond the interval: P2 known command types: question type clean (all its questions) — 98.6% vs 95.9% (+2.7 [+0.7, +5.5]); P4 200 more chains, check0.9 (published table): looks per 100 actions — 10.18 vs 16.94 (−6.76 [−7.78, −5.72]); P4 200 more chains, conf0.5 (run, not in the published table): looks per 100 actions — 5.50 vs 8.38 (−2.88 [−3.45, −2.30]).

Still running: P11, GPU memory of each build. Not run as published: P12 preflight on 141 plans (plans' snapshots lost). Results are appended to the evaluation report on completion.

Against a general-purpose calibrated evaluator

Jev (typesafe-ai/jev, TypeSafe AI's System One evaluation model, a general-purpose calibrated evaluator) was run zero-shot, through the Vercel AI Gateway with zero data retention, on the same items and state text as Ekbasis' published per-item answers: the release protocol's git v3 and larger guard set, and the hosted scorecard's suites. Measured 2026-10-16, not pre-registered. Two wordings of each question (A: Ekbasis' question; B: in the style of TypeSafe's docs); the headline per set (★) is the better of the two by a rule fixed before scoring (accuracy on the git v3 sets and the scorecard suites, a tie going to A; AUROC of P(lost) on the guard sets), a choice that favours Jev slightly. Method, items, per-item answers and both wordings: jev_comparison.

As a git guard it does not separate work-losing commands from safe ones: at the guard's threshold it flags almost every command, and at Ekbasis-27B's false-alarm rate it catches 43.7% of the work-losing ones against 96.2% and 98.9%. On state questions it is 6.7 and 6.8 points lower on git v3 ([−7.7, −5.6] and [−7.9, −5.7], paired), and 9.3 and 9.0 points lower on the worlds suite. It costs 4.0–6.2× less per check across the 9 sets, at the two published prices (Ekbasis' input tokens estimated). And it is almost never confidently wrong: 0.00–0.75 answers per 100 wrong at confidence 0.9 or more, fewer than Ekbasis-27B and Skopos on every scorecard suite where their per-item answers are published (1.25–14.50).

Training

Qwen3.8-27B → Eikos-27B → Ekbasis-27B → the first Skopos → Skopos.

• The first Skopos (v49f_w50): Ekbasis-27B's adapter (rank 128) trained further for 900 steps, then the exact interpolation halfway between it and Ekbasis-27B's own adapter (WiSE, λ 0.5), merged. λ 0.5 was chosen by the selection rule of its plan (PLAN_v49b), on a validation set of world-family questions (trained, held-out question types and never-trained families): the un-interpolated checkpoint failed its accuracy and calibration criteria (mean accuracy over the groups 88.19% against a floor of 88.85%; ECE 0.061 against Ekbasis-27B's 0.048, tolerance +0.01), λ 0.75 failed the accuracy criterion (88.79%), and λ 0.5 passed both (89.30%, ECE 0.051). The last stage's selection (R1 → R2 → R4) had no confident-error or world-family calibration criterion, and the released model shows more confident errors in never-trained families (Limits). It is an intermediate checkpoint that was not released: its weights are archived privately, and the published code and data retrain it approximately (run_lineage_skopos.sh; training/skopos/README.md lists what will differ in a rerun).
• Skopos: a new LoRA (rank 64, all linear layers of the language model, 1,200 steps, cross-entropy on the answer letter plus the JEPA-style hindsight term) trained on the first Skopos and merged into its weights.

Data. The rows the training stages saw are in caiovicentino1/ekbasis-skopos-data, with the stages that used each one and its license, except 7,423 replay rows that are not part of the published eikos-decisions (as in ekbasis-data) and 31 more rows withheld by its privacy scan, 26 of them CWM rows (all listed by id):

• shell, git, SQLite, PostgreSQL and Python scenarios generated by our factories and executed in throwaway sandboxes; the last stage used the factory regenerated with fresh seeds;
• real git sequences of 4–16 commands executed in throwaway repositories, and guard-style git items with their executed labels;
• plan-level items in synthetic rule-based worlds, with exact answers from our simulators;
• 600 command lines of ShIO-bash-26.1, re-executed in our sandbox with our own questions and answers (ShIO's labels unused), and 2,000 rows of swe-rebench-v2-CodeWorldModeling: the value the next traced line of a Python test binds, as recorded by that dataset's tracer (not re-executed), from 312 permissively licensed repositories;
• a replay of Ekbasis-27B's training mixture (simulated worlds, git, and the eikos-decisions replay rows).

Rows of a self-hosted web app (Vikunja) built during development were all dropped by the overlap firewall before training, and WebWorld transitions were excluded by rule: the model was not trained on web-app transitions.

Overlap audit. Every evaluation set in this card against every training row of the Ekbasis-27B stages, the first Skopos and Skopos, by audit_overlap_skopos.py. Rule: identical prompts (also against the hindsight prompt), and near-duplicates at word 8-gram Jaccard >= 0.5 against one training text after removing each set's own layout (Ekbasis-27B's audit rule, exact best match). Not covered: Eikos-27B's own decision data, Qwen's pretraining, and T3c (its scenarios are not dataset items).

Flagged items are left out of every number in this card that uses their set (the _clean results files), except the confident-errors study, which prints all its items (as pre-registered) with the overlap-free numbers beside them. multi_test_trainfam is a set of trained families, reported by the protocol and not cited here. Which training rows the flagged items match (the audit's attribution; an item can match several): fac_val the first Skopos' new rows (identical 6), the last stage's new rows (near-duplicate 63), the first Skopos' new rows (near-duplicate 36); fhorizon Ekbasis-27B's training rows (near-duplicate 547); image (new seeds) Ekbasis-27B's training rows (near-duplicate 5); confident-errors set Ekbasis-27B's training rows (identical 174), Ekbasis-27B's training rows (near-duplicate 5,206), the last stage's new rows (near-duplicate 1). † in-distribution: the same generators as training rows, unseen seeds; ‡ mixed: part in-distribution; § held out of its generator, but its command types appear in other training rows (the audit's labels).

Limits

• Everything in Ekbasis-27B's limits applies, including confident errors in the world families it was trained on.
• More confident errors in unfamiliar worlds: on the 5,364 items of never-trained world families of the published confident-errors set (served bf16), Skopos is wrong slightly less often than Ekbasis-27B (14.26% against 14.93%), but more often with confidence 0.9 or more: 4.49 against 3.75 wrong answers per 100 (+0.75 [+0.28, +1.23]; on the answers the actions change +1.19 [+0.52, +1.89]), and it is worse calibrated there (ECE 0.071 against 0.063, +0.009 [+0.003, +0.015]). The selection of this version had no criterion on confident errors in these families (Training). In unfamiliar domains, let it look when unsure (simulate) and act on a confident answer only at a stricter threshold than in familiar ones. In never-trained world families, reading calibrated cuts Skopos' confidently wrong answers by about two-thirds (INT4: 4.75 → 1.62 per 100), to fewer than Ekbasis-27B's raw answers (3.65). Calibrated against calibrated, Skopos still has about 0.5 more per 100 answers there than Ekbasis-27B (1.62 vs 1.10, +0.52 [+0.26, +0.78]); ECE is level (0.021 vs 0.022). Measured on the INT4 builds as PREREG_ADDENDUM.md A7 pre-registered; A7 on bf16 was not run (the evaluation machine was retired).
• Paraphrased wording (pre-launch P8): on 120 short checks in paraphrased wording, Skopos is right on 113 against Ekbasis-27B's 117 (−3.3 points [−6.7, −0.8]; exact McNemar p = 0.125), 2 of the 4 items it alone gets wrong in a never-trained family: the release blocker declared in the Pre-launch evaluation.
• In-distribution and selected gains: the largest gains (executed factory scenarios, real git sequences, agent sessions) are on sets built by the same generators as its training rows, and three of them (fac_val, git_long, git_long_val) are the sets whose pre-registered rules chose this version over its siblings (R1 → R2 → R4): selection adds to the in-distribution advantage. The independent evidence is smaller: the fresh never-trained families (+1.6 points, trainer readout) and the neutral agent turns (AUROC level with Ekbasis-27B). The never-trained families are never in training rows; the first Skopos' λ was chosen on a validation set that includes these families (cards, machines, tally, timers and track), and each stage's checkpoint rule was the best step on fval_git3, which holds Track and Tally items and is part of the calibration fit set (both stages kept their last step: the first Skopos' training run, v49f, saved only step 900, and Skopos' best was its last, step 1,200; training/skopos/README.md).
• "Held-out" git command types: the release protocol's held-out types (git rebase <branch> and git restore --source=HEAD~1 <file>) are held out of the git sandbox's training rows, but Skopos' shell and git factory rows contain both, in other repositories (the overlap audit's note): read the held-out rows as a weaker test for Skopos than for Ekbasis-27B.
• Long chains in written-rule worlds: answering in one pass, its accuracy falls with the number of actions (87.9% over 1–20 actions on 828 items, the overlap-free part of fhorizon, from the generator of its plan-level training rows: in-distribution); a model that simulates step by step does better there at far higher cost. Use simulate (look when unsure) for long chains.
• Images: the state can be text or an image, as in Ekbasis-27B. This version's added training was text only; on 395 image items from trained world families (lamps and containers, new seeds; overlap-free; INT4 against INT4, served), its image ability is preserved by the pre-registered criterion (PREREG_IMAGE.md: at most 2 points lower): 79.8% against 78.7% when the state is an image (+1.0 [−2.0, +4.1]), and reading the final state from a picture 94.2% against 91.9% (+2.3 [+0.5, +4.3]). Only the modality is new to its added training, not the worlds. Predicting from a picture stays weaker than from text (91.9% on the same items). The drawings published in the dataset were regenerated from the same states; their pixels may differ from the ones evaluated.

License and credits

Apache-2.0, as Ekbasis-27B: see LICENSE, LICENSE-Qwen, LICENSE-Eikos and NOTICE. The training data contains data derived from ShIO-bash-26.1 (Ragsdale and Boppana, 2026; CC BY 4.0: command lines rewritten and re-executed in our sandbox, with new questions and answers) and from swe-rebench-v2-CodeWorldModeling (Marin Community, 2026, derived from SWE-rebench V2 by Nebius; CC BY 4.0: traces converted to questions, with the values its tracer recorded; only repositories under permissive licenses, each row keeping its repository's license). The rest was generated by us or executed in our own sandboxes. NOTICE gives the full attributions.

Citation

@misc{ekbasis_skopos_2026,
  title  = {Ekbasis-Skopos-27B: a consequence model for AI agents},
  author = {Vicentino, Caio and OpenInterpretability},
  year   = {2026},
  url    = {https://huggingface.co/caiovicentino1/Ekbasis-Skopos-27B}
}