Code Oracle: Laya ModernBERT Decision Head Suite

Fine-tuned decision head models for Code Oracle (Sub-50ms Neuro-Symbolic Verification Oracle for AI Coding Agents).

This repository hosts two distinct model variants trained on authentic multi-language AST graphs across 4 Tier 1 programming languages (Python, TypeScript, Go, and Rust) to produce dual-decision verdicts (APPROVED vs REJECTED) and continuous calibrated risk scores (0.0 to 1.0).

Model Variants

Model Details

• Large Architecture: Laya ModernBERT 421M (convaiinnovations/laya subfolder typed-decisions)
• Base Architecture: Laya ModernBERT-base 164M (answerdotai/ModernBERT-base + Decision Head)
• Base Model License: Apache 2.0
• Fine-tuned By: Wahyu Febri Tamtomo (frugaldev.biz.id)
• Training Task: Dual-decision verification & continuous risk scoring over compact Micro-DSL (< 400 tokens)
• Dataset: 2,400 balanced multi-language mutation samples (Python, TypeScript, Go, Rust) with 50/50 PASS/REJECT parity.

Intended Use

Integrated directly into code-oracle as an in-memory neural decision head paired with deterministic symbolic gates (Tarjan's SCC cycle detector and AST contract invariant checkers).

Quick Usage with Laya / Transformers

1. Load Lightweight Base Variant (164M, ~312 MB) - Recommended for Desktop / Local CLI

import laya

# Loads the lightweight 312 MB base model
agent = laya.load("wxsys/code-oracle-laya-421m", subfolder="base-164m")

2. Load Default Large Variant (421M, 1.68 GB)

import laya

# Loads the full-scale 421M large model
agent = laya.load("wxsys/code-oracle-laya-421m")

3. Inference Example

dsl_prompt = """[DIFF_TARGET] src/calc.py::add (MODIFIED)
[METADATA] File: src/calc.py | OldLines: [1..2] | NewLines: [1..3] | Nodes: 2 | Edges: 1
[NODES]
N0: src/calc.py::add [def add(a: int, b: int = 1) -> int] (SEED, MODIFIED)
N1: src/calc.py::compute [def compute(x: int)] (CALLER)
[EDGES]
N1 -> N0 [CALLS]
[GATE]
STATUS: APPROVED (conf: 0.98)
CYCLES: 0
VIOLATIONS: NONE"""

questions = {
    "status": {
        "type": "choice",
        "instructions": "Determine if the proposed patch is valid and safe to apply.",
        "criteria": {
            "APPROVED": "The code patch preserves all AST topological invariants, interface contracts, and call signatures.",
            "REJECTED": "The code patch introduces circular dependencies, arity mismatches, broken references, or syntax errors."
        }
    },
    "risk": {
        "type": "score",
        "instructions": "Calibrate the risk level of applying this code modification.",
        "criteria": [
            "level 0: Zero risk - purely cosmetic or additive with default parameters.",
            "level 1: Low risk - well-typed modifications with full backward compatibility.",
            "level 2: Medium risk - refactoring with multi-call graph dependency propagation.",
            "level 3: High risk - potential broken callers or semantic contract drift.",
            "level 4: Critical risk - cyclic import loops or fatal signature violations."
        ]
    }
}

result = agent.predict(dsl_prompt, questions)
print("Verdict:", result["answers"]["status"]["choice"])
print("Risk Score:", float(result["answers"]["risk"]["score"]) / 4.0)

Attribution & License

• Fine-tuned derivative work of convaiinnovations/laya and answerdotai/ModernBERT-base.
• Released under the Apache-2.0 License.