Banyan Sapling

Semantic verification for policy-driven AI systems.

Banyan Sapling is a research preview from MubitLabs and the first model in the Banyan family. It evaluates whether a claim is supported by a supplied policy and evidence, with separate outputs for policy conflict, missing information, and inconsistent context.

Sapling is designed as a verification component for developers building policy checks, evidence review, and claim-validation workflows. Its structured output makes both the preferred classification and calibrated uncertainty available to the surrounding application.

Model overview

Policy, evidence, and claim spans are pooled separately. The interaction head compares their representations to produce four class logits. Native positional encoding is preserved. The evaluated inference path processes one example at a time.

Inputs and outputs

Provide three nonempty strings: policy, evidence, and claim.

raw_label is the highest-scoring class. Calibrated probabilities and the prediction set describe the model's uncertainty. status is the calibrated result and may be unsure when no unique decision is available. An unresolved result is distinct from the substantive insufficient_information class. A narrow rounding-related empty-set fallback preserves the argmax result; it does not resolve a prediction set containing multiple labels.

Sapling classifies the supplied text. Retrieval, source authentication, generated explanations, and final business decisions belong to the surrounding workflow.

Run the model

The weights package includes the checkpoint, tokenizer, calibration, bundled inference code, pinned requirements, and file checksums. Follow the weights guide to download and run an example. Use the bundled loader: the verification head and factorized adapters require reconstruction beyond loading the base Qwen architecture.

The reference environment uses Python 3.12, CUDA 13.0, and an NVIDIA H200. The BF16 checkpoint occupies approximately 46 GB on disk, with additional memory required for inference. Hardware and input length affect memory use. The reference runtime is recorded in evidence/training_runtime.json; the weights package includes the complete pinned dependency list.

Training and calibration

The final run completed one epoch over 50,000 examples: 42,809 controlled synthetic cases and 7,191 ContractNLI training claims, across 21,333 optimizer updates. Training adapted the attention projections and interaction head while keeping routers frozen.

Temperature fitting and conformal calibration use separate partitions of 1,360 examples each, balanced between synthetic and contract cases. Temperature is bounded below by 0.5; the fitted value is 5.171578. The shared nonconformity threshold is 0.995478, taking the more conservative domain threshold.

Evaluation

All 779 answered cases matched their reference labels in this evaluation; 952 cases were unresolved. There were zero false supported decisions in the calibrated output and 14 in raw predictions. These figures describe this particular evaluation and should be read together with answer coverage.

The domain gap is material: the current shared calibration leaves most contract cases unresolved. This preview is intended for integration experiments and domain evaluation, with controlled policy scenarios providing the stronger initial fit. Broader contract use requires additional data and calibration work.

The evaluation includes prior benchmark exposure and correlated document claims. It is a regression and development characterization rather than an independent estimate of performance on unseen organizations or document populations. Real-world accuracy and nominal conformal coverage are not established by these results.

Additional diagnostics include five insurance-claim variants that all return unsure; the eligible case's raw label is insufficient_information rather than supported. This identifies a target for improving evidence interpretation, in addition to calibration. Details and machine-readable summaries are in the evaluation report and evidence/.

Runtime validation

Independent reload reproduced logits exactly on 53 reference cases, including reversed and mixed request orders. API parity checks also passed. On those references, sequential pod-local HTTP latency was 97.08 ms median and 111.33 ms p95 on the reference H200 runtime. These observations describe that request set and runtime; sustained serving capacity was not measured.

The public package loader was additionally checked on eight short reference cases: logits matched exactly, including reversed request order. Its example execution and scoped validation are recorded in evidence/package_validation.json.

Feedback and development

We invite developers and researchers to evaluate Sapling on representative workflows and share results through the model's Hugging Face Community tab. Please include a redacted or synthetic reproduction, expected label, observed output, domain, and runtime version. Cases where unsure prevents a useful answer are particularly valuable.

The next development priorities are domain diversity, harder partial-evidence examples, document-level evaluation splits, and improved answer coverage while retaining explicit uncertainty.

License and provenance

Model weights are distributed under Apache 2.0, retaining the Qwen base model's attribution. Bundled project code retains its MIT license. Dataset text is not included in the weights package. See third-party notices.

The checkpoint is the completed v7 model, named Banyan Sapling for this release. The original release identity is 245970b0e5b0b22d64c1434ee21dd221b1f211f06bda5465f8fc108f70cd5c3f. The public distribution has its own file manifest, with an unchanged weights checksum. See provenance.