Orchidia-1311 — Experimental Research Preview
Experimental research preview. Not production-ready. This package contains a LoRA adapter and a small decision head for
Qwen/Qwen3.5-4B. It does not contain the Qwen base weights.Custom inference code required. The adapter lives in
adapter/, not at the repository root, and must be combined with the bundled decision head. Useinference.py; this is not a drop-inPeftModel.from_pretrained(...)repository.
Orchidia is a structured decision model. Given a STATE (text) and a DYNAMIC DECISION SCHEMA (any number of questions, each with any number of choices), it returns a probability distribution for every question instead of free-text generation.
{
"intent": {"refund": 0.71, "shipping": 0.12, "complaint": 0.11, "other": 0.06},
"urgency": {"low": 0.18, "medium": 0.55, "high": 0.27},
"escalate": {"yes": 0.34, "no": 0.66}
}
This release packages the completed v0_16k_lora_r16 experiment from the
original Orchidia V0 project. No training, fine-tuning, optimizer step, new
benchmark, or holdout evaluation was performed for this packaging release.
Model at a glance
| Item | Value |
|---|---|
| Release | Orchidia-1311 — Experimental Research Preview |
| Base model | Qwen/Qwen3.5-4B |
| Base revision (pinned) | 851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a |
| Backbone | Qwen3.5-4B trunk, frozen |
| Adaptation | LoRA, rank 16, alpha 32, dropout 0.05 |
| LoRA target modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Decision head | LayerNorm(2560) → Linear(2560→1024) → GELU → Dropout(0.1) → Linear(1024→1) |
| Head parameters | 2,628,609 (float32) |
| Adapter parameters | 21,233,664 (float32) |
| Trainable parameters | 23,862,273 |
| Max sequence length | 512 tokens |
| Modality | Text-only in this preview |
| Training data | ZefanCai/Open-Jev, config release-v2-redistributable (not redistributed) |
| Training hardware | NVIDIA A100-SXM4-80GB |
| Status | Experimental research preview |
Architecture
STATE text + question + candidate option
↓
Qwen3.5-4B trunk (frozen; LoRA rank-16 adapters on attention + MLP projections)
↓
last non-padding token hidden state [2560]
↓
DynamicDecisionHead: LayerNorm → Linear(2560→1024) → GELU → Dropout → Linear(1024→1)
↓
one logit per (question, option)
↓
per-question softmax → {question_id: {choice: probability}}
There are no fixed question classifiers. One shared scorer handles unseen
questions and arbitrary K. Each (state, question, option) pair is rendered
with a fixed template:
State: {state_text}
Question: {question}
Option: {option}
Decision:
The decision head reads the last non-padding token and computes in float32 even when the trunk runs in fp16/bf16.
Training
The packaged checkpoint is the LoRA variant of the original V0 experiment:
| Item | Value |
|---|---|
| Experiment | v0_16k_lora_r16 |
| Training rows | 16,000 shuffled rows from Open-Jev release-v2-redistributable |
| Epochs / steps | 1 epoch, 1,000 optimizer steps |
| Batch | 2 per device × gradient accumulation 8 (effective 16) |
| Max length | 512 tokens |
| Precision | bfloat16 |
| Optimizer | AdamW |
| Head LR / LoRA LR | 1e-4 / 5e-5 |
| Seed | 0 |
| Warm start | 8K frozen head (checkpoints/v0_8k_shuf/head_final.pt) |
| Final validation loss | 0.2977 (reported by the original run) |
The backbone was frozen; only the 2.63M-parameter head and 21.23M-parameter LoRA
adapters were trained. Training used ZefanCai/Open-Jev
revision c67699e13d0ae25e35b77165a4b6b079bedc8aba. The dataset is not
redistributed here.
Evaluation methodology
The original project used group-separated Open-Jev splits (train / validation / test / OOD), a 512-token evaluation protocol, and pre-registered decision gates. The evaluation samples were deterministic and did not overlap the historical first-512 slices. OOD results are source-specific to Open-Jev.
The following numbers are as reported by the original internal evaluation. They were not re-run, re-validated, or independently reproduced for this packaging release, and they are not a claim of general benchmark performance.
| Pre-registered gate | Result | Threshold | Status |
|---|---|---|---|
| Natural-prevalence-weighted OOD accuracy | 87.52% | ≥ 73.0% | reported PASS |
painting-geometry-v1 | noul | K=2 accuracy |
88.59% | ≥ 60.0% | reported PASS |
| Validation paired CI lower bound | +11.33 pp | ≥ −1.0 pp | reported PASS |
For context, the original project reports the frozen 16K baseline at 69.6% natural-weighted OOD on the same rows (+17.96 pp for this LoRA checkpoint). The primary holdout set was never loaded in the original project and remains unspent; no holdout evaluation was run for this release.
Packaging validation (2026-11-13)
This release was validated without training or re-evaluation:
- The decision head was converted to Safetensors with 6/6 tensors, identical shapes/dtypes, and matching per-tensor SHA-256.
- The PEFT adapter was copied byte-for-byte; all 256 adapter tensors match the adapter embedded in the source checkpoint.
- On the authorized two-row probe fixture, the released package produces bit-identical logits, probabilities, and loss to the original champion under the same environment (Tesla T4, fp16, torch 2.10.0, transformers 5.22.2, peft 0.21.3).
- The stored A100/bf16 reference differs by at most 0.056 in logits and 2.0e-5 in probabilities (expected fp16-vs-bf16 hardware difference); argmax decisions agree.
- No new benchmark or holdout evaluation was run.
Installation
pip install -r requirements.txt
Core requirements: torch>=2.6, transformers>=5.10, peft>=0.21.3,
accelerate, safetensors, huggingface_hub. (peft 0.21.2 had an import
dependency on torchao>=0.16 in the original environment; 0.21.3 does not.)
Validated for this packaging pass with Python 3.12, torch 2.10.0+cu128, transformers 5.22.2, peft 0.21.3, safetensors 0.8.0 on a Tesla T4 (fp16). The original training/evaluation environment was Python 3.13, torch 2.11.0+cu130, transformers 5.20.2, peft 0.21.2 on an A100 80GB (bf16).
Qwen/Qwen3.5-4B (~8.8 GB) is downloaded from Hugging Face on first load at the
pinned revision. The base weights are not included in this repository.
Usage
from inference import load_model
model = load_model(dtype="auto") # bf16 on Ampere+, fp16 on T4, fp32 on CPU
state = {
"text": "Customer received the wrong product and wants a refund. "
"Order ORD-1042 was delivered yesterday; the item is unopened."
}
schema = {"questions": [
{"id": "intent", "type": "categorical",
"question": "What is the customer intent?",
"choices": ["refund", "shipping", "complaint", "other"]},
{"id": "urgency", "type": "categorical",
"question": "How urgent is this request?",
"choices": ["low", "medium", "high"]},
{"id": "escalate", "type": "boolean",
"question": "Should the case be escalated?",
"choices": ["yes", "no"]},
]}
result = model.predict_structured(state, schema)
print(result)
A runnable version is in usage_example.py:
python usage_example.py
To point at a local copy of the base model:
ORCHIDIA_BASE_MODEL=/path/to/Qwen3.5-4B python usage_example.py
Schema
Each question needs:
| Field | Meaning |
|---|---|
id |
Output key |
type |
categorical, boolean, or ordinal (aliases choice, noul, score) |
question |
Natural-language question |
choices |
Non-empty list of option strings |
Any number of questions and any K ≥ 2 choices per question are accepted.
Probabilities are normalized per question.
Intended use
Suitable for research on:
- structured decision modeling with dynamic question sets,
- probability estimation over categorical, boolean, and ordinal choices,
- workflow and policy triage experiments,
- adapter/head architecture comparisons on Open-Jev-style tasks.
Not intended for:
- production or safety-critical decisions,
- autonomous actions without human review,
- free-text generation or chat,
- image, video, or pixel reasoning (not validated; the loader is text-only),
- general-purpose benchmark or capability claims.
The model outputs probability estimates, not verified facts. Calibration and accuracy must be validated on your own data before any downstream use.
Files
orchidia1311/
├── README.md
├── LICENSE
├── NOTICE
├── config.json
├── orchidia_head.safetensors
├── adapter/
│ ├── adapter_model.safetensors
│ └── adapter_config.json
├── inference.py
├── requirements.txt
├── usage_example.py
├── CITATION.cff
└── checksums.sha256
Known limitations and experimental status
- Experimental only. This is not a production system. Do not use it for high-stakes automated decisions without human review and independent validation.
- Text-only preview. The loader explicitly rejects image/video inputs. The original V0 media hook also raised an explicit error. No claim is made that actual image-pixel reasoning has been validated.
- Frozen backbone plus rank-16 LoRA. Only 16,000 rows and 1,000 steps were used. Under-training and domain-adaptation limits remain.
- OOD is source-specific. Open-Jev "OOD" means reserved sources/wording/ layouts within that benchmark family, not universal domain shift.
- 512-token limit. Long states are truncated.
- Calibration. Probabilities are model estimates. No recalibration was performed for this release; ECE/Brier were reported only in the original analysis.
- Source bias. Open-Jev release-v2 is ordered by source; the original training used deterministic shuffled subsets, never first-N rows.
- No independent evaluation. All reported numbers come from the original project's internal, pre-registered protocol.
Diagnostics: Stage 2 and Stage 3 findings
The original project's Stage 2 diagnostics found high-confidence errors on selected painting color-binding cases. Stage 3 diagnostics observed non-queried-side tracking behavior.
These diagnostics are case-based and not population-level rankings. They do not establish a population error rate, a general capability ranking, or a production-readiness claim. No claim is made that actual image-pixel reasoning has been validated.
License and attribution
- This release (adapter weights, decision head, configuration, and inference
code): Apache License 2.0. See
LICENSEandNOTICE. - Base model
Qwen/Qwen3.5-4B, revision851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a: Apache License 2.0, Copyright 2026 Alibaba Cloud. The base weights are not redistributed in this repository; they are downloaded from their original repository and remain subject to their own license. - Training data
ZefanCai/Open-Jev: generated records are released under CC0-1.0; the dataset is not redistributed here. See the Open-Jev repository for its data license and third-party notices. - Modification notice: the LoRA adapters and decision head were trained by Seedyai on top of the unmodified Qwen3.5-4B base model.
If you redistribute this package, keep LICENSE and NOTICE with it.
Citation
See CITATION.cff.
@misc{orchidia16k2026,
title = {Orchidia 16K: Experimental Research Preview},
author = {Seedyai},
year = {2026},
note = {LoRA adapter and decision head for Qwen/Qwen3.5-4B},
url = {https://huggingface.co/Seedyai/Orchidia1311}
}
Acknowledgements
Base model: Qwen3.5-4B by Alibaba Cloud (Apache-2.0). Training data: Open-Jev (CC0-1.0 generated records). This release is an independent research artifact and is not affiliated with or endorsed by Alibaba Cloud or the Open-Jev authors.