⚡ ULTRON // 1.189B Parameter Financial Conformer
A Foundation Architecture for Multi-Horizon Financial Microstructure Forecasting
Suryaansh Prithvijit Singh • Lead Researcher, Airborne AI / AirborneHRS
Email: Suryaansh@airbornehrs.in | GitHub: @Ultron09 | HF: @Ultron28
🏛️ Executive Summary
The ULTRON Financial Conformer (1,189,135,740 parameters across 283 parameter tensors) is a billion-parameter hybrid convolution-attention foundation model pre-trained on 14 continuous calendar years (2010–2023, 168 monthly Parquet shards, 15,030 optimization steps) of 1-minute US equity OHLCV bars (mito0o852/OHLCV-1m) across 15 liquid US equities and ETFs (AAPL, AMD, AMZN, BAC, GOOGL, INTC, IWM, JPM, META, MSFT, NFLX, NVDA, QQQ, SPY, TSLA).
All empirical claims are 100% deterministic, peer-review-audited, and reproducible via logs/defensible_tier1_audit.json and logs/pretraining_telemetry_verified.json (generated by scripts/defensible_scientific_audit.py):
- 1-Minute Base-Rate & Flat-Bar Decomposition: Across our stratified 15-ticker contiguous evaluation cohorts (
2024:8,044windows,2025:8,060windows,2026:8,100windows), exact zero-return flat bars ($y_{t,1\text{m}}=0$) account for7.86%(2024),6.85%(2025), and6.46%(2026) of observations, reducing strictly positive 1m returns $P(y_{t,1\text{m}} > 0)$ to46.51%,46.36%, and46.15%. Under BCE training, the 1m logit centers around $\bar{z}_{1\text{m}} = -0.1627$, so uncalibrated $z > 0$ thresholding fires0.0%of the time (53.49%–53.85%raw accuracy, identical to aFlatBar-Downheuristic). Removing this static threshold bias proves genuine discrimination: 1m All-Bar ROC-AUC of0.5375 / 0.5262 / 0.5274, 1m Strict Non-Zero ROC-AUC (NZAUC, $y \neq 0$) of0.5248 / 0.5138 / 0.5159, Prior-Calibrated Accuracy of52.04% / 51.23% / 51.25%, and active two-sided 5m Strict Non-Zero Accuracy of51.74%(2024) and51.18%(2026). - Exact Global Cosine Geometry, Adam Arithmetic & Pre-2024 Crisis Forgetting: Using the exact
float64Hilbert-space Law of Cosines $\cos_{\text{exact}} = \frac{1 + r^2 - \delta_F^2}{2r}$, every module strictly satisfies the Cauchy-Schwarz lower bound $\cos_{\text{exact}} \ge \sqrt{1 - \delta_F^2}$: global Frobenius drift is0.5439%($\cos_{\text{exact}} = 0.99998521 \ge 0.99998521$) forEp3 → WF-2024,0.5725%($\cos_{\text{exact}} = 0.99998361$) forWF-2024 → WF-2025, and0.7860%($\cos_{\text{exact}} = 0.99996911$) cumulatively forEp3 → WF-2025. Decomposing drift into RMS weight norm $|\mathbf{W}|{\text{RMS}}$ and absolute RMS update $|\Delta\mathbf{W}|{\text{RMS}}$ proves that the Inception Tokenizer has the largest weight norm (0.02574vs.0.01432for the backbone), yet still exhibits the highest relative drift (1.1773%) because its absolute RMS update (3.030e-4) is 4.00x larger than the backbone (7.575e-5). Direct evaluation on Pre-2024 Crisis Shards (2020-03COVID crash,2022-06Fed rate shock,2023-12;7,896windows) confirms zero overall forgetting (0.5130 → 0.5135AUC;2020-03COVID 1m $\Delta\text{AUC} = +0.0052\ [+0.0015, +0.0084]$), while 2026 paired block-bootstrap 95% CIs confirm a significant forward 1m gain ($\Delta\text{AUC} = +0.0022\ [+0.0001, +0.0040]$). - Controlled Mechanistic Ablation & Denominator Proof: Using in-distribution mean-ablation and random 341ch/682ch controls, we prove that the apparent
15mpeak in uncentered $|\Delta z_h|_2 / |z_h|2$ is a denominator effect (||z_15m||_RMS = 0.0405as $\bar{z}{15\text{m}} \approx 0$, vs.0.2041at1mand0.1285at1d), whereas denominator-free $\text{RMS}(\Delta z_h)$ decays monotonically (0.0403at1m$\to$0.0136at15m$\to$0.0058at1dforK=7) and variance-normalized centered L2 sensitivity significantly exceeds random channel controls (49.90%forMeso K=15,31vs.30.89%forRandom 682-Ch Control). - Non-Overlapping Execution Economics & Frozen 2024 Policy: Stepping by $S_h = \lceil \tau_h / 32 \rceil$ windows eliminates multi-period overlap, and tracking state-dependent turnover $|w_{i,k} - w_{i,k-1}|$ shows true state-hold breakeven is
1.15x–2.41xhigher than isolatedGross / 2. The single frozen policy selected on 2024 (60m, τ=0.0σ) achieves+4.40Gross Sharpe (+2.04Net Sharpe at1.5 bps,BE = 2.82 bps,MaxDD = 1.69%) in 2024 and+1.01Gross Sharpe untouched in 2025, while a fixed demeaned $\tau=1.0\sigma$ rule at3machieves+5.38(2024),+4.91(2025), and+3.56(2026) Gross Sharpe across all 3 years (BE = 0.35–0.44 bps), establishing that standalone OHLCV signals require passive maker/execution-timing deployment rather than taker spread crossing.
📑 Tier-1 Research Papers & Unified Monograph
This research is available both as a Unified Flagship Monograph (consolidating the entire checkpoint and log lineage into a single self-contained 11-page paper) and as three synchronized individual manuscripts authored by Suryaansh Prithvijit Singh (Lead Researcher, Airborne AI / AirborneHRS • Suryaansh@airbornehrs.in):
| # | Manuscript Title | Direct PDF Link | Core Empirical & Mathematical Contributions |
|---|---|---|---|
| ★ | The 1.189-Billion Parameter Financial Conformer: A Unified Empirical Audit of Pre-Training Scaling, Base-Rate Calibration, Continual Walk-Forward Geometry, Controlled Receptive-Field Ablation, and Non-Overlapping Execution Economics (Flagship Unified Monograph) | 📄 Unified PDF (11p) | Complete self-contained 11-page monograph consolidating Architecture, Pre-Training Telemetry, 1m Base-Rate/Flat-Bar Decomposition, Exact float64 Law-of-Cosines Drift, Pre-2024 Crisis Forgetting, Controlled Mean-Ablation, and Non-Overlapping Frozen-Policy Backtesting. |
| 1 | Scaling 1.189-Billion Parameter Financial Conformers: Pre-Training Dynamics, Base-Rate Decomposition, and Non-Zero Multi-Horizon Alpha in Market Microstructure | 📄 Paper 1 PDF (6p) | Exact 283-tensor architecture audit (1,189,135,740 params), 855-step pre-training telemetry (-15.2% loss variance contraction), 1m flat-bar & base-rate log-odds proof, Strict Non-Zero ROC-AUC (NZAUC), and $ |
| 2 | Continual Walk-Forward Adaptation in 1.189B Financial Conformers: Exact Cosine Geometry, Adam Plasticity Arithmetic, and Pre-2024 Crisis Forgetting Audits | 📄 Paper 2 PDF (4p) | Exact float64 Law-of-Cosines global cosine similarity ($\cos_{\text{exact}} \ge \sqrt{1-\delta_F^2}$), per-parameter $|\mathbf{W}|{\text{RMS}}$ vs. $|\Delta\mathbf{W}|{\text{RMS}}$ Adam arithmetic proof, Pre-2024 crisis stress tests (2020-03 COVID, 2022-06 Rates), and 2026 paired block-bootstrap 95% CIs. |
| 3 | Mechanistic Interpretability and Non-Overlapping Execution Economics in 1.189B Financial Conformers: Controlled Kernel Ablations, Denominator Decomposition, and Deflated Sharpe Audits | 📄 Paper 3 PDF (5p) | In-distribution Mean-Ablation vs. Zero-Ablation with Random 341ch/682ch Controls, proof of the 15m denominator effect vs. monotonic $\text{RMS}(\Delta z_h)$, and non-overlapping contiguous backtesting with state turnover, DSR ($M=35$), and untouched Frozen 2024 Policy transfer. |
📐 Exact Model Architecture Audit (1,189,135,740 Parameters across 283 Tensors)
| Subsystem / Module | Configuration & Shapes | Exact Parameters | % of Total |
|---|---|---|---|
| Inception Conv1d Branches ($K=3,7,15,31,63,127$) | $6 \times [\text{Conv1d}(8 \to 341, K_i, s=4) + \text{bias}]$ | 673,134 |
0.057% |
| Inception Linear Projection & RMSNorm | $\mathbf{W}_{\text{proj}} (2046 \times 2048) + \text{RMSNorm}(2048)$ | 4,192,256 |
0.353% |
20× GQA Attention (q, k, v, out) |
$H_q=16, H_{kv}=8, d_{\text{head}}=128$, RoPE | 251,658,240 |
21.163% |
| 20× Depthwise Separable Conv Module | Pointwise GLU ($2048 \to 4096$), $\text{DWConv}_{K=7}$, Norm | 251,985,920 |
21.191% |
| 20× SwiGLU Feed-Forward Network | $d_{\text{model}}=2048, d_{\text{ff}}=5504$ (w_gate, w_up, w_down) |
676,331,520 |
56.876% |
| 20× Conformer Block RMSNorms | $3 \times \text{RMSNorm}(2048)$ per block (norm_attn, norm_conv, norm_ffn) |
122,880 |
0.010% |
| Final Backbone RMSNorm & Dual MLP Heads | final_norm ($2,048$) + return_head & dir_head ($2\times 2,084,871$) |
4,171,790 |
0.351% |
| Total Financial Conformer (1.189B) | 20 Conformer Blocks, $L=2048 \to T=512$ Tokens (283 Tensors) |
1,189,135,740 |
100.000% |
🏆 Peer-Review-Hardened Empirical Results (logs/defensible_tier1_audit.json)
1. Stratified 15-Ticker OOS Evaluation of Frozen Epoch 3 (2024–2026)
| Horizon ($S_h$ Step) | Year | $N_{\text{eff}}$ | Flat % | Pos % | $P(z>0)$ | Raw Acc | NZ Acc ($y \neq 0$) | Cal Acc | All AUC | NZ AUC | Non-Ovlp IC |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 1m ($S=1$) | 2024 | 8,044 | 7.86% |
46.51% |
0.0% |
53.49% |
49.53% |
52.04% |
0.5375 |
0.5248 |
+0.0137 |
| 1m ($S=1$) | 2025 | 8,060 | 6.85% |
46.36% |
0.0% |
53.64% |
50.23% |
51.23% |
0.5262 |
0.5138 |
+0.0210 |
| 1m ($S=1$) | 2026 | 8,100 | 6.46% |
46.15% |
0.0% |
53.85% |
50.67% |
51.25% |
0.5274 |
0.5159 |
+0.0247 |
| 3m ($S=1$) | 2024 | 8,044 | 4.00% |
48.51% |
27.0% |
52.05% |
50.80% |
51.48% |
0.5256 |
0.5185 |
-0.0191 |
| 3m ($S=1$) | 2026 | 8,100 | 2.94% |
48.27% |
27.0% |
52.17% |
51.28% |
51.09% |
0.5204 |
0.5147 |
-0.0137 |
| 5m ($S=1$) | 2024 | 8,044 | 3.38% |
48.69% |
38.8% |
52.36% |
51.74% |
51.32% |
0.5256 |
0.5219 |
+0.0150 |
| 5m ($S=1$) | 2026 | 8,100 | 2.52% |
48.40% |
39.4% |
51.64% |
51.18% |
51.17% |
0.5166 |
0.5134 |
-0.0101 |
| 15m ($S=1$) | 2024 | 8,044 | 1.60% |
50.85% |
62.3% |
50.90% |
50.97% |
50.86% |
0.5170 |
0.5157 |
+0.0259 |
| 60m ($S=2$) | 2025 | 4,030 | 0.47% |
50.30% |
100.0% |
50.30% |
50.54% |
50.56% |
0.5117 |
0.5128 |
+0.0411 |
| 1d ($S=13$) | 2024 | 627 | 0.20% |
53.01% |
100.0% |
53.01% |
53.11% |
51.28% |
0.5062 |
0.5063 |
+0.0051 |
| 1d ($S=13$) | 2025 | 627 | 0.15% |
52.72% |
100.0% |
52.72% |
52.80% |
50.22% |
0.4994 |
0.4991 |
+0.0451 |
2. Exact float64 Law-of-Cosines Geometry & Adam Plasticity Hierarchy
| Subsystem / Module | Params | Ep3 → WF24 $\delta_F$ |
WF24 → WF25 $\delta_F$ |
Ep3 → WF25 $\delta_F$ |
Ep3 → WF24 Exact $\cos$ ($\ge \sqrt{1-\delta_F^2}$) |
$|\mathbf{W}|_{\text{RMS}}$ | $|\Delta\mathbf{W}{24}|{\text{RMS}}$ |
|---|---|---|---|---|---|---|---|
| Inception Tokenizer | 4.87M |
1.1773% |
1.1807% |
1.6172% |
0.99993072 (0.99993070) |
0.02574 |
3.030e-04 |
GQA Attention (L0–19) |
251.66M |
1.1705% |
1.2528% |
1.7160% |
0.99993150 (0.99993149) |
0.01241 |
1.452e-04 |
| Directional Head | 2.11M |
1.0087% |
0.9400% |
1.4724% |
0.99994962 (0.99994912) |
0.01582 |
1.596e-04 |
| All 20 Blocks Backbone | 1180.06M |
0.5291% |
0.5591% |
0.7671% |
0.99998601 (0.99998600) |
0.01432 |
7.575e-05 |
SwiGLU FFN (L0–19) |
676.37M |
0.3616% |
0.3472% |
0.4778% |
0.99999347 (0.99999346) |
0.01209 |
4.374e-05 |
| Return Reg. Head | 2.11M |
0.2419% |
0.2698% |
0.2998% |
0.99999707 (0.99999707) |
0.01106 |
2.676e-05 |
Depthwise Conv (L0–19) |
251.99M |
0.1540% |
0.1998% |
0.2802% |
0.99999881 (0.99999881) |
0.01692 |
2.606e-05 |
Global Model (283 Tensors) |
1,189.14M |
0.5439% |
0.5725% |
0.7860% |
0.99998521 (0.99998521) |
0.01438 |
7.821e-05 |
3. Controlled Mechanistic Ablation & Non-Overlapping Execution Economics
📚 Citation & Author Information
If you utilize this foundation model, dataset pipelines, or academic papers in your quantitative research or trading algorithms, please cite:
@article{singh2026financialconformer,
author = {Suryaansh Prithvijit Singh},
title = {The 1.189-Billion Parameter Financial Conformer: A Unified Empirical Audit of Pre-Training Scaling, Base-Rate Calibration, Continual Walk-Forward Geometry, Controlled Receptive-Field Ablation, and Non-Overlapping Execution Economics},
year = {2026},
institution = {Airborne AI / AirborneHRS},
doi = {10.5281/zenodo.23466965},
url = {https://doi.org/10.5281/zenodo.23466965},
note = {Lead Researcher, Suryaansh@airbornehrs.in}
}
Architected by Suryaansh Prithvijit Singh // Lead Researcher, Airborne AI / AirborneHRS
Connect: GitHub (@Ultron09) • Hugging Face (@Ultron28) • Suryaansh@airbornehrs.in