Telos-Small (50.1M) — Time-Series JEPA Foundation Model

Telos-Small (50,100,747 parameters) is a standalone zero-shot Joint-Embedding Predictive Architecture (JEPA) for probabilistic time-series forecasting developed by MAGI Labs.

Instead of autoregressively generating raw time-series tokens step-by-step, Telos encodes observed history into patch representations, predicts future trajectory representations directly in latent space across up to 32 horizon patches (1,024 steps) in a single forward pass, and projects those predicted latents onto 21 calibrated quantiles (0.01 .. 0.99).

---

How Telos Works

1. Causal Welford + asinh Patch Scaling (patch_size = 32): Each 32-step patch is normalized using running causal Welford statistics and inverse-hyperbolic-sine (asinh) compression, handling heavy-tailed spikes, zero-inflated series, and arbitrary scale shifts without lookahead leakage.
2. Two-Stream Encoder with U-Net Skips & Value-Residual Attention (12 blocks, d_model = 512): Alternates three causal time-attention blocks with one cross-variate block ([time, time, time, time_variate] × 3) using SinCosRoPE, sigmoid-gated SDPA, U-Net skip connections (layer[k] -> layer[11-k]), and first-layer Value-Residual links across contexts up to 4,096–8,192 steps.
3. Horizon-Conditioned JEPA Latent Predictor (2 blocks): Given causal context latents and horizon step embeddings, predicts future target-encoder patch latents across 1,024 future steps in one shot, trained with a momentum EMA target encoder (V-JEPA), Epps-Pulley isotropic Gaussian regularization (SIGReg), multi-origin Tiled Decoupled-Horizon Forecasting (Tiled-DHF), and orthogonalized NorMuon (Newton-Schulz-5) + AdamW.
4. Monotone 21-Knot Quantile Readout + Advancing Welford Frames: Outputs strictly non-crossing quantiles (0.01, 0.05, 0.10, ..., 0.90, 0.95, 0.99) supporting univariate series, multivariate panels, past-only covariates, and known-future covariates.

---

Quickstart (transformers)

pip install torch numpy safetensors transformers

import numpy as np
from transformers import AutoModel

model = AutoModel.from_pretrained("magilabsco/Telos-Small", trust_remote_code=True)

# 1D series, (B, T) batch, or (B, V, T) multivariate panel -> returns (B, V, H, 21) quantiles
context = np.sin(np.linspace(0, 16 * np.pi, 512, dtype=np.float32)) + 100.0
quantiles = model.forecast(context, prediction_length=96)  # shape: (1, 1, 96, 21)
median_forecast = quantiles[0, 0, :, 10]                   # 0.50 quantile

---

Zero-Shot Benchmark Results (402 Tasks Across 4 Suites)

Evaluated zero-shot across 402 tasks on four public benchmarks (higher Skill % vs. Seasonal Naive is better):

---

Model Specifications

• Model Name: Telos-Small
• Parameters: 50,100,747 (191 MB in fp32 model.safetensors)
• Architecture: 12 Two-Stream Transformer Encoder blocks (d_model=512, 8 heads, mlp_hidden_dim=1152, U-Net skips, Value-Residual) + 2 JEPALatentPredictor blocks (hidden=1024) + QuantileHead (21 knots)
• Patch Size / Single-Shot Horizon: 32 steps per patch / up to 32 patches (1,024 steps) per single forward shot
• Context Length: 4,096 steps default (8,192 max)
• Training Compute: 43,591 DDP steps (12,570,406 forecast origins) on 2× NVIDIA Tesla P100-16GB GPUs

License

Released under the Apache-2.0 License. Pretraining corpus incorporates audited series from Salesforce/GiftEvalPretrain (Apache-2.0) and synthetic physical dynamical priors (Copyright 2026 MAGI Labs).