DM-JEPA 1.1: Decision-Making Joint Embedding Predictive Architecture

Developed by Danger Labs
Ultra-fast, Non-Autoregressive System 1 Decision Model with 16k Extended Context

---

⚡ Overview

DM-JEPA 1.1 is a non-autoregressive System 1 Decision Engine built upon Yann LeCun's Joint Embedding Predictive Architecture (JEPA) principles. Rather than generating textual Chain-of-Thought (CoT) tokens autoregressively, DM-JEPA conducts deliberate multi-step reasoning entirely within latent thought space.

• Sub-35ms Latency: Delivers 34.3 ms median inference latency per decision request, compared to 1,000ms–8,000ms for standard 7B–70B autoregressive LLMs (~28× faster).
• Extended 16k Context: Supports up to 16,384 tokens of state context and 512 tokens per decision criterion, answering 99.84% (150,518 / 150,759) of the Decision Index full panel.
• Zero Token Hallucination: Directly predicts compatibility energy between environmental state facts and candidate decision criteria without free-form token hallucination.
• Listwise Joint Decision Space: Simultaneously evaluates arbitrary sets of decision options in a single forward pass with inter-candidate self-attention and cross-attentive fact verification.
• Official Decision Index Full-Panel Score: 23.16 Balanced Skill / 41.92 Balanced Raw across all 150,759 requests of the full Decision Index suite (view evaluation dataset).

---

🔬 Architecture

flowchart TD
    State["Environment State & Instructions (X)"] --> StateEnc["ModernBERT Context Encoder (RoPE Scaled to 16k)"]
    Options["Candidate Criteria Options (Y_1 ... Y_K)"] --> CritEnc["Target Projection Criteria Encoder"]
    
    StateEnc --> HState["Contextual Sequence Embeddings h_state"]
    CritEnc --> SOpt["Candidate Target Embeddings s_options"]
    
    HState & SOpt --> Predictor["M-Step Latent Predictive Verifier (GRU-Gated Thought Rollout)"]
    Predictor --> SHat["Anticipated Latent Target Vector s_hat"]
    
    SHat & SOpt --> Scorer["Latent Compatibility Scorer (Cosine Similarity * Temperature)"]
    Scorer --> Probs["Decision Probabilities P(Y_k | X)"]

1. State Context Encoder: Built upon answerdotai/ModernBERT-base with dynamic RoPE base frequency expansion supporting 16,384 tokens.
2. Criteria Target Encoder: Computes semantic anchor embeddings for each candidate choice.
3. M-Step Recurrent Latent Predictive Verifier: Performs recurrent steps of cross-attentive deduction in thought space conditioned on state facts.
4. Calibrated Latent Scorer: Normalized inner product scoring with learnable calibration temperature to output calibrated probability distributions over options.

---

📊 Decision Index Full-Panel Performance (150,759 Requests)

Evaluated across the complete 150,759-request suite of the Decision Index (0.2.1 protocol):

Head-to-Head Benchmark Highlights

---

🚀 Quickstart

Installation

pip install torch transformers huggingface_hub safetensors

Loading the Model

from modeling_dm_jepa import DMJEPA
import torch

device = "cuda" if torch.cuda.is_available() else "cpu"
model = DMJEPA.from_pretrained("DangerLabs/DM-JEPA", device=device)

---

🏷️ Citation & Contact

@misc{dangerlabs2026dmjepa,
  author = {Danger Labs},
  title = {DM-JEPA 1.1: Non-Autoregressive System 1 Decision Model with Stretched Context and Latent Predictive Verification},
  year = {2026},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/DangerLabs/DM-JEPA}}
}

Published by Danger Labs · Hugging Face Organization