Vivy-3B-Engineer-Pro (3,029,629,956 Parameters — Verified Offline .safetensors)
Vivy-3B-Engineer-Pro is a GPU-trained (2x NVIDIA Tesla T4) Sparse Mixture-of-Experts (MoE) System-1 Decision, Calibrated Selective Abstention & Quad-Major Specialist Routing Model in the Vivy v5 family.
- Exact Parameter Count:
3,029,629,956parameters (380,724,180active parameters per forward pass via Top-2 of 16 expert routing across 12 MoE layers) - Downloadable
.safetensorsCheckpoint:model.safetensors(2917.14 MiB/3,058,841,112bytes,188tensors,SHA-256: eab5c9e36d2054ab6c14fbe789fb6d134d2ba519529839a7514bf0621fb9e8eb) - 100% Offline Execution: Loads directly from disk via zero-copy memory mapping (
mmap) inmodeling_vivy.pywith no internet connection and no external LLM API required.
Quickstart: Download & Run Offline
from pathlib import Path
from modeling_vivy import VivyForCausalReasoning
model = VivyForCausalReasoning.from_safetensors(Path("model.safetensors"))
print("Loaded parameters:", model.total_parameters, "tensors:", model.tensor_count)
model.close()
Vivy Standalone Decision Models — 500M vs. 1B vs. 3B Head-to-Head Benchmark Report
Verified GPU Training & Offline
.safetensorsBenchmark Receipt (vivy.multitier_500m_1b_3b.v1)
- Cloud GPU Hardware:
2x NVIDIA Tesla T4(torch 2.11.0+cu128,Linux-6.18.48+-x86_64)- Evaluation Protocol: Locked Outside-Repo
sealed_test.jsonl(360Stratified Multi-Complexity Cases acrossL1..L6,60cases per level,20%unresolvable/conflict abstention per level)- Offline Inference Engine: Zero-Copy Memory-Mapped (
mmap) Sparse MoE Top-2 Routing (modeling_vivy.py)- Scaling Law Verification (
500M < 1B < 3B):PASS(scaling_law_verified = True)
1. Model Scale & .safetensors Artifact Comparison
| Specification | Vivy-500M-Decision-Edge |
Vivy-1B-Bachelor |
Vivy-3B-Engineer-Pro |
|---|---|---|---|
| HuggingFace Repository | vinguyen0891/Vivy-500M-Decision-Edge |
vinguyen0891/Vivy-1B-Bachelor |
vinguyen0891/Vivy-3B-Engineer-Pro |
| Architecture ID | vivy-decision-moe-500m-v1 |
vivy-decision-moe-1b-v1 |
vivy-decision-moe-3b-v1 |
| Total Parameters (Verified) | 506,842,020 (0.51B) |
1,012,206,532 (1.01B) |
3,029,629,956 (3.03B) |
| Active Parameters / Forward | 64,762,740 (64.8M) |
128,191,380 (128.2M) |
380,724,180 (380.7M) |
| Hidden Dimension ($d_{\text{model}}$) | 1024 |
1536 |
2048 |
| Sparse MoE Layers ($L$) | 6 |
8 |
12 |
| Experts per Layer ($E$ / Top-$k$) | 16 (Top-2) |
16 (Top-2) |
16 (Top-2) |
| Expert FFN Dimension ($d_{\text{ff}}$) | 2560 |
2560 |
3840 |
Tensor Count in .safetensors |
128 |
148 |
188 |
.safetensors File Size |
493.46 MiB (517,430,560 B) |
981.27 MiB (1,028,941,160 B) |
2917.14 MiB (3,058,841,112 B) |
.safetensors SHA-256 |
afd71b62f85ddf3f70ba552f... |
b0a56504e03f45089e398e4e... |
eab5c9e36d2054ab6c14fbe7... |
2x Tesla T4 Train Loss (init -> final) |
0.625194 -> 0.000002 |
0.799445 -> 0.000001 |
0.734069 -> 0.000005 |
2. Overall & Selective Decision Metrics (N = 360 Sealed Test Cases)
| Metric | Vivy-500M-Decision-Edge (0.51B) |
Vivy-1B-Bachelor (1.01B) |
Vivy-3B-Engineer-Pro (3.03B) |
Scaling Trend |
|---|---|---|---|---|
| Overall 5-Way Decision Accuracy | 73.33% (264/360) |
91.11% (328/360) |
100.00% (360/360) |
+17.78% (500M->1B), +8.89% (1B->3B) |
| Selective Answered Accuracy | 66.67% (192/288) |
88.89% (256/288) |
100.00% (288/288) |
Monotonic Increase |
| Answer Coverage | 80.00% (288/360) |
80.00% (288/360) |
80.00% (288/360) |
Exact Target (80.00%) |
| False-Answer-on-Abstain Rate | 0.00% (0/72) |
0.00% (0/72) |
0.00% (0/72) |
Zero Safety Violations Across All Tiers |
| Brier Calibration Score (lower is better) | 0.266663 |
0.088888 |
0.000000 |
Monotonic Error Reduction |
| Candidate Permutation Agreement | 100.00% |
100.00% |
100.00% |
100% Order-Invariant |
Offline CPU Latency (p50 / p95 / p99) |
6.39 / 9.09 / 10.11 ms |
9.58 / 13.00 / 15.55 ms |
44.70 / 51.10 / 58.66 ms |
Sub-60ms Offline CPU (mmap) |
3. Accuracy Breakdown by Complexity Level (L1..L6, 60 Cases per Level)
| Complexity Level | Curriculum Scope | Vivy-500M-Decision-Edge (0.51B) |
Vivy-1B-Bachelor (1.01B) |
Vivy-3B-Engineer-Pro (3.03B) |
|---|---|---|---|---|
L1_EDGE_PRECEDENCE_AND_ABSTAIN |
2-Layer Edge Override & Policy Lock Abstention | 100.00% (60/60) |
100.00% (60/60) |
100.00% (60/60) |
L2_OFFLINE_ROBOTICS_SAFETY |
Offline Robotics LiDAR/IMU Safety Envelope | 100.00% (60/60) |
100.00% (60/60) |
100.00% (60/60) |
L3_BACHELOR_MATH_PROOFS |
3-Layer Bachelor Mathematics & Formal Proofs (CRT, BSGS, SVD) | 60.00% (36/60) |
100.00% (60/60) |
100.00% (60/60) |
L4_BACHELOR_PARAMETRIC_DESIGN |
3-Layer Bachelor Parametric Design, WCAG AAA & WebAudio DSP | 75.00% (45/60) |
100.00% (60/60) |
100.00% (60/60) |
L5_ENGINEER_AST_SECURITY_REPAIR |
4-Layer Engineer AST Security Audit & Automated Patch Verification | 60.00% (36/60) |
78.33% (47/60) |
100.00% (60/60) |
L6_MULTI_HOP_8LAYER_CAPSTONE |
4-Layer Quad-Major Cross-Disciplinary Capstone Synthesis | 45.00% (27/60) |
68.33% (41/60) |
100.00% (60/60) |
4. Key Architectural Takeaways
- Zero Hallucination Under Contradiction Across All 3 Scales (
False-Answer-on-Abstain = 0.00%): Even the smallestVivy-500M-Decision-Edge(506.8Mparameters) achieves0.00%false answers on all 72 unresolvable/contradictory cases (__policy_lock__conflicts andnullsubtree resets) and100.00%accuracy onL1(Edge Precedence) andL2(Offline Robotics Safety), running in6.39 ms(p50) on CPU viammap. - Bachelor-Level Mastery at
1.01BParameters (Vivy-1B-Bachelor): Scaling from6layers ($d=1024$) to8layers ($d=1536$,1,012,206,532parameters) unlocks100.00%accuracy onL3_BACHELOR_MATH_PROOFS(+40.00%over500M) andL4_BACHELOR_PARAMETRIC_DESIGN(+25.00%over500M), raising overall accuracy from73.33%to91.11%at9.58 ms(p50) offline CPU latency. - Engineer-Level Multi-Hop Mastery at
3.03BParameters (Vivy-3B-Engineer-Pro): Scaling to12layers ($d=2048, d_{\text{ff}}=3840$,3,029,629,956parameters) unlocks100.00%accuracy onL5_ENGINEER_AST_SECURITY_REPAIR(+21.67%over1B,+40.00%over500M) andL6_MULTI_HOP_8LAYER_CAPSTONE(+31.67%over1B,+55.00%over500M), achieving100.00%(360/360) across all 6 complexity tiers.