EfficientViT-Depth-5M

A 4.85M-parameter relative-depth model with an EfficientViT-M2 encoder and a Lite-DPT decoder, trained with Depth Anything V2 Small as a teacher.

This is an independent community model, not an official Depth Anything release, and it is not a fine-tune, derivative, or pruned version of Depth Anything V2. No Depth Anything weights, layers, or architecture are inherited. The architecture is deliberately different: a compact CNN encoder with a lightweight DPT-style decoder, aimed at small size and low CPU latency. Depth Anything V2 Small is used only to generate distillation targets.

Float vs INT8 size/quality comparison.

NYUv2 indoor scene; float vs INT8 at 336×448.

Qualitative examples on ADE20K/NYUv2 scenes; per-row color scale.

Highlights

• 4,854,203 parameters, 9.84 MB fp32 ONNX (static 336×448)
• 0.8546 NYU-654 δ1 at 336×448 (inverse-depth protocol, affine alignment)
• 0.7242 teacher-fidelity δ1 on 700 held-out images
• 12.1 ms / 82.5 FPS on CPU with ONNX Runtime, 8 threads, 336×448
• Dynamic-resolution ONNX graph included
• INT8 sibling: EfficientViT-Depth-5M-int8 — 3.48 MB, 0.8319 NYU δ1

Comparison

All quality numbers are at 336×448 on the 654-image NYUv2 Eigen test split. “Teacher fidelity” is agreement with the original fp32 Depth Anything V2 Small teacher on held-out distillation data.

Relative to Depth Anything V2 Small, this model uses 80.4% fewer parameters and is 90.1% smaller in fp32 ONNX, while retaining 92.8% of its NYU-654 δ1. Relative to the 8-layer Tiny, it uses 70.6% fewer parameters and is 85.1% smaller, retaining 95.6% of its NYU-654 δ1. Teacher fidelity is lower than both, so it is a size/latency-oriented model rather than a drop-in replacement for their detail.

Architecture

Deep supervision at stride 8 and 16 is used only during training. The released float checkpoint and ONNX graph are the clean inference model without auxiliary heads.

The network pads the input internally to a multiple of 32, runs the encoder/decoder, and bilinearly resizes the depth map back to the original input size.

Metrics

NYUv2 Eigen-654, 336×448

The protocol is inverse-depth space with per-image affine scale and shift alignment. The model predicts inverse depth; the ground-truth metric depth is inverted before alignment.

Resolution sweep (PyTorch, NYU-654)

336×448 is the primary deployment point and the clear sweet spot. 224×294 is substantially weaker; use 252×336 or higher when possible.

Teacher fidelity

Scale-and-shift aligned inverse-depth agreement with the fp32 Depth Anything V2 Small teacher:

The gap to the teacher is concentrated in fine structure and high-frequency detail, not overall layout.

Usage

ONNX Runtime

pip install -r requirements.txt
python example.py --image photo.jpg --output depth.png

The example resizes the RGB image to exactly 336×448, applies ImageNet normalization, runs the ONNX graph, and writes a normalized grayscale preview plus optional raw .npy inverse depth.

Minimal code:

import numpy as np
from PIL import Image
import onnxruntime as ort

mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)

image = Image.open("photo.jpg").convert("RGB").resize((448, 336), Image.Resampling.BICUBIC)
x = np.asarray(image, dtype=np.float32) / 255.0
x = (x - mean) / std
x = np.transpose(x, (2, 0, 1))[None].astype(np.float32)

session = ort.InferenceSession("efficientvit_depth_5m_fp32.onnx", providers=["CPUExecutionProvider"])
depth = session.run(None, {session.get_inputs()[0].name: x})[0][0]
# depth is inverse depth: higher = closer, not metric depth

PyTorch

import torch
from efficientvit_depth_5m import load_compact

model = load_compact("efficientvit_depth_5m_fp32.pt").eval()
x = torch.randn(1, 3, 336, 448)
with torch.no_grad():
    depth = model(pixel_values=x).predicted_depth

efficientvit_depth_5m.py is the full model definition. It requires torch and timm.

Dynamic resolution

efficientvit_depth_5m_fp32_dynamic.onnx accepts dynamic height and width. It was verified at 252×336, 336×448, and 448×602. The network pads internally to a multiple of 32. For best quality, use 336×448.

Output interpretation

The output is affine-invariant inverse depth, not metric depth. To compare with metric depth z:

1. Compute 1 / z.
2. Fit a per-image scale and shift between prediction and 1 / z.
3. Report error after alignment.

For visualization, normalize per image. Do not interpret the raw values as meters.

Training

• Teacher: original fp32 depth-anything/Depth-Anything-V2-Small-hf
• Data: 23,805 training images from COCO, ADE20K, and NYUv2 train; 2,200 held-out validation images; NYUv2 Eigen-654 test kept separate
• Schedule: 224×294 / 252×336 / 336×448 multi-resolution training, then a deep-supervision fine-tune
• Loss: scale-shift-invariant L1 on inverse depth + gradient L1; auxiliary stride-8/16 heads during the deep-supervision stage
• Optimizer: AdamW, cosine schedule, EMA weights
• Encoder initialization: ImageNet-pretrained EfficientViT-M2, then distillation

No NYUv2 test images were used for training or model selection.

Limitations

• Relative inverse depth only; no metric scale.
• 336×448 is the intended resolution. 224×294 is much weaker, and 448/518 degrade without resolution-specific fine-tuning.
• Fine structures, thin objects, clutter, transparent/reflective surfaces, and extreme depth ranges can be inaccurate.
• Teacher fidelity is lower than Depth Anything V2 Small and the 8-layer Tiny; this model trades detail for size and CPU speed.
• No temporal or multi-frame consistency.
• Not an official Depth Anything release and not affiliated with its authors.

Files

License

Apache-2.0. See LICENSE and NOTICE. The teacher, Depth Anything V2 Small, is also Apache-2.0.

Contact

Petros Fanioudakis — @Petrouil