Depth Anything V2 Tiny — 4-bit ONNX (336×448)

This repository contains the 8-layer, 4-bit ONNX deployment variants of Petrouil/Depth-Anything-V2-Tiny-hf, a pruned and distilled community version of depth-anything/Depth-Anything-V2-Small-hf.

The float parent is the 8-layer Tiny model: 16,504,129 parameters, 66.04 MB fp32. Two graphs are provided:

• a fixed 336×448 graph for deployment pipelines that can use a static shape;
• a dynamic-resolution graph for variable input sizes divisible by 14.

TFLite and NPU-specific conversions are future work.

This is a community variant, not an official Depth Anything release.

Recommended artifacts

Fixed 336×448: dav2s_8layer_int4_matmul_336x448.onnx — 18.14 MB

• 8-layer pruned student, 4-bit weight-only quantization of the 48 linear MatMul layers.
• NYUv2 Eigen 654: δ1 0.87960, AbsRel 0.11678, RMSE 0.07968.
• Teacher fidelity (200 images): δ1 0.88909, AbsRel 0.12113, RMSE 0.36944, Pearson 0.89861, gradient correlation 0.68092.
• 72.5% smaller than the 66.04 MB float parent, with only −1.40 NYU δ1 and −0.85 fidelity δ1 points of loss.

Dynamic resolution: dav2s_8layer_int4_matmul_dynamic.onnx — 18.15 MB

• Same weights and 4-bit MatMul scheme, with dynamic batch/height/width axes.
• Output size follows the input size; height and width must be divisible by 14.
• At 336×448 it matches the fixed graph exactly: NYU δ1 0.87960, fidelity δ1 0.88909.
• NYUv2 δ1 across resolutions: 224×294 0.84093, 252×336 0.86668, 336×448 0.87960, 448×602 0.87004, 518×686 0.87213.
• Use the fixed graph for a fixed camera/NPU pipeline; use the dynamic graph when the input size changes at runtime.

Measured comparison

All values are at 336×448 with the inverse-depth protocol described below.

The int8 QAT model is larger than the 4-bit model and much worse on NYU; it is not included. Its fidelity score is higher but its detail metrics are unstable (NaN gradient correlation on the ONNX fidelity run), so it is not a substitute for the 4-bit artifact.

Files

Input and output

• Input: pixel_values, float32, ImageNet-normalized RGB.
  • Fixed graph: (1, 3, 336, 448).
  • Dynamic graph: (batch, 3, height, width), with height and width divisible by 14.
• Preprocessing: resize directly to the target width and height with bicubic interpolation, scale to [0, 1], normalize with mean [0.485, 0.456, 0.406] and std [0.229, 0.224, 0.225].
• Output: depth, float32, shape (batch, height, width) for the dynamic graph and (1, 336, 448) for the fixed graph. Values are affine-invariant inverse depth (higher = closer).
• Postprocessing: the raw output is inverse depth. For visualization, normalize per image. To compare with metric depth, fit a per-image scale and shift in inverse-depth space; do not treat the raw values as meters.

Usage

import numpy as np
import onnxruntime as ort
from PIL import Image

MODEL = "dav2s_8layer_int4_matmul_336x448.onnx"
session = ort.InferenceSession(MODEL, providers=["CPUExecutionProvider"])
input_name = session.get_inputs()[0].name

image = Image.open("image.jpg").convert("RGB").resize(
    (448, 336), Image.Resampling.BICUBIC
)
x = np.asarray(image, dtype=np.float32) / 255.0
mean = np.array([0.485, 0.456, 0.406], dtype=np.float32)
std = np.array([0.229, 0.224, 0.225], dtype=np.float32)
x = (x - mean) / std
x = x.transpose(2, 0, 1)[None].astype(np.float32)

inverse_depth = session.run(None, {input_name: x})[0][0]  # (336, 448)
print(inverse_depth.shape, float(inverse_depth.min()), float(inverse_depth.max()))

Or run the included example:

python example.py --image image.jpg

For the dynamic graph, use MODEL = "dav2s_8layer_int4_matmul_dynamic.onnx" and resize to any target whose height and width are divisible by 14; the same preprocessing and output handling apply. The included example defaults to the fixed graph.

session = ort.InferenceSession(
    "dav2s_8layer_int4_matmul_dynamic.onnx",
    providers=["CPUExecutionProvider"],
)
# x is [1, 3, height, width], height and width divisible by 14
inverse_depth = session.run(None, {"pixel_values": x})[0][0]

Quantization details

• 4-bit: ONNX Runtime MatMulNBits, 4 bits, block size 32, symmetric, accuracy_level=4. Only MatMul nodes with a constant weight are quantized; attention score MatMuls are left float.
• Graph: 48 MatMulNBits nodes and 31 Conv nodes, opset 17 plus com.microsoft opset 1.
• The dynamic graph uses symbolic batch, height, and width axes. The fixed graph uses a static 336×448 input. Both use the same quantized weights.
• All convolutions and all activations remain float32. This is weight-only 4-bit quantization, not a fully 4-bit graph.
• The fusion stage is kept float because quantizing its residual additions was unstable in the int8 experiments.

Resolution sweep (dynamic graph)

NYUv2 Eigen 654, exact RGB/GT resize to each target, inverse-depth alignment:

336×448 is the best measured point. 224×294 is below the 0.865 NYU target; the other resolutions pass it.

Portability caveat

MatMulNBits is an ONNX Runtime com.microsoft contrib operator. The standard ONNX Runtime CPU build supports it, but it is not plain standard ONNX and is not directly portable to TFLite or an NPU. A target runtime must either support MatMulNBits or replace the 48 nodes with its own 4-bit MatMul primitive. Test this on the actual deployment runtime before committing to the format.

The float parent and the int8 experiments are documented in the main project report. No target-NPU conversion has been tested.

Metrics provenance

• NYUv2 Eigen test split: 654 images, official splits.mat.
• Teacher fidelity: 200 held-out images from COCO val, ADE20K, and NYUv2 train.
• Teacher: original fp32 depth-anything/Depth-Anything-V2-Small-hf.
• Runtime: ONNX Runtime 1.31.1, CPU execution provider.
• Protocol: affine-invariant inverse depth; per-image least-squares scale and shift alignment; exact RGB/GT resize to each tested resolution.
• The original fp32 DA-V2-S scores δ1 0.91106 under the same harness. Metrics are not directly comparable to papers using different GT preprocessing or alignment.

Pruning lineage

• Start: DA-V2-S (24.785M parameters, 12 layers).
• Progressive layer removal 12 → 11 → 10 → 9 → 8, with decoder warm-up and low-LR full fine-tuning at each step.
• Final 8-layer student: 8 layers, 6 heads, FFN width 1536, decoder reassemble [48, 96, 192, 192], taps [2, 5, 7, 8].
• Float parent: 16.504M parameters, 66.04 MB, NYU δ1 0.89362 at 336×448.
• 4-bit weight-only quantization: 18.14 MB, NYU δ1 0.87960.

FFN width pruning, attention-head pruning, and all-at-once pruning were tested and gave much worse quality at this scale.

Limitations

• Indoor (NYUv2) evaluation; outdoor and non-square deployment are not fully characterized.
• Relative inverse depth, not metric depth.
• Thin structures and edges are smoothed relative to the float model; the 4-bit weight-only format preserves them better than the int8 QDQ experiments.
• The fixed graph is static at 336×448. The dynamic graph accepts other sizes, but height and width must be divisible by 14, and dynamic shapes may be slower or unsupported on some NPU runtimes.
• No temporal consistency or video evaluation.
• Not safety-critical. Validate on the target camera, scene, and runtime before deployment.

License and attribution

Apache-2.0, inherited from Depth Anything V2. See LICENSE and NOTICE. Original model: depth-anything/Depth-Anything-V2-Small-hf.

If you use Depth Anything V2, cite:

@article{yang2024depthanythingv2,
  title={Depth Anything V2},
  author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
  journal={arXiv preprint arXiv:2406.09414},
  year={2024}
}