Depth Anything V2 Metric Small, onnx for OpenCV (cv::dnn)
The metric depth models of Depth Anything V2 (Small, ViT-S),
converted to onnx that cv::dnn of OpenCV 4.10 or later reads, for dnn_depth_estimation of
opencv_apps:
| File | Model | Scenes | Max depth |
|---|---|---|---|
depth_anything_v2_metric_hypersim_vits_518.onnx |
Depth-Anything-V2-Metric-Hypersim-Small | indoors | 20 m |
depth_anything_v2_metric_vkitti_vits_518.onnx |
Depth-Anything-V2-Metric-VKITTI-Small | outdoors | 80 m |
- Input:
image, 1 x 3 x 518 x 518, RGB normalized by the mean (0.485, 0.456, 0.406) and the standard deviation (0.229, 0.224, 0.225) of ImageNet. - Output:
depth, 1 x 518 x 518, the depth in meters.
Conversion
torch.onnx.export (opset 12, the TorchScript exporter, a fixed input of 518 x 518), simplified by onnx-simplifier
(without fusing MatMul and Add into Gemm), and the weights of Gemm transposed to transB=1, so that cv::dnn reads
it: the onnx of the dynamic shapes or of the newer exporter is read only by OpenCV 5. The script is
tools/export_depth_anything_metric.py
of opencv_apps.
Read by OpenCV 4.10, 4.12 and 5.1 (not 4.6). The Hypersim model on an image of the TUM RGB-D dataset (freiburg3_walking_xyz): the mean absolute relative error to the depth of the sensor is 0.14 with CUDA and 0.29 on the CPU of OpenCV 4.12.
License and citation
Apache 2.0, as the models of Depth Anything V2 Small. Please cite:
@article{depth_anything_v2,
title={Depth Anything V2},
author={Yang, Lihe and Kang, Bingyi and Huang, Zilong and Zhao, Zhen and Xu, Xiaogang and Feng, Jiashi and Zhao, Hengshuang},
journal={arXiv:2406.09414},
year={2024}
}