MoGe-2 ViT-S Normal — Core ML

Apple Silicon Core ML conversion of Microsoft/MoGe (commit 925b8ed835a7a9cdb7578ba15c658a0afc969030). All variants share the identical op graph (traced at 1x3x512x768, fp16 compute, macOS 15 target); they differ only in weight storage (const dequantize ops). At runtime all computation is fp32.

Variants (branch per variant)

Precision	Branch	Size	Parity (worst median rel-depth err)	Mask IoU	Storage
fp16	`fp16`	72.4 MB	worst med-rel-depth n/a	IoU n/a	fp16 (baseline, all-fp32 compute)
int8	`int8`	36.4 MB	worst med-rel-depth 0.0318	IoU 0.9989	int8 blockwise linear_symmetric, bs32
int4	`int4`	18.6 MB	worst med-rel-depth 0.1664	IoU 0.9971	int4 blockwise linear (asymmetric), bs32
int4-sym	`int4-sym`	18.5 MB	worst med-rel-depth 0.2104	IoU 0.9974	int4 blockwise linear_symmetric, b64
pal8	`pal8`	36.4 MB	worst med-rel-depth 0.0336	IoU 0.999	int8 palettized (kmeans, 8-bit)

Parity = per-scene median relative depth error vs the author's reference outputs (03 scenes, same photos/artifacts as the upstream moge infer release test), computed with this repo's harness (--diff). fp16/int8/pal8 match the reference; int4 (asymmetric) and int4-sym carry a small-accuracy-gap documented per scene. The int4 palettized 4-bit variant does NOT meet parity (~0.90) and is excluded. FOV: int8/pal8 within ~0.3 deg of reference.

Per-scene parity vs reference

int8 — int8 blockwise linear_symmetric, bs32

Scene	median rel-depth err	mask IoU	FOV x	FOV y
House indoor	0.0318	1.0000	89.80 deg	67.19 deg
Office	0.0275	1.0000	71.52 deg	51.29 deg
Traffic	0.0252	0.9989	50.28 deg	34.74 deg

int4 — int4 blockwise linear (asymmetric), bs32

Scene	median rel-depth err	mask IoU	FOV x	FOV y
House indoor	0.0388	1.0000	92.94 deg	70.12 deg
Office	0.1664	1.0000	66.86 deg	47.51 deg
Traffic	0.1164	0.9971	55.19 deg	38.42 deg

int4-sym — int4 blockwise linear_symmetric, b64

Scene	median rel-depth err	mask IoU	FOV x	FOV y
House indoor	0.1483	1.0000	90.77 deg	68.09 deg
Office	0.0496	1.0000	71.04 deg	50.89 deg
Traffic	0.2104	0.9974	62.93 deg	44.39 deg

pal8 — int8 palettized (kmeans, 8-bit)

Scene	median rel-depth err	mask IoU	FOV x	FOV y
House indoor	0.0307	1.0000	89.82 deg	67.22 deg
Office	0.0336	1.0000	71.99 deg	51.68 deg
Traffic	0.0201	0.9990	50.27 deg	34.74 deg

Performance (M1 MacBook Air, 16 GB, macOS 26.5.2, CoreML ALL units)

Real-hardware latency: single predict on a 1x3x512x768 input, mean of 5 runs after warmup, measured with the tooling venv (no ANE — this model's ops are arch-gated to CPU/GPU on A13/M1):

Variant	Size	mean latency	FPS
fp16	72.4 MB	112 ms	8.9
int8	36.4 MB	114 ms	8.8
int4	18.6 MB	151 ms	6.6
int4-sym	18.5 MB	155 ms	6.5
pal8	36.4 MB	124 ms	8.1

Provenance


Source repo	`Microsoft/MoGe` @ `925b8ed835a7a9cdb7578ba15c658a0afc969030`
Checkpoint SHA-256	`79a16621928c2bf0ed04659218c55c01075e950507f40bb3332fb4c873d3e1dc`
Conversion	torch.export -> coremltools 9.0 (macOS15, fp16)
License	MIT (upstream). See `LICENSE.md`
Date	2026-08-06

Checkpoint SHA-256 verified against the reference RESULTS.md manifest before conversion. This repo is an artifact mirror of an MIT-licensed open model; conversion metadata in provenance.json.

Usage

import coremltools as ct
model = ct.models.MLModel("moge-2-vits-normal-coreml", compute_units=ct.ComputeUnit.ALL)
# input: image (1, 3, 512, 768) float32, RGB, 0..1
out = model.predict({"image": img})
# out["points"] (1,512,768,3) affine point map, out["normal"], out["mask"], out["metric_scale"]

Numerical post-processing (focal/shift recovery, depth map, intrinsics, FOV, UV) is pure numpy — see the accompanying harness/postprocess sources in the publisher's repo (zoo/moge/).

Notes & limitations

Apple Silicon only (Core ML mlprogram). No iOS deployment target shipped.
No ANE network on M1/A13 (layer_norm/gelu/sdpa/linear are arch-gated); runs CPU/GPU via CoreML.
Fixed input size 512x768 — resize before predict.
int4: recommend for storage-sensitive CPU paths; expect the accuracy gap listed above (blockwise asymmetric linear, bs32).

ebowwa/moge-2-vits-normal-coreml