Chapter 9 — SfM Backends

WHERE

Expert View → Inspector → „Cameras & Capture" section → Camera Alignment picker, entry „Apple Photogrammetry". Default in every build.

TECHNICAL

Wraps Apple's built-in Photogrammetry framework, originally developed for Object Capture. Apple internally extracts features using a proprietary pipeline (the steps are not publicly documented), verifies them via multi-view matching and solves bundle adjustment on the Apple Silicon Neural Engine + GPU. The backend is fully App Store-compliant (no external binary, Sandbox=true, on-device), but only delivers camera poses plus a rough point cloud — no diagnostic metrics such as track length or reprojection error. Scales, per Apple's recommendation, up to a few hundred images. Above ~500 frames in linear drone flights or large outdoor scenes, reproducible crashes or silent dropping of individual cameras have been observed.

WHERE

Menu „File → Import COLMAP / Metashape Workspace…" (Cmd+⇧+I) OR drag-and-drop a folder containing sparse/0/cameras.txt.

TECHNICAL

Reads the standardised COLMAP text export — three text files cameras.txt, images.txt, points3D.txt in the sparse/0/ subfolder — and converts it into the internal SfM result model. Same format definition as the COLMAP binary export, just ASCII instead of binary. Is output in exactly this layout by Agisoft Metashape, RealityCapture, PolyCam and the ETH3D benchmark. The parser shares camera model detection with the binary parser and knows all eleven standard COLMAP camera models — from SIMPLE_PINHOLE and PINHOLE through SIMPLE_RADIAL (COLMAP's own default) and OPENCV to the fisheye variants. Robust against comment lines and blank lines. Scales in tests to ~1,400 cameras (ETH3D Tunnel) without issues.

WHERE

Expert View → Inspector → „Cameras & Capture" section → Camera Alignment picker, entry „Native" (without the former „(experimental)" suffix). Incremental is the default mode of this backend; there is no mapper selection for Native. The „Mapper" line below belongs exclusively to COLMAP: in developer builds it is visible but greyed out as long as Native is selected, while in the App Store version it is missing entirely. The truly native lines — „FOV Override", „High-Quality" and „Native SfM Recipe" — no longer disappear under other backends but remain greyed out and state, in a line below, what you need to change to unlock them again. You cannot switch between the incremental and the global method yourself: incremental is fixed as the default, and the app only switches to the global method on its own (see Q5).

TECHNICAL

Custom GPU-accelerated implementation of the entire SfM pipeline: FAST+BRIEF features OR SuperPoint+LightGlue via CoreML (enabled via the „High-Quality" toggle; already active for unordered photo sets anyway), followed by Hamming KNN matching, RANSAC fundamental matrix, track building, initial-pair selection, two-view bootstrap (F→E plus DLT), a greedy incremental mapper with PnP registration and multi-view triangulation, and a final bundle adjustment via Schur-reduced Levenberg-Marquardt with Huber loss and analytical Jacobians via Cholesky solve. Fully App Store-compliant: no external binary, Sandbox=true. Featuring the built-in collapse detector: it classifies a result as degenerate if fewer than 60% of the input frames were registered, if the points-per-camera rate falls below 13, or if the point cloud is nearly flat. This does not immediately trigger a switch of method — the app first repeats the incremental run (it keeps the first unremarkable result, otherwise the best of the attempts) and only then falls back to the global mapper (Q5) if even the best attempt remains degenerate. Empirically clean on orbit/turntable scenes; on more general motions (drone flight, interiors with complex geometry) the success rate is lower — but the detector catches these cases. Scales reliably to ~200 cameras, higher with significantly longer runtime.

WHERE

Invoked automatically when the incremental mapper (Q4) triggers the collapse detector (fewer than 60% of the input frames registered, points-per-camera rate below 13, or a nearly flat point cloud) — and only after the incremental run has been repeated and even the best attempt remained degenerate. It cannot be requested manually: there is no picker for it in the Inspector and no other toggle either — the app decides on its own when to switch.

TECHNICAL

Global variant of the native pipeline. First feature extraction + matching as in Q4, then relative-pose estimation for all verified pairs, followed by rotation averaging (synchronises all camera rotations in the world coordinate system) and translation averaging (LSQR-based on a matrix-free sparse formulation, to avoid integer overflow with large camera counts). Scales to ~5,000 cameras in principle, though in practice quality drops noticeably above a few hundred cameras. Handled as a „fallback tier": it kicks in when the incremental mapper degenerates even after retrying, and is not itself submitted to the collapse detector again — if its result remains sparse, the pipeline's general quality warning appears instead.

WHERE

File menu → „Import COLMAP / Metashape Workspace…" (Cmd+⇧+I). Drag-and-drop a folder containing sparse/0/cameras.{bin,txt} and images/.

TECHNICAL

Automatically detects whether a folder selected via drag-and-drop or the open panel matches one of the three COLMAP workspace layouts (sparse/0/, sparse/, or root) and whether the reconstruction is binary (cameras.bin) or text (cameras.txt). The binary path uses the COLMAP binary parser, the text path the ETH3D loader — both produce the same SfM result model, and the rest of the pipeline (importing images, starting MCMC training) is agnostic to the source. The images are opened security-scoped via the app's sandbox bookmark system, so the import works even in the App Store version. Specifically designed for the case „Metashape export without recomputing the reconstruction". The detection mentioned in the File menu entry logs a warning to the app log if the chosen folder is not a recognisable workspace.

WHERE

Expert View → Inspector → „Cameras & Capture" section → line „Native SfM Recipe". The line is only operable when Native is selected as Camera Alignment and an unordered photo set is set as the capture type — otherwise it appears greyed out and states below what needs to be changed (the availability rules are in Chapter 2).

TECHNICAL

Three tiers of the same native backend. They don't change the algorithm, but how thoroughly it works:

Standard — the base recipe without additional stages.

Professional — additionally sets two things: detected AprilTag markers are incorporated as rigid bodies in mapping and bundle adjustment, and the finished poses are densely re-triangulated (denser starting point cloud). If the capture contains no markers, the detector finds nothing and only the dense re-triangulation remains effective.

E3 High-Accuracy — builds on Professional and additionally enables the HQ frontend (3×3 SuperPoint tiling, LightGlue with 4096 keypoints, covisibility pairs) as well as joint focal-length refinement across all cameras. This yields the sharpest camera poses and the longest matching runtime.