How photogrammetry works: structure from motion, step by step

A pile of photographs goes in; a model you can measure comes out. In between is a chain of steps that every photogrammetry program runs in roughly the same order, whatever it calls them. This page walks the chain with the pictures from a real survey, the demo quarry every Clarity account carries, so each step is shown on the data it actually produced.

The pipeline, with the real pictures

1. 1Photographs, overlapping
2. 2Features found in each photo
3. 3The same features, matched across photos
4. 4Cameras placed, sparse cloud solved
5. 5Dense point cloud
6. 6Mesh
7. 7Orthomosaic
8. 8Elevation model

347 photographs · each one overlaps its neighbours

24 shown · thousands found in a real photo

14 tie points between two neighbouring photos

347 camera positions solved · tie points in 3D

Dense cloud · a point for nearly every pixel

Triangles joined over the points

Orthomosaic · straight down, constant scale

Digital surface model · low in blue, high in rust

Step 1 of 8

Photographs, overlapping

The drone flies lines over the site, taking a photo every few seconds so that each point on the ground is in many pictures: typically 75% overlap along a line and 65% between lines.

Worth knowing

Every photo also records where the camera roughly was (GNSS) and, on better drones, to a few centimetres (RTK).

BackNext

Structure from motion: finding and matching features

The software cannot be told “this stone here is that stone there”; it has to find out. It starts by looking through every photograph for small patches that are distinctive enough to be recognised again from a different angle and distance: corners, specks, the edges of stones, the ends of tyre marks. Each one is described by the pattern of brightness around it in a way that does not change much when the picture is rotated, scaled or lit differently. A good photograph yields tens of thousands.

Then the descriptions are compared between photographs that could overlap. Where the same patch is found in two or more pictures it becomes a tie point: one spot in the world, with a pixel position in each picture it appears in. Thousands of wrong matches are proposed and thrown out, because a true tie point must sit on a consistent line of sight from every camera that saw it, and a false one will not.

Bundle adjustment: everything at once

With tens of thousands of tie points, each seen from several cameras, the software has a huge set of small facts: “point 4,812 appears at pixel (2,310, 1,044) in photo 17 and at (1,980, 1,102) in photo 18”, and so on. The unknowns are where every camera was and which way it pointed (six numbers per photograph), the lens’s own geometry (focal length, the exact centre, and how it bends straight lines towards the edges), and the position in space of every tie point.

Bundle adjustment solves for all of them together, adjusting every number a little at a time until the disagreement in every photograph is as small as it can be: that is, until the solved point, projected back through the solved camera, lands as close as possible to the pixel where it was actually seen. The remaining disagreement is the reprojection error, measured in pixels, and a few tenths of a pixel is a good result. The “bundle” is the bundle of light rays through each camera.

The result is a sparse cloud of tie points in three dimensions and a camera at every photograph, each with its position and orientation. The drone’s own GNSS record of where it took each picture goes in as a starting guess; if the drone has RTK, it goes in as a strong constraint. Ground control points enter here too, and pin the free-floating block to the ground.

From sparse to dense

Tens of thousands of points are enough to fix the cameras but not to describe a quarry. With the cameras known, the search for depth can be run at nearly every pixel: for a pixel in one photograph, the matching pixel in a neighbouring photograph must lie along one known line, so the software slides a small window along that line and picks the best agreement. Repeated across the overlap of every pair of photographs, and cross-checked between them, this produces a depth map per picture and, merged, a dense point cloud: millions of points, each coloured from the photographs it came from.

Resolution now follows the pixel size on the ground: a survey flown at a 2 cm GSD gives points a few centimetres apart. Trees, water and glass are where the depth search goes wrong, for the same reason the feature matcher struggled: there is nothing stable to compare.

Mesh, orthomosaic, elevation

The products are all made from the dense cloud, so they agree with one another.

Mesh

Points joined into triangles, the photographs projected onto them as texture. The 3D model you orbit.

Digital surface model

The surface sampled onto a grid of heights: everything the camera saw, including trucks, canopy and stockpiles.

Digital terrain model

The same grid with vegetation and objects classified out, so it is the ground. Cut and fill, contours and grading come from it.

Orthomosaic

Each photograph re-projected onto the surface and down into a map view, then blended: one image, straight down, at constant scale.

The orthomosaic deserves a word, because people measure on it most. A plain aerial photograph is taken from one point, so tall things lean away from the centre and the scale changes across the frame. The ortho correction uses the surface model to put every pixel where it would be if seen from directly above, which is why an orthomosaic can be measured for position, length and area like a map. It cannot be measured for height, because it has none: software that reads a height off an orthomosaic is reading the elevation model underneath.

Scale and coordinates

Nothing in a photograph says how big anything is. A block of photographs solved on its own is correct in shape and arbitrary in scale, position and orientation; it has to be tied to the world. There are three ways, used alone or together.

Whichever way, the honest measure of the result is a set of check points: surveyed positions not used in the solution, compared with the model afterwards. The accuracy page has the published figures.

What goes wrong, and where

Each step has its own failure, and knowing which step failed is most of the diagnosis.

Too few features

Blank surfaces, blur, over- or under-exposure. Holes in the model, or whole photographs left out.

Weak geometry

Too little overlap, all photographs nadir at one height, a long thin corridor. The block bends: the dome effect.

Bad lens model

A lens that changed focus or zoom between shots, or a rolling shutter at speed. Systematic error that no amount of control removes.

Nothing to tie to

No control, no RTK, no scale bar. A good-looking model that measures wrong, or sits in the wrong place.

The limitations page takes each of these in turn, with its cause and its fix, and the 101 is the practical version: how to plan and fly so that none of them happen. The same chain on a drone survey, from the flight to the measured map, is in the aerial photogrammetry guide; to have it run on your own photographs, Clarity’s drone image processing.