Drone structural damage detection with cracks and spalling confirmed by the engineer

The quadcopter goes under the bridge deck at 8 am because the wind is lowest then. It flies the soffit in overlapping lanes a few metres off the concrete and comes back with forty minutes of footage of a surface nobody has looked at closely since the last principal inspection. The structures engineer used to do this from a bucket truck with a lane closed, a torch and a crack gauge, one span a day. Now the engineer has the footage and a different problem: forty minutes of grey concrete, most of it fine, some of it not, and no way to tell which without watching all of it.

The model watches all of it, and the engineer looks at what it flags.

Object detection finds it and a mask measures it

A crack on a footing gets a box in a survey where the question is which footings to visit. On a structure the engineer already knows about, the question is different: is this crack longer than last time, and is that spall bigger. A rate is the same quantity measured twice, and a box cannot supply the quantity, because the box grows when the aircraft flies closer and the spall does not. The predictive maintenance on grid assets use case makes the same argument for corrosion, and it holds for concrete: the label that gives an area is a mask.

So the labeler traces the crack and the spall rather than boxing them. The classes are the engineer's: crack, spalling with the reinforcement exposed, spalling without, rust staining where the bar is corroding behind the face, map cracking. The frames come from the drone at the distance the pilot flies for the inspection, a few metres off the surface, because a crack that is a few pixels wide at that distance is invisible at the distance a drone flies for coverage. You type the classes once, Lexi proposes the masks on every frame, and the engineer checks them before anything trains.

Object detection still has a job here, as the first pass that finds the candidates, and the mask is drawn on what it found. A crack the detector misses is a crack that never gets a mask, which is why the detector is tuned to miss nothing and the engineer's confirmation does the rest.

The engineer confirms every flag, and the confirmations are the labels

What the engineer gets is not the forty minutes. It is the frames the model flagged, each with the mask drawn on it and the class the model proposed, in the order the drone flew them so the engineer can place each one on the soffit. The engineer looks at each, confirms, corrects the class, redraws the mask, or dismisses it as a shadow or a form line, and moves on. A morning's footage becomes an hour's review.

Every one of those verdicts is a label. LexData takes the inspection model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then runs on the footage each flight brings back, in the cloud or on your servers, and frames it is unsure of come back to the engineer. The corrections retrain it, and the new version replaces the old one with no downtime. The doubted frames on a bridge are the form lines and the efflorescence streaks, and the engineer's dismissals of those are what teach the next version to leave them alone.

Across our energy work, 21,000+ hazard detections per month come out of models like this one, and the ones an engineer acts on are the ones an engineer confirmed.

An aside from the deck. The engineer still carries the crack gauge, and on the frames where the mask's width is close to the reporting threshold, the next visit to that spot is with the gauge. A millimetre on a mask is a millimetre only when the scale was right.

The mask becomes an area, and the area becomes a trend

A mask is pixels until a scale is known, and the scale on a drone comes from the distance to the surface and the lens, which the flight controller records for every frame. With it, a spall is so many square centimetres and a crack so many millimetres wide at its widest. Length comes from thinning the mask to its centreline, and width from the distance between that centreline and the mask's edge.

The trend is the point. The spall on the soffit at the third pier was a known size in March and is a known size in September, and the difference is a rate the engineer can put in the report. A rule written as a sentence, with a severity and a cooldown, approved before it goes live, turns the rate into a message. A spall on any pier grown past the margin the engineer chose since the last pass, high, to the structures team in Slack with the two frames attached. The engineer sees the March mask and the September mask side by side rather than a number.

A re-planned flight path is a camera move

The next pass flies the soffit at a different standoff because the wind was up, or in lanes the other way because the pilot changed, or with a different gimbal pitch because the light was low. The concrete has not changed. The scale every mask arrives at has, and so has the angle, and a spall photographed from a metre further away is a smaller mask. Compared naively, the spall shrank.

The drift catalog calls this a camera moved: the asset has not changed, the angle and scale it arrives at have, and the model was fitted to the old ones. On a drone there are two consequences. The model, trained at the old standoff, boxes fewer defects at the new one, and the correction rate steps up on the first review of the new pass. And the two passes cannot be compared until the frames have been registered to the structure, so that the March mask and the September mask are measured on the same square of concrete at the same scale.

The fix for the first is a short window of frames from the new pass, labeled by the engineer during the review that was happening anyway, and folded in. The fix for the second is procedural: the flight plan is written down and flown the same way each time, standoff and lanes and gimbal, and a pass that had to differ is marked as such so nobody compares it blind.

My own view, which the pilots I work with do not always enjoy hearing, is that a repeatable flight plan is worth more to a structures programme than a better model. A model can learn a new standoff in an afternoon of review. A trend measured across passes flown three different ways is not a trend, and no model fixes that. Where the model runs and how the footage gets to it is in the deployment doc.