Object detection on aerial imagery and why the ground detector fails at four hundred feet

The drone's first pass over the tower on the transmission line, on a Tuesday in September, is flown at four hundred feet, the altitude the pilot chose so that one battery covers six towers before the walk back to the truck. In the frames from that pass, each insulator string is a few dozen pixels long against open sky and the conductors are lines a pixel wide. The corrosion the inspector is looking for is a change of colour on a fitting the size of a fingernail. The detector that found every insulator from the ground, on the pole-camera footage the utility already had, finds about a third of them from the air.

The pilot, asked afterwards, said the pass was flown the same way as every pass that season. That is the problem, and also the clue.

From four hundred feet an insulator is dozens of pixels

From the ground, an insulator string fills a good part of the frame, with visible sheds, visible hardware and a background of tower steel. From four hundred feet on a clear July morning it is a short grey line on blue, and the detector's input resize makes it shorter. Most of what the ground model learned about insulators, the shape of the sheds and the texture of the glaze, is simply not present in the aerial pixels, and the model has weak evidence for the thing it is asked to find.

The power line and grid inspection use case describes the same frame: the defect is often a few dozen pixels against open sky, at a distance the pilot chose for coverage rather than for the model. Tiling the full-resolution frame so the insulator keeps its pixels is the first fix, and flying lower is the second, and the second is usually cheaper than anything done to the model.

A ground-trained detector has never seen the top of anything

The larger problem is viewpoint. Every object in a ground-level training set is seen from the side, and every object in an aerial frame is seen from above. A transformer on a pole from the ground is a cylinder with bushings; from above it is a circle. A truck at the substation gate is a rectangle with no cab, no wheels and no windscreen. The ground model has learned sides, and there are no sides in the frame.

That is why the aerial model starts from the utility's own drone footage. The classes are typed once, insulator, fitting, conductor, vegetation, Lexi proposes the boxes on frames from the season's passes, and the line inspector checks each one before anything trains. His attention goes to the fittings, because a corroded fitting from the air is a slightly browner speck than an intact one, and the difference is the entire job.

The box has to turn with the pipeline

On the ground, most objects are upright and a horizontal box fits them. On the pipeline pass flown in August, nothing is upright. A pipeline crosses the frame at whatever angle the flight line made with it, a solar table is a parallelogram, and a tower's crossarm runs diagonally across the tile. A horizontal box around a pipeline at forty-five degrees is mostly desert with a pipeline through the middle, and two such boxes on neighbouring pipes overlap almost entirely while the pipes never touch.

The answer is a box that rotates with the object, a rectangle drawn along the pipeline's axis with an angle as part of the label. It costs more to label, because the person has to set the angle, and it pays back on every frame, because the box now contains the pipeline and very little else. The inspector checking rotated boxes on the pipeline pass spends most of his time on the crossings, where two lines meet and the model has proposed one box for both.

Dense scenes break box merging, and DETR does without it

A solar farm from four hundred feet is panel after panel packed edge to edge across the whole frame, and a conventional detector proposes many boxes for each and then merges the overlapping ones by keeping the most confident and suppressing its neighbours. In a dense scene, a neighbour is a different panel, and the suppression step deletes real detections because they overlap real detections. The count comes out low and nobody can say by how much.

Transformer detectors such as DETR remove that step. Each prediction is matched to one object during training, so the model learns to produce one box per panel without a merge afterwards, and dense scenes stop losing panels to their neighbours. The trade is that DETR and its successors are slower to train and were, in their first versions, weaker on small objects than the detectors they replaced, which on an aerial frame is the wrong weakness to have. I would rather fly a lower, slower pass over the solar farm than change architectures to fix a count, and only reach for a transformer detector when the pass is already as low as the pilot will fly it.

The labels come from the pilot's own passes

How much footage is the question every aerial project asks first, and the how much footage you actually need lesson gives the answer that holds here: the unit is distinct conditions rather than hours. Six towers flown at noon in July are one condition. The same six at 4 pm, with every tower throwing a long shadow across its own insulators, are another. The winter pass, with snow on the crossarms, is a third, and the model that has only seen July is at its weakest in its first January.

A tower with fourteen insulator strings in one frame is fourteen instances, which is why a dense frame is worth more than an empty one, and why the passes with the most hardware in view are the ones to label first.

A re-planned flight path is a camera that moved

The pass that found a third of the insulators was flown the way every pass that season had been flown. The passes the model was trained on, from the previous season, had been flown at three hundred feet along the line's east side, before an airspace change pushed the route west and higher. Nobody changed the camera. The angle and the scale at which every tower arrived in the frame changed anyway.

The drift catalog calls this a camera moved: the asset is unchanged, the framing is not, and the model was fitted to the old framing. On a drone the camera moves every time the flight plan does, for airspace, weather or battery, and the fix is the same as for a nudged bracket on a pole. Label a short window of frames from the new route and retrain on footage the season has already produced.

LexData takes the inspection model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then runs on each season's passes, in the cloud or on your servers. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The west-side passes at four hundred feet are the frames the next version was trained on, and the pilot flies the route the airspace allows.