Transmission line inspection with object detection from a survey flight

The drone comes back from a span at 9 am with a card full of frames and the pilot's note that the light was behind the line for the last third of the flight. Somewhere in those frames is a conductor with one strand lifted out of the lay, a few dozen pixels against a white sky. The crew that will climb to fix it wants to know which tower and which phase before the truck leaves the depot.

That is the job. A card full of frames, a handful of hazards, and a decision per hazard about who goes where this week.

Four hazard classes, and each is a different shape on the frame

The classes on a line survey are set by what the crew can act on: a damaged strand, a broken or flashed insulator, a foreign object on the conductor or the arm, and vegetation inside the clearance. Each looks different. The strand is a thin bright line where a smooth cylinder should be. The insulator is a missing shed or a scorch mark on a string the model has seen intact a thousand times. The foreign object is a kite, a nest, a strip of plastic, and the class covers anything that should not be on the line rather than any one thing. Vegetation is a region with no clean edge that has moved closer since the last flight.

Boxes suit the first three. Vegetation wants an outline, because clearance is measured from the edge of the canopy to the conductor, and a box around a tree puts sky inside the measurement. The power line and grid inspection use case makes the same point about cracked insulators: a box says an insulator is there, and the dispatch decision needs the extent of the crack.

You type the four classes once, Lexi proposes the boxes on every frame from the flight, and a person who has walked a line checks them before anything trains. The person's job is the strand class above all, because the strand is where a labeler who has never seen one draws the box around the whole conductor.

Object detection finds a strand only if the frame is cut into tiles

A survey frame is large and the strand is tiny. Resize the whole frame to the size a model trains at and the strand becomes a pixel wide, then disappears. The fix is to cut the frame into overlapping tiles, run detection on each tile at full resolution, and stitch the boxes back into frame coordinates. The overlap matters, because a strand that sits on a tile edge is otherwise cut in two and found in neither half.

Tiling multiplies the frames the model sees and the time inference takes, and on a survey that is fine, since nobody is waiting on a frame from the air. It also means most tiles contain nothing. On the flight that landed at 9 am, most tiles were sky, and the training set has to say so, with tiles of clean conductor, clean sky and clean steel labeled as empty so the model learns what a hazard is against.

The line crews on one of the networks we work with call a lifted strand a bird cage, because the strands splay out around the core like the bars of one. The name is older than any camera.

Recall per hazard is the number to set, and it differs by class

One accuracy figure across the survey says almost nothing, because the survey is nearly all clean frames and a model that finds nothing scores well on it. The number that matters is recall for each hazard class on its own: of the damaged strands on the 9 am flight, how many came back flagged. A missed strand becomes a broken conductor in a storm. A missed kite is a nuisance.

So the thresholds are set per class, with the strand and insulator classes set to flag generously and accept the extra frames a person has to clear, and the foreign object class set tighter. The frames the model doubts come back to a person either way, and a person clearing a false strand is cheaper than a crew finding a real one the hard way.

My view, which the people who tune these models do not always share, is that the recall target for the strand class should be agreed with the crew lead rather than the engineering team. The crew lead is the one who has to explain the miss.

The flagged frame reaches the crew with the tower number on it

A flagged frame is worth little on its own. What the crew needs is the frame with the box drawn, the tower number from the flight log, the phase, and the flight date. With those, the planner can put it against the last cycle's frame of the same tower and see whether the hazard is new.

The alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live. A damaged strand on any flight is critical and goes to the line planner's Slack channel; vegetation inside clearance is routine and goes to the weekly trim list. Across the energy work we run, that path is what produces 21,000+ hazard detections per month, each one a frame with a box on it rather than a row in a report. The alert written as a sentence is the same mechanism a substation camera uses; here it is fed by a flight rather than a live feed.

Snow and backlight are where the first version is weakest

The flight that came back at 9 am had the sun behind the line for its last third, and those frames are the ones the model has the least evidence for. Backlit conductor loses its highlight, the strand that stood out as a bright thread against grey sky is now a dark thread against white, and the insulator string is a silhouette with no sheds to count. Snow does the reverse a season later: the line is dark against white ground, and the ground is where the model learned vegetation lives.

These are the frames that come back for review, and they should be kept rather than dropped. The drift catalog calls this rain, fog and dust: conditions the model rarely saw in training arrive for a week, detection thins out, then recovers. The week's frames are worth more in the training set than a month of clear ones, because they are rare there.

LexData takes the line model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then runs on the flights as they land, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The backlit frames from the 9 am flight are what the second version learned from.

A re-planned flight path is a new image of the same tower

The other change that arrives without warning is the flight itself. Airspace, wind or battery range moves the pilot's path on the Tuesday flight, and the same tower now arrives a few degrees off and at half the scale. The strand is unchanged; the pixels it occupies are not, and a model fitted to the old approach is weak on the new one in the same way it was weak on the backlit third.

The signal is the correction rate on one flight against the flights before it. When a person is overriding the model on frames from one path and on no others, the path is what changed. A short window of frames from the new approach, labeled and folded in, is the fix, and it comes from footage already on the card.