Slicing the frame so the small things get found

The drone flies the distribution line along the desert track at first light in March, and each frame it brings back is several thousand pixels wide. The pole in the frame is a few hundred pixels tall. The insulator on the crossarm is forty pixels. The crack in the insulator, the thing the whole flight was for, is a line a handful of pixels long, and the detector reported the pole, the crossarm, and nothing else.

The detector did not miss the crack because it is a poor detector. It missed it because it never saw it. Before a detector looks at a frame, it resizes the frame to the size it was trained at, which for most models is a few hundred pixels on a side, and at that size the crack is smaller than a pixel.

The detector shrinks the frame before it looks at it

Every detector has an input size, and a frame larger than it is scaled down on the way in. A frame several thousand pixels wide squeezed to a few hundred keeps its poles and loses anything under a certain size; the insulator becomes a smudge and the crack becomes nothing. The model's training set was full of cracks, labeled carefully by a person after Lexi drew the first box, and the model learned them at the resolution they were shown, which was the frame's. At inference it is shown the same frame at a fraction of that, and the thing it learned is gone.

This is the single most common reason an inspection model that scored well in training finds nothing in the field, and it has nothing to do with the model's family or its weights.

Slices keep the pixels the crack lives in

The fix is to cut the frame into tiles that are about the size the detector was trained at, run the detector on each tile at full resolution, and put the results back together. A frame several thousand pixels wide becomes a grid of tiles a few hundred pixels square, and inside each tile the insulator is forty pixels again and the crack is a visible line. The detector sees what the labeler saw on the March flight.

Slicing changes nothing about the model. The same trained weights run on the tiles, so a model that was good at cracks in training becomes good at cracks in the field without retraining, which is why slicing is the first thing to try before anyone collects more frames.

Overlap stops a defect being cut in half

Tiles laid edge to edge cut objects at the seams. An insulator that straddles a tile boundary is half an insulator in each, and neither half looks like the thing the model learned. So the tiles overlap, by a fifth or a quarter of their width, so that any object smaller than the overlap appears whole in at least one tile. The overlap is set from the largest small object on the line, which on the desert line in March is the insulator string, and a crossarm that spans three tiles is found by the full-frame pass instead.

Keep the full-frame pass. Run the detector once on the whole resized frame for the large things, poles and crossarms, and on the tiles for the small ones, and merge both sets of boxes. Slicing alone loses the pole the way resizing alone loses the crack.

Merged boxes need a rule for the duplicates

An insulator that sits in the overlap of two tiles is found twice, once per tile, with two slightly different boxes. The merge step has to decide they are one insulator: boxes that overlap heavily, on the same class, are collapsed to one, keeping the box the model was most sure of or the union of the two. Get the rule wrong in one direction and every insulator in the overlap band is counted twice; in the other and two insulators standing close together become one.

The desert line has that second case at every pole, where the three insulators on a crossarm sit a few pixels apart in a frame from 6 am, and the merge rule was tuned on those frames rather than on a default.

An R-CNN pass per tile multiplies the compute bill

Here is the cost. A frame that was one pass through the detector is now a few dozen passes, one per tile, plus the full-frame pass. A two-stage R-CNN style model, which proposes regions and then classifies them, is a good fit for small dense objects. It is also the slower kind, and running it a few dozen times per frame turns a flight that took minutes to score into one that takes an hour. A single-pass detector is quicker per tile and the multiplier is the same.

That bill is paid per frame, forever, on every frame the camera produces. For a drone survey scored after landing it is an hour of a server's time and nobody minds. For a fixed camera on a substation fence, sampled about every two seconds, it is a permanent load on the runner beside the recorder, and it may not fit.

My own view is that slicing is a tax, and the better answer where it exists is to fly closer or mount the camera nearer, so the insulator is a few hundred pixels and no tile is needed. Slicing is for the cameras that cannot move.

Slice only the cameras whose defects are small, and keep the review

On the energy sites we run, the tiled pass is switched on per camera rather than per model. The drone footage of the power line and grid inspection work is sliced, because the defect is a crack on an insulator forty pixels wide. The yard camera watching the gate is not, because a person is three hundred pixels tall from where it hangs. The same model serves both, and the decision is written in the camera's settings rather than the model's.

The frames the sliced pass is unsure of, the insulator half in shadow, the crack that might be a stain, come back to a person the same way any doubted frame does. The corrections retrain the model, the new version replaces the old with no downtime, and the tiles keep showing it the crack at the size it learned. That path, across the grid assets we watch, is what produces the 21,000+ hazard detections per month in our energy work, and most of those are things a few dozen pixels wide in a frame that would have shrunk them to nothing.