Solar panel detection in aerial imagery without counting the skylights

The June survey flight over the eastern suburbs came back with several thousand frames, and the distribution planning team wants one number from them per feeder: how many rooftop panels are feeding back into the line. On a frame from survey altitude, a panel is a dark rectangle a few dozen pixels wide, in a grid of identical rectangles, on a roof that may itself be dark. Two streets over, a skylight is a dark rectangle of about the same size on a roof of about the same colour, and a covered pool is a bigger one.

The first model, trained on a few hundred boxes drawn on a Tuesday afternoon, found the panels and the skylights and the pools with equal enthusiasm.

A bounding box per array or per panel decides what the count means

Before a single label goes down on the June frames, the question is what the box goes round. A bounding box per panel gives a panel count and asks a person to draw forty tight boxes on every roof, at a size the survey camera barely resolves. A box per array gives a count of installations and a rough area per roof, and asks for one box per roof, which the camera resolves easily. The planning team's question was capacity per feeder, and capacity follows area more closely than it follows a count of modules the survey cannot reliably separate.

So the box goes round the array, hugging the outer edge of the grid, and the panel count is estimated from the area where it is needed. The vision lesson puts a floor under the alternative: below a few dozen pixels a side, accuracy on any detector falls to a fraction of its headline, and a per-panel box at survey altitude is under that floor on most roofs.

My own view is that per-array is the honest default for anything flown at survey height, and that per-panel should only be promised where the capture was flown low enough to resolve the gaps between modules.

Skylights, pools and dark roofs are the false positives worth labeling

A model trained only on arrays has never been shown what an array is not. The negatives that matter are the ones that share the array's shape: skylights, covered pools, dark solar thermal collectors, flat commercial roofs with rectangular vents, and the shadow of a chimney that falls as a dark rectangle at 10 am. Each of these gets a frame in the set, and the ones that look most like an array get their own class so a person can rule on them at review rather than have the model quietly merge them.

The survey footage already contains all of them. The work is keeping the frames without arrays rather than discarding them, and tagging a few hundred skylights as skylights so the model trains on a roof with a skylight as often as on a roof with an array.

An aside from the flight crew: the pilots keep a running count of backyard pools on every suburban survey, and it is the one number from the flight that the planning team has never asked for.

Tiling keeps a panel at the size the camera saw it

A survey frame is large, and a training pipeline that resizes it to a standard input size shrinks a roof to a few pixels and an array to nothing. The fix is to cut the frame into tiles at native resolution, label on the tiles, and run detection on the tiles, so an array is the size the camera saw it. An array that straddles a tile boundary gets a box in each tile and the two are merged afterwards by their overlap.

Labeling at full resolution is slower per frame and the only version that produces boxes the model can learn from. You type "solar array" once, Lexi proposes a box on every array in every tile, and a person checks them, mostly for the arrays on dark roofs where the edge is hard to see and the skylights the model boxed anyway.

The count per feeder goes to planning with the frames behind it

The boxes per tile become arrays per roof, the roofs become addresses, and the addresses roll up to the feeder. What the planning team receives in July is a count and an area per feeder, and behind each number the frames with the arrays boxed, so a planner who doubts the count on a rural feeder can open the roofs and look. That is the energy work in its usual shape: a question about assets along a line, answered from the air with the evidence attached.

The same survey footage over the utility's own farms is asked a different question: is this table degrading faster than the one next to it. The predictive maintenance use case explains why that question needs a mask rather than a box. Soiled area is a rate measured twice, and a box grows when the aircraft flies closer while the soiling does not.

Snow and leaf cover are the season the model has not seen

The model shipped from a June survey. The November flight comes back with arrays under a dusting of snow, arrays under a fall of leaves from the maple over the garage, and low sun that throws the ridge line's shadow across half of every south-facing roof. The drift catalog covers this as the season turned: same roofs, same arrays, different pixels, and accuracy slides over weeks rather than falling over.

The signal is the review queue. Tiles coming back where the model was unsure whether a white rectangle was an array under snow or a skylight under snow, and a person ruling on them. The correction rate rising through the November survey is the model asking for its first winter, and the frames are already flown. A short window of them, labeled by the person who already ruled on them, is the retraining set, and the model that flies in February has seen snow.

The survey's doubted tiles are the next version

LexData takes the array model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the survey footage, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The same alert path across grid assets is what produces the 21,000+ hazard detections per month in our energy work, and a rooftop array is the quietest asset on that list.

The June count goes to planning in July. The November count disagrees with it on two feeders, and the disagreement is snow on the roofs and a model that had not seen it, which the February count settles.