Thermal drone solar panel hotspot detection, and the palette that fools the model

The pilot flies the thermal pass at 11 am, because a hotspot only shows when the array is under load and the sun is doing the loading. On the tailgate the thermographer watches the feed in ironbow, the palette that paints a warm cell orange against a purple module, and points at a cell in the third row that is a shade brighter than its neighbours. That shade is the whole job. It is also, on the frames the model sees, a number that depends on which palette the pilot chose and what span the camera picked that morning.

The hotspot is the easy part. The picture it sits in is the problem.

A hotspot is a box, and object detection has an easy target against a uniform module

A working module under load is close to uniform in the thermal frame: one temperature across its face, a little warmer at the junction box. A hotspot is a region hotter than the rest, and it comes in a few shapes. One hot cell is a diode or a crack or a bird. A hot stripe a third of the module wide is a whole substring with its bypass diode conducting. A whole hot module is a module that is not producing. A hot junction box is a connection.

Each is a box on the module, and object detection finds them well, because the background is flat and the target is a bright patch on it. You type the classes once, Lexi proposes the boxes on the thermal frames from the pass, and the thermographer checks them. The checking is where the shapes get their names, since a model that has only been shown "hotspot" cannot tell the crew whether to bring a diode or a module.

The palette and the span change the pixels, and the array does not

A thermal camera measures temperature per pixel. What the pilot sees, and what most pipelines save, is a rendering of those temperatures through a palette and a span. White-hot maps cold to black and hot to white. Ironbow, the one on the tailgate at 11 am, runs black through purple and orange to yellow. The span is the temperature range the palette is stretched across, and on most cameras it resets itself every frame to whatever is in view.

So the same cell, at the same temperature, is a pale orange on one frame and a yellow on the next, because a hot junction box entered the frame and pulled the span. A model trained on rendered frames has learned the rendering. It is right on ironbow with a wide span and lost on white-hot with a narrow one, and nobody has changed anything on the array.

I would train on the radiometric values, the temperature per pixel, and never on the rendered picture, and I know that is not how most pilots who like ironbow want to work. The compromise that works is one palette and one fixed span, written down and set on the camera before every flight, so that the rendering is at least the same rendering every time.

Reflected sky is the hotspot that is not there

Module glass reflects. At the right angle it reflects the sun as a glint, which reads as the hottest thing in the frame. At other angles it reflects the sky, which is cold, and a cold sky on a warm module reads as a cool patch that moves with the drone. The thermographer dismisses both in a glance and calls the second one the sky. The model has to be taught the same dismissal, with the glints and the reflections labeled as what they are rather than as hotspots.

The flight plan removes most of them. A camera angle a little off the vertical, chosen so the sun is never reflected into the lens, and a heading that keeps that angle across the whole array. On the arrays we fly, the angle is written on the flight card next to the palette and the span, and the 11 am pass is flown to it.

The thermographer confirms the flagged frames, and the confirmation is a label

A hotspot the model doubts, a warm cell at the edge of a shadow or a patch that might be a reflection, comes back to the thermographer with the frame and the radiometric reading beside it. The reading is the cell's temperature and the difference from the module's mean. The thermographer's verdict is the finding the crew acts on, and it is also a label the next version learns from. On a pass at 11 am the doubted frames are a small share of the whole, and they are the interesting ones.

The alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live. A module with a hot substring is a ticket to the operations tech's Slack with the thermal frame and the row named. A single hot cell is routine and goes on the list for the next visit. Across the energy work we run, that path, pointed at grid assets, is what produces 21,000+ hazard detections per month.

A new thermal camera is a sensor replacement, and its first flight is a new baseline

The failure that arrives with an upgrade is the new payload. A sharper sensor, a different lens, a different manufacturer's rendering of the same palette, and every frame from the new camera looks different from every frame the model trained on. The thermographer thinks the picture is better, which it is. The model treats a familiar array as unfamiliar, and the hotspot counts on the first flight with the new camera are not comparable with the counts from the old one.

The drift catalog calls this a camera was replaced, and its advice is to do the first flight with the new sensor properly: label a short window of frames from it, checked by the thermographer, before the counts are trusted. The predictive maintenance for grid assets use case makes the sharper point: progression tracking is the one job where a better camera can make the trend worse, because the step change reads as degradation.

LexData takes the thermal model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then runs on each pass as it lands, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The first flight with the new payload in March is what the version running by summer learned from.

The flight card carries the settings the model was trained on

Palette, span, camera angle, time of day, payload. Those five lines on the flight card are the conditions the model was trained under, and a pass flown outside them, at 3 pm in white-hot, is a pass the model will be wrong on in ways that look like the array changed. The card is kept with the model version it belongs to, and when the payload changes, so does the card.