Aerial fire detection from a drone patrol, smoke before the flame reaches the line

The August patrol runs the right-of-way north from the substation at a few hundred feet, the pilot watching for canopy that has grown into the clearance since the spring trim. Halfway along the third span there is a grey smudge above the ridge to the east. It could be haze off the valley, it could be dust from a farm track, or it could be the first ten minutes of a fire that will reach the line by evening. The pilot notes it, the patrol continues, and the note is read the next morning.

A model on the patrol footage does not have a next morning. It sees the smudge on the frame it appears in and either raises it or lets it pass.

Object detection on smoke that is a few dozen pixels wide

At the August patrol's altitude, early smoke occupies a patch of the frame a few dozen pixels across, low in contrast, with no hard edge. It looks like haze because for the first minutes it mostly is haze, with a source. The features a detector can hold onto are a plume shape that narrows toward the ground, a colour that sits slightly apart from the sky behind it, and movement across consecutive frames that cloud shadow does not share.

Object detection for a target that small is a resolution question before it is a model question. A frame from a high-resolution drone camera, resized down to a standard training size, loses the smudge entirely. Tiling the frame and running the detector on each tile at full resolution keeps it. The vision lesson puts a rough floor under this: below a few dozen pixels a side, accuracy on any detector falls to a fraction of its headline, and the fix is in the capture and the tiling rather than the weights.

Smoke and flame are two classes with boxes, labeled apart

The temptation is one class, "fire". The right-of-way argues for two. Flame is bright, high contrast and rare in daytime patrol footage, because by the time flame is visible from altitude the fire is well established. Smoke is dim, common and the earlier signal. A single class merges the easy target with the hard one and the model learns the easy one.

So the classes are smoke and flame, boxes on both, and the box on smoke hugs the plume rather than the sky around it. You type the two classes once, Lexi proposes the boxes on every frame of the patrol, and a person checks each one. The checking is where the smoke class gets its definition: a person has to rule on whether a grey patch is a plume or a cloud shadow, and the ruling is written down so the next patrol is labeled the same way.

An aside from the patrol crews: the fire lookout towers along the range are still staffed in August, and the lookout's log of what they called haze and what they called smoke is a labeling guide nobody thought to ask for.

Haze, dust and cloud shadow go into the set on purpose

A model trained only on frames with smoke in them has never been told what smoke is not. The negatives matter as much as the positives here, and they have to be the right negatives. Valley haze at 7 am. The dust plume behind a tractor on the access track. The shadow of a cumulus cloud moving across a hillside, and the steam off a cooling pond. Each of these is a frame the model should see and pass.

The patrol footage already contains them. The work is keeping them rather than discarding every frame without a plume, and tagging a few hundred of them as negatives so the model trains on the ridge with nothing on it as often as on the ridge with a smudge.

The alert is a sentence with a cooldown so a plume does not page forty times

A plume that persists across the whole patrol leg is one event, and a detector sampling the footage will find it on every frame. Without a cooldown, the vegetation lead gets forty messages about the same ridge. The alert is a rule written as a sentence, "smoke or flame within the right-of-way corridor", with a severity and a cooldown, approved before it goes live, and delivered to Slack, email or a webhook with the frame attached.

On the energy patrols we run, the smoke rule carries the highest severity and goes to the person on call rather than a channel, and the cooldown is long enough that one plume produces one message with the best frame. That alert path across grid assets is what produces the 21,000+ hazard detections per month in our energy work, and a fire plume is the one detection on the list where the frame arriving in time matters more than any other. The energy page covers the rest of that list.

Rain, fog and dust are the week the model goes quiet

The patrols that matter most for fire are flown in the conditions the model saw least. Late August brings dust off the dry fields, fog that sits in the valley until mid-morning, and the smoke of a controlled burn three ridges over. The drift catalog covers this as rain, fog and dust: the share of frames where the model finds nothing rises for a week, then returns, and nobody touches anything.

The fix is the cheapest on the catalog. Keep the bad-weather footage. Those days are rare in a training set and disproportionately valuable in it. The fog frames are where the model has to learn that fog is not smoke, and the burn frames are where it has to learn that smoke three ridges over is still smoke. The corrections from those days, checked by a person, are the retraining set for the next season.

The patrol's own frames retrain the model before the next season

LexData takes the smoke model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the patrol footage, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The August patrol's doubted frames, the haze that looked like a plume and the plume that looked like haze, are what the model flying in September learned from.

My own view is that smoke is the class to optimise for, and flame is confirmation. A model that finds flame well and smoke poorly is a model that finds fires the crew could already see. The right-of-way patrol exists to find the ones they cannot, and the vegetation management work it was flown for is the same footage with a different question asked of it.

The smudge above the ridge on the third span comes back as a doubted frame with a box on it. A person looks at it at 4 pm rather than the next morning. Whether it was haze, the model is better for having asked.