Wildfire smoke detection from a fixed ridge camera at dawn

At 5:50 am the camera on the ridge above the corridor sees the same thing it saw yesterday: a line of towers running down into the valley, the treeline on the far slope, and mist lying in the low ground where the creek runs. At 6:10 am there is a thin grey column above the treeline on the left, a little darker than the mist, leaning with the wind. By the time a person on the ground would notice it, the column is a plume.

The utility put the camera there because the corridor runs through the trees. The question it is asking is old, and the fire lookouts who used to answer it from a tower did it by knowing exactly what the ridge looked like without smoke.

Object detection on smoke means boxing a shape with no edges

A tower has edges. A person has edges. Smoke is a soft grey region that thins into nothing, and the first job of the labeling standard is to decide where the box goes. Object detection needs a box, so the rule is written down and kept. The box runs from the point where the column meets the treeline to where it thins into the sky, and it is drawn on the column's extent in that frame rather than on where it will be in a minute.

You type "smoke" once, Lexi proposes the boxes on every frame, and a person checks them. The checker's job is consistency more than correctness, because a model trained on boxes that stop at different places on the same kind of column learns a blurred idea of what it is looking for. The frames come from the ridge camera at its mounted angle, at dawn and at noon and at dusk, since the same column looks different under each.

Cloud, mist and road dust are labeled as what they are

The lookout's whole craft was knowing what is not smoke. A low cloud caught on the far slope at dawn. The mist that lies on the creek every morning in autumn and lifts by 9 am. The dust a truck pulls up on the access road. Steam off the cooling towers at the plant beyond the ridge on a cold morning. Each of these has been called in as a fire by somebody, and each of them will be boxed as smoke by a model that has only ever been shown smoke.

So the mist gets its own class, and so do cloud and dust. Not because anyone wants an alert for mist, but because a model that has a name for it stops calling it smoke. On the ridge camera the mist class ends up with more labels than the smoke class, which is right, since the ridge has mist most mornings and fires very rarely.

The retired lookout who used to sit in the tower on the ridge kept a logbook of every column that turned out to be nothing, with the date and the weather. It is the best negative set the project has.

A camera that never moves makes the deviation visible

On most jobs a fixed camera is a trap: the model learns the background instead of the thing, and the day the background changes it breaks. On a ridge the fixed camera is the point. The model sees the same slope, the same towers and the same treeline about every 2 seconds, and a column that was not there in the last frame stands out against a background the model has been shown more often than any lookout saw it.

That only holds while the camera stays where it was put. The energy and utilities work we do runs across line corridors, substations and solar fields, and the fixed camera is the cheapest sensor on the list, with one condition: the mount does not move. A wind-shifted bracket puts a new slice of sky at the top of the frame and a new slice of trees at the bottom, and the model that was trained on the ridge now sees a slightly different ridge. The fix is short, a window of frames from the new framing, labeled and folded in, but somebody has to notice.

The alert has a cooldown and a person who calls it in

A column at 6:10 am is not yet a fire call. The rule is written as a sentence: smoke above the corridor treeline on the ridge camera, high severity, with a cooldown so a growing column produces one alert rather than one every two seconds. It is approved before it goes live, and what arrives is the frame with the column boxed, the camera name and the time.

On the energy sites we run, that alert goes to a person first, in Slack, and the person looks at the frame and decides whether to call the fire service. The model is not the caller. That decision is one a person makes with the frame in front of them, and the frame is what the alert exists to deliver. The same alert path across the corridors and grid assets we watch is what produces the 21,000+ hazard detections per month in our energy work.

My own view is that the cooldown is the most under-thought setting on a smoke camera. Too short and the operator gets a stream of alerts for one column and stops reading them. Too long and the second fire on the same ridge, the one started by an ember from the first, is inside the quiet window.

Fog weeks are when the model sends its frames back

A model trained on a clear summer meets its first week of valley fog and starts doubting. Fog fills the low ground and rises past the treeline, so the place the model learned to look for a column is now a wall of grey. Rain on the lens turns the towers into smears. The model does not fail loudly on those mornings; it sends frames back.

The drift catalog files this as rain, fog and dust: conditions the model rarely saw in training arrive for a week, detection falls off, then the weather clears and it recovers. On a smoke camera the falling off is worth having, because the doubted frames from the fog week are the frames the next version needs.

LexData takes the smoke model through its whole life. You type what to look for, Lexi puts a box on every column and every bank of mist in every frame, and a person checks each label before anything trains on it. The model then watches the ridge camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The fog frames from October are in the version that watches the ridge in November.

The evening light turns the ridge orange

There is a second hard hour every day, and it is the opposite of dawn. At 7 pm in late summer the sun drops behind the far slope, the whole ridge goes orange, and haze that was invisible at noon is lit from behind and reads as a plume. The evening frames are the other batch that comes back for review in the first month, and they are the reason the labeling standard has a class for lit haze.

The corridor question is wider than smoke. The vegetation management use case asks which spans will be in contact with the conductor before the next trim cycle, and the ridge camera is one of the cameras that answers it, on the same footage, with a different class list.