Presence and absence detection, is the barrier in place and how long has it been gone

The safety barrier across the mezzanine stairs is a hinged gate that a person opens to carry a box down and is meant to close behind them. On Thursday afternoon it stood open for forty minutes while a pallet was broken down beside it, and nobody on the floor noticed because an open gate looks like nothing is wrong. The camera above the stairs had it in frame the whole time.

Whether the gate is closed is a question with two answers, and the camera can give one of them every couple of seconds. The useful part is what happens with a long run of the same answer.

Object detection with one class and a place to look

The model for this is as narrow as a model gets. One class, the gate, and one region of the frame where the gate lives when it is closed. The question on every sampled frame is whether the class is detected inside that region. Present or absent, and nothing else. There is no need to tell one gate from another or to find gates elsewhere in the building.

Object detection at this scale is mostly a labeling exercise in what absent looks like. The training frames need as many empty regions as full ones, from the same camera, at every hour the floor works, or the model learns the region's background as the gate. You type "gate" once, Lexi proposes the boxes on frames from the stairs camera, and a person checks them, paying most attention to the frames where the gate is half open and the label has to say which side of the line that falls.

The state is a run of frames, and a single frame proves nothing

A frame with no gate detected is not an open gate. It is a frame where the model did not find the gate: a person standing in front of it, a glare off the rail at 3 pm, a box carried past. The state is what a run of frames agrees on. The gate becomes absent when the model has not found it for long enough that a person could not still be walking through, and present again when it has been found for the same span.

The record is the change, with the time. Gate absent at 14:02. Gate present at 14:41. Everything the safety lead wants is a query over rows like that, across the gate, the charging bay and the shadow board.

The rule fires after four minutes gone, and says so with the frame

An alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live: the mezzanine gate absent for more than four minutes, high, to the floor supervisor, once per absence. Four minutes is the floor's number, long enough that a person carrying boxes down has finished, short enough that the gate is closed before the shift forgets it. What arrives is the frame with the empty region outlined and the time the gate went, which is the difference between an alert and a nag.

The alert written as a sentence is the same mechanism for the other two. A forklift absent from its charging bay for longer than a shift is a forklift that was never plugged in. A torque wrench absent from the shadow board at the end of the day is a wrench on a machine somewhere, and the frame shows the empty outline.

The floor supervisor asked for the wrench alert to arrive at 5 pm rather than at four minutes. A wrench gone for the afternoon is normal. A wrench gone at close is a search.

The shadow board is a presence check on every tool at once

The shadow board is the same pattern multiplied. Each outline on the board is a region, each tool is a class, and the model answers present or absent for each at every sample. The 5 pm frame with three empty outlines is the record the toolroom used to make on a clipboard. It is the store safety and validation use case in a plant instead of a shop: is the thing where it is supposed to be, on every camera, without a person walking round to look.

My own view is that presence checks are the best first model most sites can run. The classes are few, the labeling is quick, the failures are obvious, and a floor that has seen the gate alert land twice trusts the next model more than any demonstration.

A pallet in front of the camera is the case to rule out before retraining

The failure that looks like a model failure and is not one: on Monday a pallet is stacked in the corner by the stairs, and from the camera's height it covers the bottom half of the gate's region. The model finds the gate on some frames and not on others, the state flickers, and the alert fires twice an hour on a gate that is closed.

The drift catalog calls this something is now in the way, and it is spatial, which is the clue. The detections fail in one region of the frame while the rest of the frame is fine, and no amount of retraining recovers a gate the camera can no longer see. The fix is a person moving the pallet or the camera, and the check before any retraining is to look at the frame.

LexData takes the presence model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cameras the floor already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The frames from the flickering week come back too, and the person reviewing them sees the pallet before the model is blamed for it.