Security camera monitoring with computer vision, turning the camera wall from a recorder into a watcher

The control room at a distribution site has sixteen feeds on the wall and one guard on the night shift. At 2 am she is watching the gate camera because a truck is due, and a person climbs the fence on camera eleven, walks along the back of the yard and leaves through the pedestrian gate. The recorder has all of it. She sees it the following Tuesday, when the missing pallet is noticed and somebody scrubs back through the footage.

The cameras were never the problem. Sixteen feeds and one pair of eyes is the problem, and a recorder answers the question of what happened only after somebody knows to ask.

Object detection is the first step, the rule the second

A model watching the same sixteen feeds finds people and vehicles on every frame, sampled every couple of seconds, on every camera at once. That is object detection, and on its own it produces nothing a guard can use: a box on every person in the yard, all night, including the forklift drivers. What turns the boxes into an alert is a rule about where they are and for how long.

The rule is written as a sentence in LexAlert, with a severity and a cooldown, approved before it goes live. For this yard it reads: a person inside the fenced compound on camera eleven between 10 pm and 5 am, critical, to the control room and to the security lead's phone, with the frame. The guard sees the frame with the person boxed against the fence, and decides in one glance whether it is the contractor who was booked or somebody who was not. The monitoring and alerts guide covers the routing; for a control room, critical goes to the desk and routine goes to a morning list.

The model does not know what a break-in is. It knows what a person looks like on camera eleven at night, and the rule knows the rest.

Entry into a restricted zone is a line crossing

The compound behind the warehouse is drawn once as a polygon on camera eleven's view, in image coordinates. A person counts as inside it when the bottom of their box crosses the polygon's edge, which is the same geometry the hazard zone intrusion detection use case describes for a plant floor. Everything about it depends on the camera meaning tonight what it meant the day the polygon was drawn, so a nudged bracket is checked weekly against the live frame.

Time of day is part of the rule rather than the model. The same person in the same compound at 11 am is a colleague, and the rule says so.

Lingering, a left bag and a gate queue are rules

Once people and vehicles are found and tracked from frame to frame, the other questions a control room asks are rules on the same output. A person who has been within a few metres of the perimeter fence for longer than a walk past is lingering, and the rule is dwell against a zone. A bag on the ground by the visitor entrance that was carried in and is still there ten minutes after the person left is an object left behind, and the rule is a box that stops moving after its person moves away.

The queue at the vehicle gate at 6 am is a count of vehicle boxes inside the approach lane polygon, and the alert fires when the count stays above the gate's capacity for longer than a truck takes to check in. Each of these is one sentence, one severity, one place it lands, and the same model underneath.

My own view is that a control room should receive fewer alerts after the model goes in than it received from the motion detection before. Motion alerts on a windy night are a reason to turn the sound off. A person in the compound at 2 am is a reason to look.

A camera that drops and reconnects is its own alert

Camera eleven's network cable runs through a gland that lets water in, and the feed drops for a few minutes every heavy rain. To the recorder that is a gap in the timeline nobody sees until Tuesday. To a model watching the stream it is a camera that is no longer watched, which is itself something the control room should know about while it is happening.

So the health of each stream is a rule too: a camera that has sent no frames for longer than a set window, routine, to the facilities channel. The reconnect is automatic. The alert is what gets the gland fixed before the winter.

Recorders overwrite on a rolling basis, so an event older than the retention window is gone by the time anybody asks. A model that raises the event on the night it happens is also the reason the footage is still there to look at.

Night and rain frames come back for review

The model that goes live in the first week was trained on the yard's own frames, which means the frames the site had when somebody labeled them, mostly daylight and dry. Camera eleven at 2 am in rain is a different picture: the fence lights bloom, the wet ground reflects the boxes, and a person in a dark coat against a dark fence is the hardest case the model will ever see. Those are the frames it doubts, and doubted frames come back to a person.

LexData takes the yard model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the sixteen cameras the site already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The drift catalog calls a week of storms rain, fog and dust, and its advice is to keep the bad-weather footage, because those nights are rare in the training set and worth the most in it. The guard's verdicts on the doubted frames from the first wet week are what the second version learns from, and by the second winter the person on the fence in the rain is a case the model has seen.