Motion detection in computer vision that ignores the parked truck

Dock 3 at a regional distribution centre has had a trailer backed onto it since the night shift left at 5 am. At 7:40 a second truck swings into the yard, reverses toward dock 4, and the dispatcher's phone buzzes. That buzz is the useful one. The recorder that used to watch the same yard buzzed on both trucks, on the forklift, on the tree line whenever the wind got up, and on a plastic wrap flapping off a pallet, and by the second week the dispatcher had muted it.

Motion detection is the oldest trick in the camera business, and the reason it gets muted is that it answers the wrong question. Something changed in the frame. The dispatcher wants to know whether something arrived.

A detection becomes an arrival only when the box moves

A detector on the dock camera boxes every truck in every sampled frame. On its own that is a count, and the count for dock 3 has read "one truck" since 5 am. What turns the count into an event is comparing each box with the box the same object had in the previous sample, taken about two seconds earlier.

The trailer on dock 3 has a box that has not moved by more than a few pixels all morning. The truck reversing toward dock 4 has a box whose centre has shifted by a third of the frame between samples. The first is a presence. The second is an arrival, and only the second should reach the dispatcher.

That comparison needs the object to keep its identity from one sample to the next, which is what instance tracking does, so the truck on dock 4 is the same truck across the twenty frames it takes to reverse in. Without the track, a truck that moves is indistinguishable from one truck leaving and a different one appearing.

Frame differencing finds every change, including the trees

The classic method subtracts the previous frame from the current one and thresholds what is left. It is cheap enough to run on the recorder itself, and it is what the muted system at the distribution centre was doing. Its weakness is that it has no idea what it is looking at. A truck reversing produces a blob of changed pixels. So does a branch swaying, a cloud crossing the sun, a rain streak on the housing at 4 pm, and the compression artefacts that ripple across a low-bitrate stream when the light drops.

A team can tune the threshold up until the trees go quiet, at which point a slow truck creeping into the yard goes quiet with them. There is no setting in between that holds through a windy afternoon and a still one.

Differencing still has a job. Used ahead of the detector, it decides which sampled frames are worth running the model on at all, and on a yard that is empty for most of a night shift that is most of them. What it should never do is raise the alert itself.

Compare each bounding box with the last sampled frame

The detector's boxes are what make the comparison meaningful. A bounding box has a class, so the swaying branch never gets one, and it has a position, so the trailer on dock 3 keeps a still box while the truck heading for dock 4 has one crossing the frame. The rule for an arrival is the displacement of a tracked box between two samples, above a margin chosen for the camera's distance from the yard.

The margin matters more than it looks. A truck at the far end of the yard moves fewer pixels per second than a truck at the near dock, so the margin is set for the far case and the near case clears it easily. On the energy sites we run, the same rule watches for a vehicle entering a substation yard, and the margin there is set for the gate at the back of the frame rather than the one in the foreground.

A box that jitters by a pixel or two between frames is the detector settling, and it stays below the margin. A box that jumps and jumps back is a detection error, and the pipeline holds it rather than raising anything.

The parked trailer goes quiet and the arriving truck alerts

By 8 am the yard has three trucks in it. Two have been still for long enough that their tracks are marked stationary, and the third, the one on dock 4, has just come to rest. The dispatcher got one message, at 7:40, with the frame attached and the truck boxed.

An alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live, and delivered to Slack, email or a webhook. "A truck arrives at any dock" is the whole rule. The cooldown keeps a truck that stops, creeps forward and stops again from sending three messages. The severity is routine, because a truck arriving is the yard doing its job. The monitoring and alerts guide walks through attaching the model to the feed and describing the alert in those words.

Nobody at the distribution centre wants the trailer on dock 3 in their inbox. They want to know when it leaves, which is the same rule run backwards: a stationary track whose box starts moving again.

Wind in the trees is the false motion to label out

The first week of any yard camera produces a set of frames the model is unsure about, and on a site with a tree line they cluster on windy afternoons. Leaves against a bright sky, a branch shadow crossing the tarmac, a tarpaulin lifting off a parked load. These come back to a person, who marks them as no truck, and that verdict is a label the next version learns from.

LexData takes the dock model through its whole life. You type what to look for, Lexi puts a box on every truck in every frame, and a person checks each label before anything trains on it. The model then watches the yard cameras the site already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The windy afternoons from the first week are what the second version was trained on.

Weather is the drift the yard camera meets first, and the drift catalog has a chapter on what rain, snow and low sun do to a fixed camera. The trees are the same problem in a smaller frame.

A stopped object that starts moving is the case to test first

My own view is that a motion rule should be tested on the boring case before the exciting one. Everyone checks that an arriving truck alerts. Fewer people check that the trailer on dock 3, parked for nine hours, stays silent through a wash-down, a shift change, and the forklift driving across the front of it, because that is the case that decides whether the dispatcher keeps the notifications on.

The forklift crossing in front is a partial occlusion of the trailer's box, and the box shrinks and grows as the forklift passes. The displacement rule reads that as movement unless the track is stable enough to ride through it. The drift catalog covers occlusion as its own chapter, and on a dock it is a forklift, every few minutes, all day.

The dispatcher at the distribution centre keeps a paper log of arrivals by the door, from the years before the camera, and still writes the time in it when the message comes through.