Automated pallet accounting from the camera over the staging zone

At 4 pm the stretch wrapper's counter says the line produced one number of pallets, the warehouse system says a second, and the forklift driver's tally sheet on the clipboard by the door says a third. The pallet that collapsed at lunch and was re-wrapped is two wraps on the counter. The pallet that was moved out of the zone and back in while a truck was late is two entries on the tally. The difference between the numbers goes to the shipping manager, who spends the last hour of the day walking the dock counting.

The camera over the staging zone saw every pallet arrive and every pallet leave. A ledger built from that camera has one entry per pallet, with the time it landed and the time it left.

The zone is a polygon on the frame, and the ledger is entries and exits

The staging zone is the floor between the end of the line and the dock door, and it is drawn once on a frame from the overhead camera as a polygon. A pallet is in the zone when the centre of its box is inside the polygon. It entered the zone at the first frame that was true, say 3:12 pm, and left at the first frame it stopped being true, and those two timestamps, with the pallet's identity between them, are a row in the ledger.

That is the whole design. Everything else in this post is about making the identity hold and the polygon mean what it meant on the day it was drawn.

Object detection finds the pallet, and tracking counts it once

The model finds pallets and forklifts as two classes. Object detection on its own gives a box per frame with no memory, so a pallet in the zone for an hour would be an hour of boxes rather than one pallet. Tracking joins the boxes across frames into one identity, and the identity is what gets an entry and an exit.

Tracking breaks in two known ways. A forklift drives between the camera and a pallet and the pallet vanishes for a few frames; without care that is an exit and a new entry. Two pallets set down side by side swap identities when their boxes touch. The fix for the first is a timeout longer than a forklift takes to pass, so a pallet that reappears in the same place within it is the same pallet. The fix for the second is a merge rule for a track that reappears where another one ended.

You type the two classes once, Lexi proposes the boxes on frames from that camera across a shift, and a person from the dock checks them. The checking matters on wrapped pallets under the sodium lights at the far end of the zone, where the shrink film reflects and the box wants to stop at the film rather than the load.

The forklift is its own class, and a pallet leaves only on a forklift

The best guard against a phantom exit is to require a forklift. A pallet cannot leave the staging zone on its own, so an exit is only written when a forklift box was within reach of the pallet box in the frames before it disappeared. A pallet that vanishes with no forklift near it was hidden, and its track is held open rather than closed.

The same rule gives the ledger a second column for free: which forklift took it, from the forklift's identity, and at what time. On the docks we run, that is the column the shipping manager wanted at 4 pm and never had.

The stretch wrapper's counter still counts wraps. Nobody has found a use for it since.

The runner sits beside the recorder, and the ledger stays on site

The camera is already on the recorder, as an RTSP stream, and the recorder is in a cabinet by the dock office. The right place for the model is a runner in the same cabinet. It reads the stream, samples frames at about two seconds apart, which is plenty for a pallet that takes longer than that to cross the zone on a forklift, and writes the ledger to the warehouse system over the local network. The footage never leaves the cabinet. Only the frames the model doubts go out, for a person to check.

I would give that runner no route to the internet except the one the doubted frames take, and I would put it on the same power circuit as the recorder so they fail together and recover together. The deployment page covers how a model leaves the platform and how pixels never do; on a dock, the cabinet is where that promise is kept.

The wider warehouse spatial mapping question, what is where in this facility right now, is the same ledger asked across the whole floor rather than one zone, and it is built the same way.

A wash-down that nudges the camera moves the zone

The failure to expect is the Friday wash-down. The crew hoses the dock, the camera housing gets wiped, and the lens now points a few degrees left of where it pointed when the polygon was drawn. The pallet detector still finds every pallet. The polygon now covers a strip of the dock apron and misses the near corner of the zone, and pallets set down in that corner are never entered. The ledger runs short, quietly, from Friday onward.

The drift catalog calls this a camera moved, and for a zone in image coordinates it is the quietest failure on the list, because no model metric moves. What moves is the ledger against the line's own count: a step down in entries on a maintenance date, while the wrapper's counter holds. That step is the signal, and it is why the ledger is compared with the line count every day rather than trusted on its own.

The check is cheap. On Monday morning someone sets a pallet at each corner of the zone and confirms the box lands inside the polygon on the frame. If it does not, the polygon is redrawn from the new framing, which takes a minute, and a short window of frames is labeled from the new angle so the model has evidence for it.

The doubted frames are what keep the ledger honest

LexData takes the dock model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the overhead camera the dock already has, on a runner beside the recorder, or in the cloud or on your servers if that is where the footage is allowed to go. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The reflective pallets under the sodium lights, which the first version doubted, are what the second version learned from.

The shipping manager still walks the dock at 4 pm. The walk is shorter, because the ledger and the line count agree on most days, and on the days they do not, the ledger says which pallet and which forklift.