People counting and occupancy from the entrance camera, without a turnstile

The store on the retail park has a clicker. A member of staff stands by the door on the two busiest days of the year and presses it for every person who comes in, and for the other three hundred and sixty-three days the footfall figure is an estimate somebody makes from till transactions. There is a turnstile quote in a drawer, and it has been in the drawer since the refit in March, because nobody wants a barrier at the door of a shop.

The camera above the door has been recording the same threshold since the refit. It sees every person who enters, at every hour, and until now nobody has asked it for a number.

A bounding box on every person in the sampled frame

The model does one thing: it draws a box around each person in a frame. The frame comes from the entrance camera, sampled about every two seconds, and on a quiet Tuesday at 2 pm the frame holds one shopper, the box sits neatly around them, and the count is one.

Labeling is the same task, done once by a person. You type "person" as the class, Lexi proposes a bounding box on every figure in every frame, and a reviewer checks them before the model trains. The reviewer's attention goes to the edges: a person half out of frame at the left, a child beside a trolley, a delivery driver with a stack of totes who is a person and not a customer. That last distinction is a label decision and the reviewer makes it before training, because the model can only tell staff from shoppers if someone told it the difference on the frames it learned from.

Occupancy is the count in the frame right now

There are two numbers a store wants from the door and they are different questions. Occupancy is how many people are inside now. Entries are how many came in today.

Occupancy is the easier one, and the one the camera answers directly. At 2 pm on the Tuesday, count the boxes in the sampled frame from the camera that covers the threshold, add the counts from the cameras that cover the floor, and the total is a reading the way a thermometer is a reading. It has no memory. It is right or wrong on this frame, and the next frame two seconds later corrects it.

Entries need memory. The same person appears in dozens of sampled frames on the way in, and counting boxes would count them dozens of times. A track has to follow one box from frame to frame, and the count increments once, when the bottom edge of that box crosses a line drawn across the threshold in the image. The traffic counting and conversion use case is built on that line, and the line is drawn once, in image coordinates, on the day the camera is set.

The Saturday crowd is where the count goes wrong

At 11 am on Saturday the door is different. Four people come through abreast, a pushchair in front of a parent, a couple side by side, a child between two adults. The camera sees a cluster, and the model that learned people one at a time on Tuesday afternoon draws two boxes where there are four.

The drift catalog files this under something is now in the way. The thing in the way is other people. Detections thin out in the middle of the frame, where the crowd is, while the edges of the frame are unchanged, and the count under-reads on exactly the days the store cares about most.

Nobody notices from the number. A count of two hundred on a Saturday looks like a count. What surfaces it is the review queue: the model sends the frames it is unsure of, the reviewer draws the two missing boxes on the pushchair frame, and the correction rate for the door camera climbs on Saturdays and falls on Tuesdays. That pattern is the signal, before any figure has moved.

LexData takes the door model through its whole life. You type what to look for, Lexi puts a box on every person in every frame, and a person checks each label before anything trains on it. The model then watches the cameras the store already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The Saturday frames that came back in the first fortnight are what the second version learned crowds from.

Faces are blurred before the frame is kept

A person count does not need a face. The box needs a head and shoulders and a pair of legs to find the bottom edge, and nothing in the count depends on who the person is. So faces are blurred on the frame before it is stored or sent for review, and the reviewer drawing the missing boxes on the Saturday cluster is drawing them on blurred heads.

My own view is that this should be the default for every entrance camera whether or not a regulation requires it. A count that staff trust is a count that staff will use, and staff do not trust a system that keeps their customers' faces. The blur runs at the point the frame is taken from the feed, on a runner beside the recorder if there is one, so the footage that leaves the site never carried a face in the first place.

The store's security recording is untouched. It is a separate stream with a separate purpose, and the counting model never reads it.

The line is drawn in the image and the door is in the world

The one thing the count depends on that nobody writes down is where the camera points. The threshold line lives in image coordinates. A cleaner knocks the housing in March, the camera now looks a hand's width to the left, and the line that crossed the door mat crosses the floor tiles in front of the trolley bay instead. Every person who parks a trolley becomes an entry.

The count changes on a day footfall did not. Compared with the till, it drifts in one direction from one date, and the date matches a maintenance visit. The fix is to re-draw the line from the new framing and re-label a short window of frames, and the frames already exist.

The retail work behind our 4M+ annotations and validations came from stores where the cameras were already above the doors and along the aisles. The retail picture as a whole, from queue length to shelf gaps, is built the same way, one camera and one rule at a time. The clicker stays in the drawer for the two busiest days, as the number the camera is checked against.