Entrance counting with computer vision from the camera already over the door

Entrance counting with computer vision means the camera that already looks at the door becomes the counter. A model draws a box on every person in a frame sampled from that camera. A tracker keeps the same box on the same person from one frame to the next, and the count goes up by one when the bottom of the box crosses a line drawn across the threshold in the image. No turnstile, no beam across the doorway, no new hardware over the mat.

The store on the retail park has a clicker. A member of staff stands by the door on the two busiest days of the year and presses it for every person who comes in, and for the other three hundred and sixty-three days the footfall figure is an estimate somebody makes from till transactions. There is a turnstile quote in a drawer, and it has been in the drawer since the refit in March, because nobody wants a barrier at the door of a shop.

The camera above the door has been recording the same threshold since the refit. It sees every person who enters, at every hour, and until now nobody has asked it for a number.

Entries and occupancy are two different questions

There are two numbers a store wants from the door and they are different questions. Entries are how many people came in today. Occupancy is how many are inside right now.

Occupancy is the easier one, and the one the camera answers directly. At 2 pm on a Tuesday, count the boxes in the sampled frame from the camera that covers the threshold, add the counts from the cameras that cover the floor, and the total is a reading the way a thermometer is a reading. It has no memory. It is right or wrong on this frame, and the next frame two seconds later corrects it.

Entries need memory. The same person appears in dozens of sampled frames on the way in, and counting boxes would count them dozens of times. A track has to follow one box from frame to frame, and the count increments once, when the bottom edge of that box crosses the line. The traffic counting and conversion use case is built on that line, and the line is drawn once, in image coordinates, on the day the camera is set.

Queue length is a third question and belongs to a different camera. It is a count inside a polygon over the tills rather than a line at the door, and queue counting with a polygon is its own post.

A bounding box on every person in the sampled frame

The model does one thing: it draws a box around each person in a frame. The frame comes from the entrance camera, sampled about every two seconds, and on a quiet Tuesday at 2 pm the frame holds one shopper, the box sits neatly around them, and the count is one.

Labeling is the same task, done once by a person. You type "person" as the class, Lexi proposes a bounding box on every figure in every frame, and a reviewer checks them before the model trains. The reviewer's attention goes to the edges: a person half out of frame at the left, a child beside a trolley, a delivery driver with a stack of totes who is a person and not a customer. That last distinction is a label decision and the reviewer makes it before training, because the model can only tell staff from shoppers if someone told it the difference on the frames it learned from.

Direction is what separates an entry from an exit

A line on its own counts crossings. A door at the retail park has people going both ways, so each crossing needs a direction, and direction comes from the track. The box was on the car park side of the line in the earlier frames and on the shop side in the later ones, so that crossing is an entry. The reverse is an exit.

A person who stops on the line, turns to talk to someone and walks back out has crossed it twice in opposite directions and should count as nobody. A person who comes in, stands on the mat reading the offers board for a minute and then carries on has crossed it once and should count as one. Both are decided by the track rather than by the frame, which is why a counter that works on single frames cannot do entries at all.

Where the camera is decides most of the accuracy

The counting camera is usually the security camera, mounted where the installer could reach, at an angle that shows faces to a guard. That angle is the wrong one for counting. A person walking towards an angled camera hides the person behind them, and the box on the front shopper swallows the one at their shoulder.

Manufacturers who sell purpose-built counters publish the geometry, and it is worth reading even when the camera is a general one. Axis asks for its overhead counter to be mounted straight above the point where people pass and facing straight down, with a level floor and ceiling. It wants the view clear of a door that swings into frame and of large signs hanging from the ceiling, enough white light on site, and no loitering people or swaying objects under the lens (Axis P8815-2 installation guide). Their table of coverage against mounting height is specific to that device, but the shape of it holds for any camera: higher means a wider doorway in one view and less of one person hiding another.

A store that cannot move the camera can still move the line. Drawn across the floor a step inside the door rather than on the mat, it sits where people have separated into a single file and are walking rather than pausing, and the swing of the door is out of the picture.

The Saturday crowd is where the count goes wrong

At 11 am on Saturday the door is different. Four people come through abreast, a pushchair in front of a parent, a couple side by side, a child between two adults. The camera sees a cluster, and the model that learned people one at a time on Tuesday afternoon draws two boxes where there are four.

The drift catalog files this under something is now in the way. The thing in the way is other people. Detections thin out in the middle of the frame, where the crowd is, while the edges of the frame are unchanged, and the count under-reads on exactly the days the store cares about most.

Nobody notices from the number. A count of two hundred on a Saturday looks like a count. What surfaces it is the review queue: the model sends the frames it is unsure of, the reviewer draws the two missing boxes on the pushchair frame, and the correction rate for the door camera climbs on Saturdays and falls on Tuesdays. That pattern is the signal, before any figure has moved.

LexData takes the door model through its whole life. You type what to look for, Lexi puts a box on every person in every frame, and a person checks each label before anything trains on it. The model then watches the cameras the store already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The Saturday frames that came back in the first fortnight are what the second version learned crowds from.

The clicker checks the camera, one hour at a time

A count nobody has checked is an estimate with a decimal point. The check is the clicker. Someone stands by the door for an hour on a quiet weekday and an hour on a Saturday morning and counts by hand. The two hours are then compared with the camera's count for the same two hours, read from the recorder's clock so the windows line up.

The weekday hour usually agrees to within a person or two. The Saturday hour does not, and the frames from that hour show why, which is the pushchair cluster above. The manual count is itself imperfect in a crowd, so a disagreement of one or two in a busy hour says nothing, and a disagreement that grows with the crowd says everything. The check is repeated after each retrain and whenever the camera has been touched.

The retail work behind our 4M+ annotations and validations came from stores where the cameras were already above the doors and along the aisles. The retail picture as a whole, from queue length to shelf gaps, is built the same way, one camera and one rule at a time, and each rule is checked against the person who used to do the job.

What the GDPR asks of a counting camera

A person count does not need a face. The box needs a head and shoulders and a pair of legs to find the bottom edge, and nothing in the count depends on who the person is. So faces are blurred on the frame before it is stored or sent for review, and the reviewer drawing the missing boxes on the Saturday cluster is drawing them on blurred heads.

In the EU and the UK the law behind this is the General Data Protection Regulation, and the European Data Protection Board's guidelines on video devices are the reading that matters for a shop (EDPB Guidelines 3/2019 on processing of personal data through video devices, adopted in January 2020). Three parts of them shape a counting camera. The guidelines name simple counting algorithms as the less intrusive end of video analysis, against biometric systems at the other end. They restate the regulation's data minimisation principle: the processing is limited to what the purpose needs, and a count needs no identity, so the frames that leave the camera should carry none. And they ask for a storage period set for the purpose and defended, with anything beyond three days needing more argument, which for a counter means keeping the count and discarding the frames.

The sign at the door, the first layer of information in the guidelines' words, says the camera counts.

My own view is that the blur should be the default for every entrance camera whether or not a regulation requires it. A count that staff trust is a count that staff will use, and staff do not trust a system that keeps their customers' faces. The blur runs at the point the frame is taken from the feed, on a runner beside the recorder if there is one, so the footage that leaves the site never carried a face in the first place.

The store's security recording is untouched. It is a separate stream with a separate purpose, and the counting model never reads it.

The line is drawn in the image and the door is in the world

The one thing the count depends on that nobody writes down is where the camera points. The threshold line lives in image coordinates. A cleaner knocks the housing in March, the camera now looks a hand's width to the left, and the line that crossed the door mat crosses the floor tiles in front of the trolley bay instead. Every person who parks a trolley becomes an entry.

The count changes on a day footfall did not. Compared with the till, it drifts in one direction from one date, and the date matches a maintenance visit. The fix is to re-draw the line from the new framing and re-label a short window of frames, and the frames already exist. The same nudge moves every zone on the ceiling cameras over the floor, so a store that checks one camera after a maintenance visit should check them all.

Four questions store teams ask about entrance counting

Does the counting camera have to be the one over the door

No. Any camera that sees the threshold from above, with the whole width of the door in view and people walking rather than pausing, can carry the line. The camera over the door is usually the best one because it is already there and already pointed at the right place.

What happens to the count when the store is full

Occupancy is read from the frame and stays right. Entries under-read in a crowd, for the occlusion reason above, and the under-read shows up as a rising correction rate on busy days before it shows up in the number. A store that sets a hard occupancy limit sets it as a rule with a margin and checks the count by hand on the first few busy days.

Can the same camera count staff and customers separately

Only if the labels did. A delivery driver, a member of staff in uniform and a customer are the same shape to a model that was taught "person". Teaching it the difference is a second class, labeled on the store's own frames, and the reviewer decides the ambiguous cases before training rather than after.

How long are the frames kept

Long enough to review the doubted ones and retrain, and no longer. The count is the record. The frames behind it are blurred on capture, kept for the review window the store sets under its own retention policy, and then gone.

The clicker stays in the drawer for the two busiest days, as the number the camera is checked against.