RETAIL
Shelf & Planogram Compliance
Shelf & Planogram Compliance. What it takes to run it.
The empty facing and the pack out of plan, from the aisle camera the store already has.
// 01 · The question
“Is the right product in the right facing, and what is out of stock?”
// 02 · Why it is hard
The class count is enormous, hundreds of SKUs, many visually near-identical, and it changes constantly. Distinguishing two flavours of the same brand at shelf angle is the hard part.
// 03 · What the labels have to be
Bounding box
Decided before the first frame
Compliance is checked position by position against a planogram grid. Boxes map directly onto facings, and the comparison the system runs is between a grid of detections and a grid of expectations.
This is the decision that is expensive to reverse. The geometry has to match what the answer contains, and finding out it does not means labelling the set a second time.
// 03a · What it finds
The empty facing
a box on the gap in the aisle camera's frame, with the rack section around it boxed as well, so the gap has a bay and a shelf without the model needing to know what is missing
The product in each facing
a box per pack on the shelf, one class per product the store names, learned from this store's frames at the ceiling camera's angle rather than from catalogue photographs
The pack that is not where the plan puts it
a box of one product's class inside the block of shelf drawn for another: the family size on the second shelf because the night crew found room there
The shelf label under the facing
boxed on the shelf edge, with the price on it read by asking a question of the frame when the store wants the tag checked against the pack above it
// 03b · What it checks
- 01
The facings in each block are the facings the plan calls for. The plan is written as rules on zones drawn over the bay on the camera, which product belongs in this block of shelf and how many facings of it, and a count below that is a report.
- 02
A block holding a product that does not belong in it is a report, named by the zone, so the pack filled from the wrong case in the own-brand block is the one that gets flagged.
- 03
The report is read on the frame the schedule names. A bay out of plan on the morning frame before opening goes to the morning list; the same bay at four in the afternoon, mid-restock, is not a report, because the rule carries a schedule.
- 04
The rule carries a cooldown and a severity, so a gap that stays open is reported once and not on every frame, and a bay out of plan is told apart from a facing short by one.
- 05
The rule was approved before it went live. It is written as a sentence, run against footage from this aisle, read by a person and only then switched on.
- 06
A frame the model is unsure of is not a report. It comes back to a person with the boxes and the zones drawn on it, the correction is recorded against the frame, and the model is retrained on the hardest frames of the week.
// 03c · What the store provides
- A camera that sees the bay. The one already on the ceiling over the aisle, at the angle it has, with the aisle lights on. Any RTSP feed or recorder; no shelf-edge camera, no rail, no robot and no new hardware in the aisle.
- Frames from this store, not catalogue photographs. The model learns your packs at your camera's angle and under your light, and the empty facing as it looks on your shelving, with the back panel and the pusher it has.
- The plan, in the store's words. Which product belongs in which block of the bay and how many facings, written as rules on zones drawn over the camera once per bay, and rewritten when the bay is reset.
- Someone with the morning list. The report reaches the store on Slack, email or a webhook into whatever the store already carries, with the frame attached and the bay and shelf named, so the fix is made from the picture.
// 04 · How we run it
Three parts of one loop, on this job.
LexAnnotate
Shopper Tracking
Annotate in-store video for customer detection, employee identification, door events, and product placement. Distinguish between customers, staff (by uniform), and children within each camera's region of interest.
LexInsight
Conversion Analytics
Turn raw foot traffic data into conversion intelligence. Link visual detections to store zones, time slots, and operational events. Surface patterns that static dashboards miss.
LexAlert
Store Alerts
Monitor detection accuracy across store locations. Detect when camera repositioning, seasonal displays, or lighting changes degrade model performance. Route the failures back for retraining.
// 05 · What breaks it after launch
Packaging refreshes are constant and the model is never told. The new pack is correct on shelf and unrecognised by the model, which reports an out-of-stock on a full facing.
It is not the only one that can get this use case, it is the one that usually gets it first. All ten conditions.
// 06 · In retail
Store Intelligence
The Challenge
The models lacked high-quality labeled data, activity alerts came back false, and unmonitored registers left customers waiting at checkout.
Our Execution
Labeled customer and staff positions. Validated alerts and traffic counts for accurate footfall tracking.
The Result
Faster AI training with accurate labeling. Smarter alerts for open doors and empty registers.
4M+
Annotations & validations
// 06a · Questions we get asked
Do we need shelf cameras, a rail or a robot?
No. The ceiling camera that already looks along the aisle is the one we use, over RTSP or from the recorder. What it has to see is the whole bay down to the shelf edge, at a distance where one facing can be told from its neighbour. Where the angle at the far end of the bay is too steep, or the bottom shelf is behind the shopper standing in front of it, we say so before anything is built.
Can it tell two flavours of the same brand apart at shelf angle?
Sometimes, and that is the honest answer at the start. The empty facing needs no product knowledge, so the gap is found on the first version. Telling two near-identical packs apart depends on what the camera resolves at that distance, and on a person checking the labels on the frames where the difference is one word on the pack. The store decides which distinctions matter for the plan, and the labeling standard carries that decision rather than a default we ship.
What happens when a pack is redesigned, or the bay is reset?
Nothing in the footage announces either, and both change the right answer. A redesigned pack is correct on the shelf and unknown to the model, which reports a gap on a full facing. A reset moves the family size on purpose, and yesterday's rule flags a correct bay. The first frames after a changeover are labeled and folded in before the model is trusted on them, the zones and rules for the bay are rewritten the same morning, and versions keep what they were trained on, so the store that has not reset yet keeps its own.
What does the store receive, and when?
A report on the frame the schedule names, usually the one before opening: which camera, which bay and shelf, what the rule saw against what the plan says, and the frame it was read from, on Slack, email or a webhook into the system the store already uses. A cooldown keeps a gap that stays open from reporting on every frame, and a severity tells a facing short by one from a bay out of plan. It arrives as fixes with pictures rather than as a score.
Read next
Planogram compliance from one frame of the aisle camera, the full walk-through →On-shelf availability, the empty facing as the class and the glare that fakes it →Shelf item detection, a hundred products in one frame →Shelf price verification from the same aisle camera →What a pack refresh or a planogram reset does to a shelf model →