Monitoring a computer vision model in production, and what to do when the corrections start climbing

The shelf model went live in September in a store with plain white lighting and a bakery aisle that looked the same on every frame. It found the empty slots, the alert went to the morning list, and for six weeks the department lead had nothing to correct. In the second week of November the store hung the seasonal displays: a cardboard arch over the bread aisle, red foil on the end caps, and warm lighting on a timer that comes on at dusk. The model's boxes started landing on the arch.

Nothing broke. The model was doing what a model trained on a September aisle does in a November one, and the only person who noticed was the department lead, overriding the empty-slot alert on a full shelf every morning.

The test set stopped being the world in November

The held-out frames the model was evaluated on were September frames. They are a snapshot of one aisle in one light, and every accuracy figure from them is still true of September. Production is the aisle as it is today, and a model that is right on the snapshot can be wrong on the aisle without any figure on a dashboard moving, because the dashboard is scored on the snapshot.

That is the reason to stop treating launch as the end of evaluation. The model is a system under observation from the first day it runs, and the observation has to come from the aisle rather than from the test set.

The correction rate is the signal, and it climbs before anything else does

What the department lead did every morning is the signal. A frame came back with a box on the arch, the lead marked it as no gap, and that override is one correction. Chart the corrections per week per camera and the bread aisle's line turns up in the second week of November while every other aisle in the store holds flat. That is drift, read from the only place it is visible, which is people overriding the model in a consistent direction.

The lesson on what model drift actually is makes the general case, and the shelf aisle is the plain version of it: same shelves, same definition of a gap, different pixels. The frames coming back, the corrections landing and the override rate rising are the three views of one number, and it is the number to watch before any accuracy figure, because the accuracy figure is scored on September.

The lead keeps a tally of the overrides on the back of the morning list. It matches the chart.

Doubted frames from object detection go first in the review queue

A model that returns the frames it is unsure of produces a queue, and in a normal week the queue is short and mixed. In the second week of November the queue is long and it is all bread aisle. Those frames go first: they are the ones where object detection has already flagged that the aisle no longer looks like the training set, and every verdict on them is a labeled frame from exactly the conditions the model is failing in.

The confidently wrong frames do not come back on their own. The model boxed the arch without doubt, which is why the lead's override on those frames matters as much as the queue. Both paths end in the same place, a corrected label with the frame, and the correction is what trains the next version. Asking a question about the footage does not.

The new version is compared on the same frames the old one failed

When the corrections cross the project's threshold a new version trains on them. Before it goes anywhere it is compared with the old version on the same frames, including the November ones the old version got wrong and the September ones it got right. A version that learned the arch and forgot the plain aisle has not improved. The comparison is per class and per camera, because an aggregate hides a version that got better on the bread aisle and worse on dairy.

LexData takes the shelf model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the aisle cameras the store already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. Each version keeps what it was trained on, so the question of what changed between the September model and the November one has an answer in frames.

My own view is that a version should be rolled out on the strength of the frames it fixed, and not on the strength of a number that went up. The department lead can look at twenty frames and say whether the arch is still a gap. A figure that improved in the third decimal place tells nobody anything.

The rollout has no downtime, and the old version stays

The new version replaces the old one on the bread aisle camera without the store noticing. The old one is kept, with its training set, so a rollback is a decision rather than a rebuild, and so that the comparison in January, when the arch comes down, can be made against both. The platform does this the same way for every camera on every site, which is what makes twenty stores in December a repeat of one store in November rather than twenty new problems.

The correction rate on the bread aisle went back to flat within a week of the rollout. In January the arch came down, the warm lights went off, and the rate stayed flat, because the November frames were added to the September ones rather than replacing them. The aisle the model was trained on now has two seasons in it, and the retail work behind our numbers, 4M+ annotations and validations, is mostly seasons like that, folded in one at a time by the person with the morning list.