Production deployment is the halfway point of a computer vision model's life

The carton model on the packaging line was signed off on a Friday with a per-class score on a held-out set that everyone in the room was happy with. On Monday the line ran a new flavour for the first time, in a carton with a different colour band, and the label detector that had been sound on Friday started returning doubted frames on every third carton. Across town, the store's model that counted empty facings on the front-of-store displays had shipped in July. On the last night of September the store reset for autumn, and by opening the model was looking at shelves it had never seen.

Neither model had changed. The score from Friday, and the score from July, were both true on the day they were written down.

A held-out test set is a photograph of a world that has since moved

The evaluation on the packaging line was done properly. A slice of frames held out by time, per-class precision and recall, a confusion matrix, a look at the worst cases. It measured the model on cartons the line had run up to that Friday, and the new flavour was not one of them, because it did not exist yet. The score was a photograph of the line as it was.

The drift lesson says this without softening it: the model is a fixed function, and what changes after deployment is the distance between the frames it learned and the frames it is shown. That distance grows on a packaging line at the rate marketing adds flavours, and in a store at the rate the seasons change and the merchandising team acts on them. An evaluation cannot see either, because both are in the future when the evaluation is run.

The seasonal display rewrites the store's shelves and the model's inputs

Retail teams call the overnight rebuild of the front-of-store a reset, and it is done after close so that the morning's shoppers walk into a different shop. The autumn reset moved the display fixtures, changed the products on them, swapped the summer signage for darker boards and put a stack of pumpkins where the model had learned to expect a gap.

On the ceiling camera's frames, most of what the model used to find an empty facing has moved or changed colour. It has weak evidence for the new fixtures and none for the pumpkins, and so it does what a model does with weak evidence. It returns the frames as doubtful, its detections on the display cameras fall against their own history from August, and the store manager's morning list carries gaps that are not gaps.

That morning is the second half of the model's life beginning, and the question is whether anything was built for it.

The first half of the life produces a model and the second half keeps it

The first half is what most teams budget for: footage, labels, training, evaluation, sign-off. It ends on the Friday. The second half runs for as long as the line runs, and it is made of the moments when the world moves: a new flavour, a reset, a new supplier's board, a lens that creeps out of focus, a coolant that leaves a different film. Each one is ordinary and each one is a place where the model stops matching the world unless something closes the gap.

My own view, and I would draw the budget this way if I were asked, is that half of a vision project's money belongs after the deployment date. A team that spends everything before the Friday has bought a photograph. A team that has kept half for the second half has bought a model.

Edge cases arrive one at a time and the review queue is where they land

The new flavour is an edge case on Monday. It arrives as a stream of doubted frames from the station camera, each with Lexi's best guess at the label box, and a person on the line opens the queue and confirms or corrects them. The corrections are labels checked against the same guideline the first half used, and they are counted: the correction rate on the label class rises on Monday, on one line, on the new flavour's lot codes.

The store's reset arrives the same way, all at once rather than one flavour at a time. The display cameras' doubted frames fill the queue on the morning of the first of October, the store's merchandising lead reviews them with the planogram open, and the corrections say what an empty facing looks like on the autumn fixtures. That rate rising is the drift signal in both places, and it moved on the first morning, weeks before any accuracy figure could have been computed.

The retraining lesson makes the point that a correction on a frame the model got wrong teaches it exactly what it does not know, and is worth many times a label on a random frame. The Monday cartons and the October shelves are those frames.

The new version replaces the old one while the line keeps running

When the corrections cross the project's threshold, a new version trains on the original set plus the corrections. The packaging line's model learns the new flavour's colour band without forgetting the old ones, because the Friday set is still in the training run. The store's model learns the autumn fixtures alongside the summer ones, so that when spring comes and the fixtures move back, it has evidence for both.

Each new version is compared per class against the running one, on the frames that triggered it, and replaces it with no downtime. The line does not stop for the new flavour. The store does not close for the reset. By the second week the doubted frames on the new cartons have thinned to the ordinary rate, and the store's morning list is gaps again rather than pumpkins.

LexData takes both models through their whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cameras the line and the store already have, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The platform is the second half of the sentence built as the default, because the second half is where the model spends its life.

The score worth reporting after deployment is the correction rate

The Friday score answered whether the model was ready. After Monday, the number that answers whether it is still right is the rate at which the people reviewing its frames overrule it, per class, per camera, against that camera's own history. That number needs no fresh labels beyond the ones review produces anyway, it moves on the first morning of a reset, and it falls again when the new version rolls out.

A team built for the second half reports that number every week. A team built for the first half reports the Friday score for a year, and the Friday score does not change, which is the problem with it.