Three reasons a computer vision pilot never makes it from the lab to the floor

The pilot at the machining cell scored well in the lab. A set of photographs of finished housings, taken on a bench with a ring light, trained a model that found chatter marks and tool scoring on nearly every one. The slide went to the operations director in May, the pilot was approved for the floor in June, and by August the cell's operators had stopped looking at the screen.

Nobody could say what had gone wrong, because three separate things had, in sequence, and each one looked like the model being bad. The model was the least of it.

The cameras were fitted for security and cannot see the defect

The camera the pilot used on the floor was the one already on the cell. It was mounted high in the corner one February by the contractor who did the fire alarm, with a wide lens chosen so one camera would cover the whole cell for the safety file. It sees the robot, the fence, the machine door and the operator. The machined face of a housing, on the outfeed table, occupies a few dozen pixels in that view, and a chatter mark on it occupies none.

The lab photographs were taken from a hand's width away under a ring light. No model trained on those can find a mark in a picture where the mark is smaller than the sensor's ability to resolve. The what vision can see lesson has the test: if the best inspector on the floor could not make the call from that camera's frame, neither can the model. The pilot's first failure was optics, and it was decided the day the safety contractor chose the lens.

This one is the failure I think kills the most pilots and gets blamed the least, because it is invisible in the lab. The fix is a camera at the outfeed, close, with its own light, and it costs a bracket and an afternoon. The pilot spent its budget on a model instead.

A stock model has never seen your parts

With a camera at the outfeed the second failure appeared. The model on the pilot was a general defect detector, sold as pre-trained on surface defects, and it had learned what scoring looks like on somebody else's parts: a different alloy, a different finish, a different light. On the cell's housings it boxed the machined witness lines as defects, missed the fine chatter on the bore face, and produced a queue of false alarms the operators cleared for a week and then ignored.

A model has to be trained on the plant's own frames, from the outfeed camera, across the parts the cell actually runs and the shifts it runs them on. That is not a large ask once the camera is right. A window of frames from the new mount, labeled against the plant's own definition of a reject and checked by a person before anything trains, is a model built for these housings. The surface defect detection use case is the same shape at any cell, and the manufacturing work behind our numbers, 99%+ accuracy maintained in production, was built one plant's frames at a time.

The pilot's stock model had one thing right: the general shape of a scratch. A model trained on the cell's frames starts from that and learns the rest in a fortnight.

Drift arrives a month after launch and nobody is looking

The third failure is the one the pilot never reached, and the one that ends most of the pilots that get past the first two. The retrained model went live in late September and was good. In October the plant switched to its winter coolant, which leaves a different film on the housings, and the days shortened so the skylight over the cell stopped contributing to the outfeed light after 4 pm. The model had weak evidence for both, and its accuracy slid over weeks rather than falling over.

The drift lesson puts this plainly: the model did not change, the world did, and the gap between the two grew. The drift catalog covers the calendar version as the season turned, and its advice is that the first winter is where the corrections matter most, because a model trained in one summer is always weakest in its first winter.

What made it a failure rather than a maintenance task is that nobody was looking. The pilot had an accuracy figure from September and no plan for a second one, and an accuracy figure needs fresh labels, which nobody had budgeted. So the slide stayed at the September number and the operators, who could see the misses on the outfeed table every afternoon, stopped trusting the screen without anyone upstairs knowing why.

The correction rate is the signal and the corrections are the fix

The operators were the monitor. Every afternoon one of them overrode the model on a housing it passed, and if that override had been counted, the count would have risen through October in a consistent direction on the afternoon shift. That rise is the drift signal, and it arrives weeks before any accuracy figure could, because it needs no fresh labels beyond the ones the operators were already, in effect, making.

Those same overrides are the fix. The frames the model doubted after 4 pm come back to a person, the frames the operators overrode join them, the corrections are checked and counted, and when they cross the project's threshold a new version trains on the September set plus the winter frames. It is compared per class against the running version on the afternoon frames that caused the retrain, and it replaces the old one with no downtime. The version that goes into November has seen the winter coolant.

LexData takes the housing model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the outfeed camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The platform is that sentence with the operators' overrides counted, which is the part the pilot had no place for.

Fix the three in the order they arrive

The order matters because each failure hides the next. A model retrained on the plant's own parts cannot be judged through a security camera that cannot resolve them. A drift monitor cannot be trusted on a stock model that was wrong from day one. So the camera first, then the plant's frames, then the loop, and each step is smaller than the pilot's original budget.

The operations director's August question was whether the model was any good. Once the outfeed camera was up and the model was trained on the cell's own housings, the answer was yes, and it stayed yes through the winter because the operators' overrides had somewhere to go.