Computer vision assembly verification and error proofing

Computer vision for assembly verification and error proofing is a camera at the station where a sensor can confirm a part is present but cannot tell which way round it went in. A detector trained on the right build and the known wrong ones gives a verdict on every cycle, and the verdict is kept with the frame that produced it. The threshold is retuned when the mistake rate moves, and the person at the station is shown the frame rather than a code.

Station 9 fits a cable clip into a housing, and the clip goes in either way round. One way holds the cable. The other way looks identical from the operator's side, passes the photoelectric sensor that confirms a clip is present, and lets the cable chafe against the housing for the two years it takes the customer to find out. The plant found out first from a warranty return on a Wednesday, then from the bin of housings behind station 9, which were all fine, and then from the shipped ones, which were not.

The sensor did its job. It was asked whether a clip was there, and it said yes. Nobody had built anything that could be asked which way round.

Object detection judges what the limit switch can only confirm

A limit switch and a photoelectric eye answer a presence question and answer it well. Orientation, seating, finish and completeness are judgment questions, and a camera with a model is the device that answers them. A box on the clip carries its position in the housing and its shape, and a clip fitted backwards has its tab on the wrong side of the box. Object detection puts the box there on every housing that passes, and a rule reads the tab.

The assembly verification use case puts it as three questions: present, in the right place, the right way round. The sensor at station 9 covers the first, and the camera takes the other two, which the sensor was never asked.

Poka-yoke with a camera is a class per mistake the station can make

The classic poka-yoke is a jig that makes the mistake physically impossible, and where a jig can do that it should. The camera is for the mistakes no jig prevents. At station 9 those are known, because the plant has been making them: clip reversed, clip missing, screw missing, connector not fully seated, and the wrong variant of housing on the fixture.

Each becomes a class, and the classes come from the plant's own footage. You type them in the words the line uses, Lexi proposes a box on every clip and screw and connector on every frame, and a person checks each label before anything trains. The reversed clip is the class that takes the checker's time, because from the camera's angle the tab is a few pixels of shadow, and the labeling guide has to show which shadow is which.

The frames are from the station camera at its mount rather than from a phone held over a housing on the bench, since a bench has diffuse light and station 9 has a strip lamp and an operator's hand in the way.

The term poka-yoke came out of Japanese factories in the last century and first meant the jig. The camera is the version for the mistakes the jig cannot reach.

Every decision is kept with its frame, so the escape can be traced

A housing leaves station 9 with a verdict, pass or fail or review, and the frame the verdict was made on, with the boxes drawn. That record is the difference between a warranty return that starts an investigation and one that ends in an afternoon. The serial on the return points to a frame, the frame shows the clip, and the plant knows whether the model missed it, the rule passed it, or the clip was fitted correctly and failed some other way.

The verdict also goes to the station. An alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live: a reversed clip at station 9, critical, to a webhook the PLC reads. The station holds the housing on the fixture until the operator has turned the clip, and the monitoring and alerts doc covers how that webhook is wired. With a runner beside the recorder the alert is raised on the plant floor, which is where the fixture is.

My own view is that the camera should be allowed to stop the station. A plant that only warns has an operator who learns to clear the warning, and the reversed clip ships with a yellow light that flashed. The plants that hold 99%+ accuracy maintained in production on our manufacturing work are the ones where the verdict has a hand on the fixture.

The threshold is retuned when the mistake rate moves

A new operator starts on nights, and the rate of reversed clips triples for a fortnight. The model is finding every one of them correctly. What breaks is the review queue, which was tuned to a base rate of a few a shift and now sends the quality engineer more than she can look at, so she starts skimming, and a skimmed queue is where the next escape comes from.

The drift catalog calls this a change in the defect rate, and its advice holds here: retune the thresholds against the current rate before touching the model, and treat the rate itself as an operations finding. The reversed clips are a training problem at station 9 on nights, and the camera is the thing that found it in a fortnight rather than in two years.

The cheap monitor is the review count by shift against its own history. When nights sends twice what days does, the plant knows where to look.

The operator's override is the signal, and the correction is the label

A housing the model doubts goes to the operator with the frame and the box, and the operator either confirms or overrides. The override rate is the earliest signal that something has changed: a new clip supplier whose tab is shaped differently, a fixture that has been rotated a few degrees after a service, a strip lamp replaced with a brighter one. Every override is a correction, and the corrections are what the next version learns from.

LexData takes the station model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the camera at station 9, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The clip still goes in either way round. The difference is that station 9 now holds the housing until someone has looked, and the plant learns about the new supplier from the override rate on a Thursday rather than from a return two years on.