Computer vision applications on a factory floor, four jobs for the cameras already there

The plant makes a small kitchen appliance, and by 7 am on a Monday all three floors of it are running. Steel housings come off a press on the ground floor and go to an assembly line on the mezzanine, where the heater, the thermostat and the wiring go in. They come down again to a packing line where each unit is boxed and stacked on a pallet. There are cameras over all of it, installed at different times for different reasons, recording to two different boxes in the electrical room.

The plant manager wants to know what those cameras could do. The honest answer is four things, on the frames they already produce, and the four have more in common than they look.

Defect detection at the press is a severity call

The camera at the press outfeed sees every housing for a couple of seconds. Scratches, dents and a burr along the edge are the defects, and each is a class a person labels from the plant's own frames. What the plant has to decide before the first label is which of those defects matter where. A scratch on the visible front of the housing is a cosmetic defect and the unit goes to rework. The same scratch on the inside of the base is on a face no customer will see and the unit ships. A dent near the thermostat mount is functional whatever it looks like.

That distinction goes into the labeling schema as the class name, scratch on the visible face and scratch on the hidden face, rather than into somebody's head. Automotive drawings have a word for this, the A-surface, and the plant's own drawing has an equivalent. In LexAnnotate you type the classes in those words, Lexi puts a box on every frame, and a person checks each label before anything trains on it. On the manufacturing lines we run, 99%+ accuracy maintained in production is measured against a schema that says which face the scratch is on.

Assembly verification is a completeness check on absence

The camera over station 6 on the mezzanine sees each housing after the wiring goes in. The question is whether the thermostat, the earth strap and the four screws are present, in the right place, the right way round. The assembly verification use case describes why this is a box question: presence, position and orientation are all read from a box and its aspect, and no measurement of extent is needed.

The hard part is that the signal is an absence. A housing with three screws looks almost exactly like a housing with four, and the missing one is a small dark hole in a cluttered frame. The labels that matter are the housings with something missing, which are rare on a good line, and a person goes looking for them in the footage deliberately rather than waiting for enough to turn up.

An alert in LexAlert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live: a housing leaving station 6 with fewer than four screws or no earth strap, to the line lead, with the frame. The hold is the line's decision. The frame is the evidence.

Safety at the press brake is a zone and timer

The camera over the aisle by the press brake was installed for a near miss. The job now is a polygon drawn on its view around the brake's working area and a person class trained on the plant's own frames. The rule is a sentence: a person inside the polygon while the brake is cycling, or for longer than a walk past at any time. A nudged camera on a Sunday wash-down is what breaks it. Detection is the cheap part. The zone lives in the image, so a moved camera moves the zone without moving anything on the floor, and somebody checks the live frame against the polygon every Monday.

The alert goes to the shift supervisor with the frame, once per person, and the supervisor's verdict on each one, a real intrusion or a colleague reaching for a broom, is a correction the next version learns from.

Inventory is a count of cartons crossing a line

The packing line camera sees cartons pass a scanner on their way to the pallet. A carton class, a line drawn once across the belt, and the count per hour is a number the plant has until now taken from the scanner's log and the pallet count at the end of the shift, which disagree. The camera's count is a third opinion with a frame behind every disagreement, and a count that falls while the line is running is a jam upstream that the line lead hears about before the pallet is short.

My view is that a plant should start with the job that already has a person doing it badly at the end of a line. On this plant that is the press outfeed, where an inspector looks at a housing every few seconds for a whole shift and is worse at it by 2 pm than at 7 am. The camera does not tire, and the inspector's judgement is what trains it.

The four computer vision applications share one review queue

Four jobs on four cameras, each with its own rule, and underneath them one arrangement. The frames come from cameras the plant already owns. The labels come from the plant's own footage, checked by a person who knows the drawing. The alerts are sentences, with a severity and a place they land. And the frames each model doubts, the scratch that might be a reflection, the screw hidden under a wire, the person half behind the brake, the carton on its side, go to the same review queue, where a person's verdict is a label.

LexData takes each of the plant's models through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cameras the plant already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The two recorder boxes in the electrical room are still there. What sits beside them now is a runner, and what leaves the building is a handful of doubted frames a day.