The smart factory on the floor, where the camera closes the loop

On the machining floor at 9:40 on a Tuesday, the PLC on cell 7 knows the spindle load, the OPC UA server has published it to the historian, and the MES has the work order the part belongs to. Every number the stack holds is about what the machine did. None of them is about what the part looks like. Whether the bore edge on the part that came off the fixture at 9:40 has a chip in it is a fact the inspector will discover at the end of the shift, with a loupe, if that part happens to be in the sample.

The fourth industrial revolution, as the trade press numbers it, was supposed to connect everything. On most floors it connected the machines and left the parts to the inspector.

The camera is the sensor the stack never had

A pressure transducer, a thermocouple and an encoder each report one quantity, and the PLC has a tag for each. A camera over the fixture reports an image, which is not a quantity until a model turns it into one: a chip on the bore edge, a missing insert, a part seated the wrong way round. Once it does, the camera is a sensor like the others, with one difference. The transducer was calibrated at install and will drift in a way the maintenance schedule already covers. The model was calibrated on the frames it trained on, and nobody wrote it into the schedule.

That difference is the argument of this post. The stack on the floor at a manufacturing plant already has a place for a sensor that emits events. What it lacks is a routine for a sensor whose calibration is a set of labels.

A detection becomes an event the MES can file

The MES does not want a picture. It wants a record: part serial, station, timestamp, verdict, reason. The model over cell 7 produces a box with a class on a frame, and the layer between the two turns that into the record and posts it where the MES already listens.

On the floors we work with, that layer is a webhook. The alert is a rule written as a sentence, "a chip on the bore edge on cell 7", with a severity and a cooldown, approved before it goes live, and the delivery target is the endpoint the MES exposes rather than a person's inbox. The MES files the event against the work order, and the chip that used to surface at the end of the shift is on the record at 9:41 with the frame attached. The monitoring guide covers the rule, the severity and the endpoint.

The same event can go to the PLC as a tag, so the cell holds the part rather than passing it downstream. That is a controls decision the plant makes, and the camera's job ends at the record.

The rule fires on the floor before the historian sees it

The floor's network was designed for PLC traffic, and the plant's IT policy on footage leaving the building was written before anyone proposed streaming a cell camera to a cloud. Both are reasons the model runs on a runner beside the recorder. Frames stay on site, the rule fires locally first, and what leaves the building is the event and the frames the model doubted.

There is an operational reason too. A cell that waits on a round trip to decide whether to hold a part is a cell whose cycle time now depends on the network. A rule that fires beside the recorder does not.

Every other sensor gets recalibrated and the camera has to be too

Here is where the smart factory argument usually stops short. The camera goes in, the model reaches its accuracy on the launch frames, the events flow to the MES, and everyone moves on. Six months later a new insert supplier ships a slightly different grind, or the lighting over cell 7 is replaced with a cooler tube, and the model has been quietly wrong for a fortnight before an inspector notices the MES record and the part disagree.

LexData takes the model through its whole life on the assumption that this will happen. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cell cameras the plant already has, on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. That review loop is the calibration routine the transducer has and the model did not.

The signal that the routine is due is the correction rate: frames coming back for review, the inspector overriding the model's call in a consistent direction. When the corrections cross the project's threshold a new version is trained on the frames the inspector already judged. That is how the plants in our manufacturing work hold 99%+ accuracy maintained in production, and the word doing the work in that phrase is maintained.

My own view is that the camera is the only sensor on the floor that should ship with a review queue attached, and that a vision vendor who offers a model without one is selling a transducer with no calibration port.

Two streams on the same moment need a registration first

Once the visual camera is a sensor, the next request is usually a thermal one beside it, so the same event can carry a temperature. That is a fusion problem before it is a model problem. Two sensors have two frame rates and two fields of view, and the fusion depends on knowing which thermal frame belongs with which visual frame. The multi-sensor monitoring use case makes the point that a firmware update shifting a timestamp on one stream leaves both feeds looking perfect and the pairing wrong.

Rule that out on the Monday before retraining anything. A step change to the exact hour of an IT patch is a pipeline event, and the correction rate will show it as a cliff rather than a slope.

Start with the cell whose scrap the MES cannot explain

Every floor has one. The MES shows the work orders, the historian shows the spindle loads, and the scrap bin at the end of cell 7 fills faster than either can account for. That is where the camera earns its place in the stack: one cell, one class the inspector already names, one rule with one endpoint.

An aside from the tag database: the OPC UA node names on that floor still carry the initials of the engineer who commissioned the PLC back in 2009 and left the plant long ago. The alert sentence should use the words the current shift uses, because the alert will be read by them and not by him.

The camera goes in over cell 7. The event lands in the MES at 9:41. The inspector's corrections come back through the review queue, and in six months the model that watches cell 7 is not the model that shipped, which is the point of putting a sensor on a schedule.