What a manufacturing execution system records and what only a camera can confirm

At station 12 on the controller assembly line, the operator seats the cable loom, presses the confirm button on the terminal, and the manufacturing execution system records that step twenty-three of the work order is complete. The unit moves on. At final test, four stations later, the unit fails because the loom is seated in the housing and not connected to the board. The MES record for that unit says step twenty-three passed, and it is telling the truth about what it was told.

That gap, between a logged pass and a physically complete unit, is the one this post is about. The MES cannot see the loom. The camera over the fixture can.

The MES sits above the SCADA layer and below the business system

The usual way to draw the plant is in layers. At the bottom are the sensors and actuators on the machines. Above them the controllers. Above those the SCADA layer, which watches the process values and lets an operator act on them. Above that the MES, which turns the process into work orders, routes and steps, and records what happened to each unit. At the top the business system, which knows about orders, materials and money.

The MES is where the plant's memory of each unit lives. It schedules the work, tracks the unit through each station, records the quality checks, keeps the genealogy of which parts went into which serial number, and reports downtime and efficiency by line and shift. Everything the plant later needs to prove about a unit, it looks for in the MES.

That record is only as true as what the stations put into it.

A logged pass is the operator saying so

At most manual stations the MES learns a step is complete because a person pressed a button, scanned a barcode, or the fixture's sensor closed. Each of those is a proxy. The button says the operator believes the step is done. The barcode says the part was present. The sensor says something is in the fixture. None of them says the loom connector is on the header.

On the controller line the proxy fails in ordinary ways. The operator seats the loom, is interrupted by the material handler, comes back and presses confirm out of habit. A new operator on nights follows the work instruction on the screen and misreads the picture. A supplier's connector changes shape and no longer clicks. In each case the MES record is complete, and the unit is not, and the plant finds out at final test or at the customer.

The manufacturing page describes the wider picture, the lines where a person is the only check and the check gets worse as the shift goes on.

The camera over station 12 sees the cable that the MES cannot

A fixed camera over the fixture at station 12 sees the housing, the loom and the header. A model trained on frames from that camera, with the connected and unconnected states labeled, returns for each unit whether the connector is on the header. That is the assembly verification use case in its plainest form: is every part present, in the right place, the right way round. It is a completeness check, and the absence of a small part in a cluttered assembly is the signal, which is why the labeling has to include the frames where it is missing.

The model's verdict is a state per unit, connected or not, with the frame behind it. It runs on the frames the camera already records, on a runner beside the line's recorder. The accuracy the plant launched with is the one it keeps, because the frames the model is unsure of come back to a person and the corrections retrain it. In our manufacturing work that has meant 99%+ accuracy maintained in production, on lines where the first version went live in days.

The detection goes into the work order as evidence

The useful place for the camera's verdict is inside the MES record rather than beside it. When the operator presses confirm at station 12, the step is marked complete as before, and the camera's verdict for that unit is attached to the same step: connected, with the frame. When the verdict is not connected, the step is held, the terminal shows the frame, and the operator reseats the loom before the unit moves.

That changes what the MES record means. Step twenty-three now carries a fact rather than a proxy, and the genealogy for that serial number includes a picture of the loom on the header. At final test, a failure on the loom can be traced back to the frame. The answer is either that the camera was wrong, which is a correction that retrains the model, or that the loom came loose between stations, which is a different problem with a different fix.

An aside: the terminal at station 12 already had a picture on it, the work instruction's photo of the correct seating. Putting the camera's frame beside it was the change operators noticed most, because for the first time the screen showed their unit rather than the reference one.

Counts by station and shift come from the same detections

Once every unit at station 12 has a verdict, the verdicts add up. LexInsight answers the questions the quality manager asks at the Monday meeting, with the frames behind them: how many units were held at station 12 last week, on which shift, and whether the count changed after the new connector arrived. The MES has the counts of units and steps. The camera adds the counts of what was physically true at each step.

The same pattern extends along the line. A camera at the fastener station verifies that all six screws are present, a camera at the label station reads the serial number and checks it against the work order, and each verdict lands in the record for the unit. The MES stays the system of record, and the cameras become the witnesses it never had.

My own view is that the plants that get the most from this are the ones that do not build a separate quality dashboard for the camera. The verdict belongs in the work order, next to the button press it corrects, and anywhere else it becomes one more screen nobody opens.

The camera's verdict has to be checked the way the MES is audited

A camera verdict that is trusted blindly is the button press again, in a newer form. The check is the review queue: the frames the model was unsure of come back to a person on the line, and the person's corrections are the signal that something at station 12 has changed. A new connector, a moved camera after a fixture swap, a new operator whose hands cover the header at the moment the frame is taken.

When corrections rise, the model needs to see the new condition, and the corrected frames are already the training set for the next version. That version replaces the old one with no downtime, and the MES record for the units built that afternoon shows which version supplied the verdict, because a record that cannot say what checked it is a proxy again.