Machine guarding with computer vision, and the steel guard it does not replace

The press brake on the north bay starts at 6 am. The operator feeds a sheet from the front, the light curtain across the front is what stops the ram if a hand crosses it, and the fixed guard on the back gap is bolted on. Then the sheet catches on the back gauge, and the second operator, who has done this for eleven years, reaches round the side of the fixed guard to free it while the first one has a foot near the pedal.

The light curtain never saw the hand. It was not designed to. The camera on the column above the press did, and the question is what to do with that.

The steel guard stays, and the camera watches the gap it leaves

Fixed guards, interlocked guards, adjustable guards and presence-sensing devices exist because they work, and because a regulator can inspect them. A fixed guard is steel in the way; an interlock cuts power when the guard opens; a light curtain stops the ram when a beam is broken. Each covers a defined approach to the point of operation, and each has a gap it cannot cover. The reach round the side. The adjustable guard left open at the end of the night shift on Friday. The interlock defeated with a spare key so the changeover goes faster.

The camera covers approaches, which is the thing steel cannot do without blocking the work. It sees the reach round the side of the guard, the open adjustable guard, and the person standing where the light curtain does not reach. It does not stop the ram. On the platform the cameras are sampled about every two seconds, which is a fine rhythm for knowing the guard is being defeated and no rhythm at all for a machine stop, and the design has to be honest about that from the first drawing.

The point of operation is a polygon drawn once on the frame

The zone is the area a hand or a body should never be in while the press is live: the die, the back gauge, the strip between the guard and the frame where the reach-round happens. It is drawn on a frame from the column camera as a polygon, in image coordinates, once. The hazard zone intrusion use case is the same shape with a person box and a red zone on a well pad; here the box is a hand or a person, and the zone is the size of a die.

The model finds people and hands as two classes. You type them once, Lexi proposes the boxes on frames from that camera across a shift, and a person who runs the press checks them. The checking matters on the hand class, because a hand holding a sheet is a hand, and a labeler who boxes only bare hands has taught the model to miss every one that is working.

The test is whether any part of the hand box is inside the polygon, with no margin for perspective, because from a column camera looking straight down the die is where it looks like it is.

A person inside the polygon is a rule that warns, and can signal a supervisory stop

The alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live. A hand inside the point of operation zone on the north bay press is critical, and it goes to the shift supervisor in Slack and to the operator's own screen by the press. What arrives is the frame with the hand boxed and the polygon drawn, at the moment of the reach-round, so there is no argument later about what happened.

The same rule can fire a webhook to the machine's controller, and what the controller does with it is the plant's decision. On the lines we run, the sensible one is a supervisory hold: the press will not start its next cycle until a supervisor clears it. That is a delay, and a delay is the right shape for a signal that arrives on a two-second cadence. The light curtain remains the safety-rated stop. The camera is the thing that shows how often the light curtain is being worked around.

I think a camera should never be the device that stops a press, and I think any vendor who says otherwise has not stood next to one. The camera's job is to make the reach-round visible to the people who can redesign the job so it stops happening.

With a runner beside the recorder the alert fires on the plant network first, and the footage never leaves the bay. The deployment page covers where the model runs.

A wiped lens moves the zone, and no accuracy figure shows it

The failure to expect is the cleaning bump. The column camera gets wiped on a Friday, the housing is nudged a few degrees, and the polygon that covered the die on Monday now covers a strip of the bed beside it. The hand detector is unchanged and still finds every hand. The zone is in the wrong place, every alert is wrong, and the reach-round it was drawn to catch is now outside it.

The drift catalog calls this a camera moved, and for a zone in image coordinates it is the quietest failure on the list, because detection accuracy does not move at all. What moves is the correction rate: the supervisor starts dismissing alerts for hands that were on the bed, or the reach-rounds stop appearing when everyone knows they still happen. Either way a person is overriding the rule in a consistent direction, and that is the signal.

The check is cheap. Once a week someone puts a gloved hand at each corner of the die with the press locked out and confirms the box lands inside the polygon. The press shop in this post does it on the Monday morning before the first cycle, in the same book where the light curtain test is signed.

The corrections keep the hand detector honest across shifts

LexData takes the guarding model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the column camera the bay already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The gloved hands from the night shift, which the first version doubted, are what the second version learned from.

The foot pedal on that press has a cover over it so a dropped spanner cannot cycle the ram. Nobody has designed the cover for the reach-round yet, and until they do, the camera is the record of how often it happens.