Fall detection with pose estimation on the warehouse cameras a plant already has

At 2:10 am the parts warehouse behind the assembly hall has one picker in it. He is in aisle 14, between the racking, with a pallet truck and a pick list, and the nearest other person is at the dock office on the far side of the building. If he goes down here, the first person to find him is the forklift driver who starts at 6 am.

The camera at the end of aisle 14 has been recording all night. The frames are already there. The question is whether anything watches them, and what it would take for a watching model to tell a man who has collapsed from a man who has knelt to read a bin label.

Pose estimation turns each worker into a torso line the rule can watch

A person box tells the rule where somebody is. Pose estimation adds the keypoints, the shoulders and hips and knees, and the line from the midpoint of the shoulders to the midpoint of the hips is the torso. On a picker standing in aisle 14 that line is close to vertical and its top sits high in the frame. On the floor it is close to horizontal and low.

A box lies. A picker bending over a low shelf produces a short wide box that looks like a fallen person, while his torso line has merely tilted. The site safety use case makes the same point about helmets: a box in the frame is not a box on a body, and the keypoints are what attribute the geometry to the person.

The drop between samples is the signal, and a single frame never is

A frame with a horizontal torso in it is a person on the floor. It is also a person on a creeper under a conveyor, or a person who has lain down on a break because the floor is cool. The signal is the change: a torso line in aisle 14 that was upright in one sample and horizontal in the next, then horizontal again in the samples after that.

The cameras are sampled about every two seconds, and a fall takes less than that, so the rule never sees the falling. It sees standing, then down, then still down. That is enough, and it means the tracker has to hold the same identity across those samples, since a rule that cannot say the person on the floor is the person who was standing there will fire on a second worker who walked in.

How long down is the second part. Somebody who gets up within a couple of samples has tripped and does not want a paged supervisor. Somebody still horizontal a minute later needs one.

Kneeling and maintenance are false alarms that get labeled on purpose

The false alarms are the job. A picker kneeling to read the bottom bin, a fitter lying under the conveyor at the end of the aisle on a Tuesday night, a driver crouched beside a tyre: each is a torso line near horizontal and none is a fall. If the first version learned only standing and fallen, those are the frames it will send every night until somebody turns it off.

So they become classes. Kneeling, crouching and lying-for-work are labeled from the plant's own footage, Lexi proposes the keypoints and the class, and a person checks the labels before anything trains. The public fall footage most teams start from is staged on mats by volunteers. It is worth having in the set for the shape of a collapse. The frames that decide whether the plant keeps the alert on are the ones from aisle 14.

The most common thing lying on a warehouse floor at 2 am is a pallet strap. The first version usually boxes it as a person once.

Racking in the way is a maintenance ticket before it is a model problem

Aisle 14 is racked on both sides to the top beam. From the camera at the end, a worker three bays in is visible from the waist up when he stands and invisible when he goes down, because the lowest shelf and the stock on it are between the lens and the floor. The keypoints the rule needs most are the ones the racking hides first.

This is occlusion in the drift catalog's terms, and its fix is physical. A model that loses a track behind a stack of stock should report a lost track for that region of the frame, and the plant should read a lost-track rate rising in one bay as a stack that has been put somewhere new. No amount of retraining puts the floor back in view. The honest answer for a deep aisle is a second camera at the other end, and the honest answer for a bay the camera cannot see is to say so on the coverage map rather than to report it as safe.

The alert has to reach a person who can walk to the aisle

An alert is a rule written as a sentence, with a severity and a cooldown, approved before it goes live: a person down in any aisle for longer than the plant chose, critical, sent to the shift lead's phone and the security desk. The frame arrives with the torso line drawn on it and the aisle named, so the person reading it at 2:11 am knows whether to run or to radio.

The monitoring and alerts doc covers the delivery: Slack, email or a webhook into the plant's own system. With a runner beside the recorder the alert fires on site first, which matters at 2 am when the link to the cloud is the one thing nobody is watching. My own view is that a fall alert should have the shortest cooldown of any rule in the plant and should be allowed to be wrong. A supervisor woken twice a month for a fitter under a conveyor is a cost the plant can carry. The other error is the one it cannot.

The night shift is where the first version has the least evidence

Most of the footage the model trained on was recorded in daylight with the hall lights on and a dozen people in shot. The frames it will be judged on are dim, empty and sampled two seconds apart. Those are the frames it doubts, and they are the ones that come back for review.

LexData takes the fall model through its whole life. You type what to look for, Lexi puts the keypoints on every worker in every frame, and a person checks each label before anything trains on it. The model then watches the warehouse cameras the plant already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The corrections are also the monitor. When the night supervisor starts overriding the kneeling class in a consistent direction, or the lost-track rate climbs in one bay, that is the signal to look, and it arrives before any accuracy figure moves. A plant that reads the override rate every Monday learns about the new stack in bay 3 from the model rather than from the fall it hid.