New footage lands in the bucket and shows up in the labeling queue

The three cameras over the packing stations write a clip to an S3 prefix every few minutes, and until March the way those clips reached the labeling queue was that someone downloaded Friday's folder, picked a few frames, and uploaded them by hand. The frames arrived a week late, the person doing it chose the tidy ones, and the crushed carton that went out on a Tuesday afternoon was never in the set.

The pipeline that replaces the Friday folder is short. An event on each new object, a function that forwards the frame, and a queue at the other end. What makes it work is less the code than the three decisions made before the first frame went through: what counts as new, what happens on a failure, and where the record of it all lives.

Each new object in the prefix raises an event

The bucket can be told to raise an event whenever an object is created under a prefix, and that event carries the object's key, its size and the time it landed. Scoping the event to the prefix the cameras write to, rather than the whole bucket, is the first decision, because the same bucket holds the exported models and last year's archive, and nobody wants a training export to trigger its own labeling job.

The event is the whole trigger. Nothing polls the bucket, nothing runs on a schedule, and a clip written at 2 am on a Sunday is handled at 2 am on a Sunday. The camera writes, the bucket notices, and the rest follows.

The event carries the key and the frame is forwarded to the queue

The function that receives the event does three small things. It fetches the object by key. It checks that the object is what it claims to be, a video clip or a frame of the expected shape, rather than a zero-byte placeholder the camera wrote when its card was full. It forwards the frame, or samples frames from the clip every couple of seconds, into the labeling queue, with the camera, the station and the time attached, for Lexi to draw the first boxes on.

The sampling is where the volume is decided. Three cameras writing clips all day is a lot of footage and very few distinct scenes, and a frame every couple of seconds from each clip is plenty. Nothing is re-encoded on the way; the frame that reaches the queue is the frame the camera wrote.

The annotation tool shows the frame minutes after the camera wrote it

At the other end the frame appears in the annotation tool as an item in the queue: the packing bench at station 2, boxes, labels, the crushed carton, with Lexi's first boxes already drawn, waiting for a person to check them. The gap between the camera writing the clip and the person seeing the frame is minutes, and the person sees Tuesday afternoon's crushed carton on Tuesday afternoon rather than next Friday.

The quickstart covers how the first labels come out of the platform once footage is in it. The bucket path is one of the ways footage arrives, alongside a shared drive and a plain upload, and it is the one that runs without a person remembering to do it.

The prefix, as it happens, was named by the installer after the packing hall's old door number, and everyone has stopped noticing.

A record of what entered and when is the pipeline's memory

Every frame that goes through gets a line in a record: the key, the camera, the time it landed, the time it was forwarded, and whether it went through or failed. That record is the answer to the question that arrives a month later, "why is there no frame from station 3 on the night of the fourteenth", and without it the answer is a guess. It also tells the team what the pipeline is actually carrying: how many frames a day, from which camera, at which hours, which is the first thing to check when the training set turns out to be all day shift.

My own view is that the record is the most valuable part of the pipeline and the part most often skipped. The function is an afternoon's work. The record is what makes the function trustworthy six months later.

Retries and duplicates are decided before the first frame arrives

Two decisions have to be made before the pipeline runs, because they cannot be made honestly afterwards. The first is what happens when the forwarding fails, because the queue was briefly unreachable at 3 am on a Sunday. Retrying is right, and retrying forever is wrong; a small number of attempts with a pause between them, and then a line in the record saying the frame was dropped, is the shape that survives.

The second is duplicates. Events can arrive twice for one object, and a camera that reconnects can rewrite a clip it already wrote. The function has to recognise a key it has already handled and do nothing, or the queue fills with the same frame and the person checking it labels the same crushed carton twice. A record of handled keys is the simplest version, and the same record answers the first question too.

The frames the production model doubts join the same queue

Once a model is trained and watching the packing stations, it produces a second stream into the same queue: the frames it is unsure of. A carton that might be crushed or might be in shadow, a label at an angle the model has not seen, a new box size from a new supplier. Those go to the same place as the bucket frames, with the model's own boxes drawn on them, and the person's corrections are what the next version trains on.

LexData takes the packing-station model through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cameras the hall already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. One queue, fed from the bucket at the start and from the model's doubts once it is live, is the shape the platform is built around, and it is why Friday's folder is no longer anyone's job.