Deploying computer vision models to edge devices beside the camera

The pick-and-place cell on the packing line has a light curtain, and the light curtain has a blind corner where an operator can reach in past it to clear a jam. The camera above cell 6 sees the hand. A model in the cloud would see it too, about a second later, which is a second after the arm has moved. The model has to live in the cabinet.

Forty miles away, a tractor is driving a row of Gala apple trees with a camera on the bonnet, and the orchard has no uplink at all. A model that needs the internet is a model that does not run.

Both of these are the same job: get the trained detector onto a small computer beside the camera, run it there, and keep it current when there is no network, or a slow one.

A YOLO detector exports by the device it will run on

The model that came out of training is a file, and the file has to be in a form the device can execute. Rather than asking a team to know which runtime a given board wants, export goes by device preset. Describe the device rather than the architecture, a Jetson or a Raspberry Pi or a GPU server, and the platform produces the model as PT, as ONNX or as TorchScript for it. The deployment guide lists the presets and what each one hands back.

Two things about that are worth knowing before reading a spec sheet. The export is never a vendor-specific accelerated format, and the labels that went into training can be taken out again as COCO, which is what a team keeps when it wants to train elsewhere. A YOLO model trained on the platform is still the team's model, on the team's device.

The engineer fitting the robot cell chose the Jetson preset on Tuesday, copied the file to the cabinet's computer, and had the model running against a recorded clip of the blind corner before the cell was powered on.

The robot cell cannot wait for a round trip

The cell's camera streams to a small recorder in the same cabinet. The runner sits beside it, pulls the stream, and samples frames for the model. When the model finds a hand inside the zone drawn around the blind corner, the alert fires locally first, into the cell controller's stop input, before anything is sent anywhere else. The frame with the hand boxed goes to the line lead's Slack a moment later, so someone knows why the arm stopped.

That ordering, local first, is the whole reason for the edge. Footage from the cell stays in the cabinet. Only the frames the model was unsure of leave the building, and those go back for a person to look at.

A zone drawn in image coordinates depends on the camera meaning today what it meant when the zone was drawn. The cabinet gets opened for maintenance, and the robotics page covers why perception failures in a cell are silent until they are not. The camera's bracket in this cell has a witness mark painted across it so a nudge shows.

The orchard has no uplink and the tractor still needs a model

The tractor's job is to keep to the lane between two rows of Gala trees, where satellite positioning drops out under the canopy. The model boxes the drivable path and the tree rows on every sampled frame from the bonnet camera, and the steering follows the box. It runs on a board behind the seat, exported with the same preset flow as the robot cell, and the board has no connection until the tractor is back in the shed.

So the orchard version of the loop runs on a delay. Through the day the board keeps the frames the model was unsure of, a low branch across the lane, a puddle that reads as path, the end of a row where the pattern breaks. In the shed at night the tractor is on the farm's wifi, the doubted frames go up for review, and the next morning's version, if the corrections crossed the threshold, comes back down the same way and replaces the old one before the tractor leaves.

The autonomous navigation use case is this problem in the platform's own words: the model is the fallback when positioning fails, so it is the thing that has to be right when nothing else is.

The runner is what makes the new version arrive

On a connected site, the runner beside the recorder is the mechanism for everything after export. It watches the streams, samples them, runs the model, fires alerts locally, sends doubted frames back, and when a new version of the model is ready, replaces the old one with no downtime. The robot cell's model has been retrained twice since the cell went live, both times on frames of gloves the first version had not seen, and the line never stopped for either.

LexData takes the cell model through its whole life. You type what to look for, Lexi puts a box on every hand and glove in every frame, and a person checks each label before anything trains on it. The model then watches the cell camera, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

Versions keep what they were trained on. If the third version turns out worse on the night shift than the second, the second is still there.

Power, heat and a full disk are the edge failures that matter

My own view is that the model is the least likely thing on an edge device to fail. The board in the cabinet gets warm in August and throttles, and the model slows down without anyone changing it. The board behind the tractor seat loses power when the ignition goes off, and if the model was not saved to disk it is gone. The disk that holds the doubted frames for the night sync fills up in a wet week when everything looks unfamiliar.

Each of those is a plain operations problem with a plain fix: a fan, a shutdown that saves state, a cap on the frames kept per day. None of them shows up in the accuracy figure, and all of them show up in the age of the newest frame the model processed, which is the one number worth watching on an edge device.

The orchard's tractor driver keeps a paper map of the rows in the cab with the bad patches circled in biro. When the model's doubted frames from a day are laid out, they are the circles.