Computer vision projects worth building on the cameras you already have

The packhouse camera over the sorting line was installed to settle disputes about who dropped a crate. In apple season it sees several thousand apples an hour pass under bright lights, and the grower has never asked it a question. The same is true of the camera over the assembly station at the plant, the one on the substation control building, the one on the ceiling of aisle 4 and the one over bay 6 at the warehouse. All of them were bought for a reason that has nothing to do with a model, and all of them are the right place for a first project.

The wrong first project is the ambitious one. The right one is scoped by four questions before anyone opens a labeling tool.

The computer vision projects that survive answer four questions first

What decision does the answer produce. A count that nobody acts on is a dashboard; a count that changes the harvest forecast is a project.

What footage exists. A camera that has recorded the scene for a year, at the hours the question matters, is a project that can start on Monday; a camera that has to be bought and mounted is a project that starts in a quarter.

How wrong may it be. A missed apple costs nothing; a missed person in a forklift lane is a near miss in the safety log, and the two projects are labeled and thresholded differently from the first day.

Who acts on the answer. If nobody's name is against the alert, the alert goes unread by Friday.

Each of the five projects below passes all four. Most of the projects we are asked about fail the fourth.

A plant can start with a missing component on the assembly station

The camera over the assembly station on line 2 sees one housing per frame with its fasteners and cable laid out, and the decision is whether the assembly is complete before it moves on. The footage exists, the tolerance for error is a rework rather than a recall, and the person who acts is the operator at the next station, who gets the frame with the missing fastener boxed. The assembly verification use case is this project on its own page. It is the shape of work behind the 99%+ accuracy maintained in production in our manufacturing cases, and it starts with a few hundred checked frames from that one camera.

A utility can start with a person inside the substation fence

The camera on the substation control building has watched the yard for years. The decision is whether someone is inside the fence outside working hours, the footage is a year of nights, the tolerance is low in one direction, a missed intruder, and generous in the other, a guard walking out to check a fox. The person who acts is the operator on the night desk, who gets the frame in Slack with the person boxed and the fence line drawn. The substation and equipment monitoring use case adds the thermal camera and the hot bushing to the same project.

The night desk's own name for the fox is on the class list as a class, so that its detections can be counted and ignored on purpose.

A grower can start with a count on the packhouse line

Back at the packhouse, the decision is the count: apples per hour per line, which feeds the harvest forecast and the labour plan for the week. The footage is a season of the sorting line under the same lights. The tolerance is generous, because a count a few percent out is still a better forecast than the one from the bin tally, and the person who acts is the packhouse manager planning the next day's shifts.

The counting is detection with a line drawn across the belt and an identity held on each apple across frames so that a stalled one is not counted twice. Our agriculture work, 80k+ image annotations in production, is mostly this kind of counting and the navigation that goes with it.

A store can start with the empty facing on the milk shelf

The ceiling camera over aisle 4 sees the dairy case. The decision is whether a facing is empty and for how long, and the footage is every trading day for a year. The tolerance is a wasted walk to the back if the model is wrong, and the person who acts is whoever has the morning list. The shelf and planogram compliance use case covers the harder version, position by position against a planogram grid. Its warning applies to the simple version too. A packaging refresh makes a full facing look empty to a model that has never seen the new pack, and staff overriding the alert is the signal that the model needs to see it.

A warehouse can start with a person in the forklift lane

The camera over bay 6 was fitted for insurance. The decision is whether a person is standing in the forklift lane while a forklift is moving, which is the near miss the safety log keeps recording. The footage is there, at every hour, and the tolerance is the strictest of the five: a miss is a near miss nobody saw, and a false alert is a supervisor who stops reading. The person who acts is the shift supervisor, on a phone, from the far side of the building. This project is labeled with a written rule for the box edge and a set of night frames collected on purpose, because the night shift is when the lane is worst lit.

The operational path is the question most projects skip

LexData takes each of these models through its whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the cameras you already have, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime.

The fourth question is about the last sentence of that paragraph. Who is the person the doubted frames come back to. On the packhouse line it is the manager for an hour on Friday; at the substation it is the night desk; at the warehouse it is the supervisor who already reads the alerts. A project with that name written down keeps improving, because the corrections keep landing. A project without it has a model that is exactly as good as the day it shipped, for as long as the site stays the same, which is never long.

My own view is that a first project should be small enough to be boring, and that a site which gets a boring camera right will trust the interesting one later. The full list of projects by industry, each with the label it needs and the way it drifts, is on the use cases pages.