In this lesson
Three questions, three different problems
“Is there a person in this frame” is a detection problem. “How many people crossed this line today” is a tracking problem wearing a detection costume. “Is this weld going to fail” is not a vision problem at all, it is a materials question that vision can only supply evidence toward.
The three need different data, different annotation, and different definitions of working. Teams routinely scope the first, budget for the first, and then discover in month four that the stakeholder wanted the third.
Vision is good at what is visible and consistent
A model learns the appearance of a thing across the variation you show it. It is strong when the target has a stable visual signature and the variation is something you can sample: lighting, angle, weather, wear.
It is weak when the distinguishing feature is not in the pixels. A cracked insulator and a shadow that looks like a crack are the same photograph. If a human expert needs to tap it, you cannot label your way out.
A useful test: can your best inspector make the call from the footage alone, with no site context? If not, neither can the model.
Can the camera even resolve it
Before any question about models, a question about optics: how many pixels does the target actually occupy? Detection benchmarks class anything under roughly thirty-two pixels a side as a small object, and accuracy on that class runs at a fraction of the headline number for every architecture ever tested. A defect that is eight pixels wide is not a modeling problem.
The fixes live in capture, not in training: fly lower, zoom tighter, mount closer, or tile the image and run detection on the pieces. For aerial work the arithmetic has a name - ground sample distance - and it sets a floor under everything downstream: you cannot measure what you cannot resolve, and no amount of labeling buys back pixels the sensor never captured.
Rare is harder than hard
Difficulty is not the enemy, rarity is. A model can learn a subtle defect if you can show it a few hundred examples. It cannot learn a defect you have seen four times, no matter how obvious each one was.
This is the single most common surprise in industrial vision. The failure you most want to catch is, by definition, the one that has almost never happened. Plan for how you will get examples: historical archives, staged captures, synthetic augmentation, or accepting that the first year is about collection.
Write the acceptance sentence first
Before any labeling, write one sentence: “This works if it finds X percent of Y, with no more than Z false alarms per shift.”
If that sentence is hard to write, the project is not ready. If it is easy, you have just specified your dataset, your annotation type, and your monitoring thresholds in one line. Most of the rest of this guide is downstream of that sentence.
The bar is not a perfect inspector
One number worth carrying into every scoping conversation: trained human inspectors miss roughly twenty to thirty percent of defects under sustained attention. That is not an insult to inspectors, it is the vigilance literature's most replicated finding, measured everywhere from precision parts to airport screening. Attention decays within the first half hour of watching, and it decays fastest when the thing you are watching for is rare.
So the honest question for a vision project is not “can the model match a perfect inspector” - no such inspector exists. It is whether the model, which does not tire and applies the same criteria at 3 a.m. as at 3 p.m., plus a human who only reviews what it flags, beats a human watching everything and fatiguing. On sustained monitoring of rare events, that pairing usually wins, and it is the fair comparison to write into the acceptance sentence.