Self-driving car perception and the white trailer against a bright sky

At 2 pm on an overcast afternoon a forward camera on a test vehicle is looking down a two-lane state road. Ahead, a white box trailer is crossing from a side road, side-on, its flat side filling the lane. The sky behind it is the same washed-out white. On the frame, the trailer's side and the sky meet without an edge, and the only things the model boxes are the trailer's wheels, low down and dark, which it calls a vehicle far ahead in the lane rather than a wall across it.

The frame is not unusual to a person. A person sees a trailer. The model has never been shown one like this, and that is the whole of the problem.

Object detection finds what the labels showed it

The perception model on that road at 2 pm is not reasoning about trailers. It is matching the frame in front of it against everything its labels showed it. Those labels came from a fleet driving highways and city streets where the vehicles ahead are seen from behind, at a distance, with a dark road under them and a lighter sky above. Object detection on that footage learns that a vehicle is a dark rectangle on a grey road. A white wall from lane edge to lane edge, with sky above and behind, is not in the evidence.

That is not a failure of the model. It is a description of the training set, and the training set is the road the fleet has driven, labeled by people who boxed what they saw. The trailer is rare on that road. The trailer side-on is rarer. The trailer side-on under a sky it matches is rare enough that a fleet can drive for a year and label none.

The robotics work we do lives in that gap. Every autonomous system meets a set of edge cases in the physical world that its training set did not contain, and the question is not whether the set is complete, since it never is, but how quickly it catches up.

The miss looks like nothing on the frame

The dangerous part of the 2 pm trailer frame is how ordinary it looks in the model's output. There is a box on something far ahead. There is no warning, no flag, no drop in any metric a dashboard would show, because a model that has never seen a thing does not doubt it. It sees sky.

The pedestrian and obstacle detection use case says what the standard is for this class of miss. It is the safety-critical class, the acceptable miss rate is effectively zero, and it has to hold for things that are partly hidden, at the edge of the frame, or, in this case, the same colour as the sky. A vehicle whose perception has a hole shaped like a white trailer will drive into it with the same confidence it drives past a parked car.

A second sensor sees the trailer. Radar returns a solid object across the lane where the camera returns sky, and the disagreement between the two is the first honest signal that the camera's model has a hole. On the vehicles we work with, that disagreement is a trigger: the frame is saved and goes to a person.

Doubted frames from the fleet are how the training set catches up

A fleet is a machine for producing rare frames. Fifty vehicles driving for a month meet the trailer, the low sun, the flooded underpass and the cyclist with a mattress on the rack that one vehicle meets once a year. The mechanism that turns those meetings into labels is the loop: the frames the model is unsure of, and the frames a second sensor disagrees with, come back to a person, and the corrections go into the next version.

LexData takes the perception model through its whole life. You type what to look for, Lexi puts a box on every vehicle, pedestrian and obstacle in every frame, and a person checks each label before anything trains on it. The model then watches the forward camera, on the vehicle, or on your servers when the footage comes back from the fleet. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The white trailer, once labeled, is in the version the fleet runs next month.

In our robotics work the figure we hold to is 95%+ navigation reliability in changing environments, and it is held by that mechanism rather than by any single training run. The road changes. The set catches up.

The test driver on the state road keeps a notebook titled "things the car did not like", with a date and a road for each entry. It is the best list of scenes to go looking for in the fleet's footage, and a labeler with that notebook finds the trailers faster than any sampling scheme.

The frames that are not doubted are the harder problem

A doubted frame comes back on its own. A confident miss does not, and the trailer at 2 pm was a confident miss. That is the part of the loop that needs a person choosing where to look rather than waiting for the model to ask.

My own view is that a perception team should spend more of its review time on frames the model was sure about than on frames it doubted. The doubted ones are already coming back. The sure ones are where the trailer lives, and the way to find them is to sample by scene rather than by score: side roads at junctions, overcast afternoons, white vehicles, and every frame where a second sensor and the camera disagreed. A person looks at a few hundred of those and finds the holes the model does not know it has.

Weather is the drift that comes back every winter

The trailer is a training set hole. The washed-out sky is weather, and weather comes back on a schedule. The drift catalog files it as rain, fog and dust: conditions the model rarely saw in training arrive for a week, detection falls off, then recovers. On a fleet the first overcast week of autumn produces a batch of doubted frames, and the same thing happens again with the first low sun of winter, when the camera is looking straight into it at 4 pm.

Those batches are the calendar for review. A perception model that was trained on a bright summer and is doubting in November is not broken; it is telling the team where the next labels come from. The corrections at review are what the next version learns from, and a fleet that reviews its November frames in November runs a model in December that has seen the sky.