Two stages to read a plate, and why each needs its own score

At 5:40 am on a wet Thursday a tipper truck stops at the yard gate, and the camera on the pole reads its plate as one character off. The barrier stays down, the driver leans on the horn, and the gatehouse lets him through by hand. Later that morning somebody asks the useful question: did the camera fail to find the plate, or find it and read it wrong? The gate log cannot say, because the gate log only holds the answer.

Reading a plate is two jobs. The first is finding the rectangle in the frame. The second is turning the pixels inside that rectangle into characters. They fail for different reasons, they are fixed with different frames, and a pipeline that reports one score for both is a pipeline that cannot be debugged.

An R-CNN style detector finds the plate before anything reads it

The first stage is a detector with one class, "plate". A two-stage R-CNN style model, which proposes regions and then classifies them, is a common choice here because plates are small, rectangular and few per frame, and the second stage's care with a tight box pays for itself when the crop goes to a reader. A single-pass detector works too, and the choice matters less than the frames it was trained on: this gate, this pole height, this angle, with the headlights on.

The detector is scored as a detector. On a held-out set of gate frames a person has boxed by hand, how many plates did it find, and how many of its boxes were on something else, a reflective sign, the back of a trailer, the "Stop here" notice on the barrier? Recall is the number to watch. A plate that was never boxed is a truck that was never read, and no reader can recover it.

The reader turns the crop into characters and is scored per plate

The second stage takes the crop and returns a string. It is scored two ways, and both matter. Per character: of all the characters on the held-out plates, how many came back right. Per plate: how many whole strings matched exactly. A reader that gets nineteen characters in twenty right sounds excellent and reads roughly one plate in three wrong, because a plate has seven characters and one miss spoils the string.

The per-plate figure is the one the gate experiences. The per-character figure is the one that tells the team what to fix. If the misses are all the same pairs, a zero and a letter O, an eight and a B, the reader needs more crops of those characters under this camera's light, and the fix is frames rather than a different reader.

Scoring the pipeline end to end hides which stage failed

If the only number kept is "plates read correctly at the gate", the tipper truck's wrong read is one tally mark, and the tally mark is silent about its cause. Keep three numbers instead: plates found, characters right, plates right, each on the same held-out set. Then the Thursday failure has an address. If the plate was found and the read was wrong, the reader needs rain frames. If the plate was never found, the detector needs frames of tippers at the angle they stop at, which is further from the pole than the cars.

On the sites we run, the gate camera's two stages are reviewed separately and the corrections go to whichever stage produced the miss. A correction to a box retrains the detector; a correction to a string retrains the reader; and a frame where both were wrong is rarer than people expect.

The gate has a sign asking drivers to stop with the plate under the light, and in the archive of a year of frames there is no evidence anyone has read it.

Night glare and rain are the frames to keep

The frames that break both stages are the same ones: headlights straight into the lens at 5 am, rain on the housing that turns the plate into a smear, low sun in the afternoon that bleaches the characters to nothing. The drift catalog covers weather as the condition the model rarely saw in training arriving for a week, and a gate camera lives that every autumn. A model trained on a dry summer's frames meets its first wet week as a stranger.

Those frames are kept. Every wrong read at night, every plate the detector missed in the rain, goes into the training set with the person's correct answer attached, and the version trained after the first wet week is the one that lasts through the second. Throwing away the hard frames because they are hard is how a gate model stays a summer model.

A read the reader is unsure of goes to a person

A wrong plate in the gate log is worse than no plate, because a wrong plate is a fact somebody will act on. The reader can say how sure it is of each character, and a string with a doubtful character should be held and sent to the gatehouse with the crop attached, so a person reads the two characters the reader could not. My own view is that a reader that says "unsure" on one plate in ten is worth more than one that is confidently wrong on one in thirty, and that the second kind is what most gates run.

The hazard zone intrusion use case is the other half of the same camera: the rule at the gate is ultimately about who and what is inside the yard, and a plate the gate is unsure of is a vehicle the yard is unsure of.

LexData takes both stages through their whole life. You type what to look for, Lexi puts a box on every frame, and a person checks each label before anything trains on it. The model then watches the gate camera the yard already has, in the cloud, on your servers, or on a runner beside the recorder. Frames it is unsure of come back to a person, the corrections retrain it, and the new version replaces the old one with no downtime. The wet Thursday at 5:40 am is a frame in the next version's training set, with the right plate written under it by the person who let the truck through.