Overdetection vs. Missed Detection — Which One Costs More?

Tune a defect-inspection algorithm long enough and you eventually hit a moment where you have to choose between two kinds of mistakes: filtering out a good part as defective (false positive), or missing an actual defect (false negative). Without explicitly addressing this trade-off, tuning drifts on forever.

The Two Kinds of Mistakes, Seen Through a Confusion Matrix

Lower the threshold and false negatives drop but false positives rise; raise the threshold and false positives drop but false negatives rise. Neither can be driven to zero — which mistake costs more becomes the criterion for setting the threshold.

Setting a Threshold That Accounts for Cost Asymmetry

For defects directly tied to safety or quality, the cost of a false negative is far higher, so the threshold is set conservatively (accepting more false positives). Conversely, for a cosmetically minor defect, the re-inspection cost from false positives can be the larger burden, which can justify loosening the threshold. A single metric like “accuracy X%” cannot capture this asymmetry.

How Semiconductor Inspection Equipment Solves This Problem

The balance between false positives and false negatives isn’t a concern unique to machine vision. The wafer-inspection-equipment industry has a dedicated term for it entirely: nuisance — something the inspection tool reported as a defect that isn’t actually a defect, in other words, a false positive. Looking at a patent (KR102203112B1) from KLA, the top wafer-inspection-equipment maker, filtering out nuisances appears as a standard component of the inspection recipe. The step after tuning the threshold is thinning out false positives with a classifier — this two-stage structure is the industry’s basic grammar.

The same patent points to a deeper problem as well. The question that comes before where you set the threshold is whether the distribution the threshold sits on top of stays stable in the first place. When process variation shifts an image’s brightness distribution lot to lot, a threshold that was optimal at recipe-setup time ends up misaligned at actual inspection time. This is how you get the phenomenon where the same recipe, inspecting product from the same process, catches zero defects in one lot and floods you with them in another. KLA’s solution was to fix the coordinate system of judgment using a defect-free standard reference image — using the median image of multiple normal samples, correcting the histogram against a reference distribution, or even generating a noise-free synthetic image from design data to serve as the reference.

It’s also worth noting that judgment isn’t left dependent on a single basis. The classification stage of this patent combines three things — properties of the captured image, properties of the difference image against the reference, and properties of the standard reference image itself — to decide defect versus nuisance. Translated into a field lesson: don’t fight the battle with a single threshold, and when judgments start wobbling, suspect the reference before the threshold.

Summary

Tuning an inspection algorithm isn’t about maximizing accuracy — it’s about first defining the cost of each of the two mistakes, then finding the point that fits.

Related Reading — Deep Learning vs. Rule-Based Algorithms: When to Use Which