Optical surface inspection methods gain increasing importance since such inspection systems operate without contact and offer various possibilities for online defect detection and classification. In order to quantitatively evaluate different systems used for on-site inspections, we have performed an (a) over cap versus a-analysis to determine the Probability of Detection ( POD) for planar specimens with boreholes and notches of different dimensions and orientations. Following the international standard according to MIL-HDBK-1823 in the 2007 update version we have determined POD-curves using the digitized and algorithmically processed data. We show that the application of more complex sensor technology and algorithms correspondingly leads to improvement of the POD.