Geometric Distortion Calibration using a pinhole mask

Adaptive optics (AO) systems deliver high-resolution images that may be ideal for precisely measuring positions of stars (i.e. astrometry) if the system has stable and well-calibrated geometric distortions. Calibration units equipped with back-illuminated pinhole masks are often utilized to measure instrumental distortions. AO systems on the largest ground-based telescopes, such as the W. M. Keck Observatory and the Thirty Meter Telescope, or with large fields of view, such as the 'imaka ground-layer adaptive optics experiment, require pinhole positions known to 20 nm to achieve astrometric precisions of 0.001 of a resolution element. We characterized the non linear geometric distortion of a simple imaging system using a photo-lithographic pinhole reference grid to be 1650 nm RMS with a final residual of 41 nm RMS (20.5 as for TMT). Our system model uses fourth order polynomials to model the distortion and allows the reference positions of the pinholes to vary. The nonlinear deviations in the pinhole pattern with respect to the manufacturing design of a square pattern are estimated to be 29 nm over a 1200 mm2, which reflects the additional error induced in a distortion measurement when assuming the pattern is perfectly manufactured.