The Fine Guidance Sensors of the Hubble Space Telescope were originally designed to serve as the ultimate stabilization and guidance component of the spacecraft's Pointing Control System. Partly because of these capabilities, their role was enlarged to serve as the principal astrometric instrument of the observatory. Extending engineering hardware and engineering level calibrations to garner the maximum scientific output is a nontrivial task. Indeed, the demands of a milliarcsecond level performance, the diversity of information and observable objects (position, brightness, color temperature, angular size, and limb darkening for single stars; the same plus magnitude difference, separation, position angle, and color temperature difference for double stars; as well as analogous attributes for nonstellar sources such as active galactic nuclei, quasars, supernovae remnants, multiple nuclei galaxies, and so on), and the lack of adequate calibration standards all serve to make the calibration of the Fine Guidance Sensors an extremely challenging project. This paper discusses some of these issues, extensions of the scientific use of the Fine Guidance System beyond traditional positional and double star astronomy, and develops the major scientific calibration procedures for positional, single star, and binary star astrometry.