We determine the behavior of a time-varying fine structure "constant'' alpha(t) during the early and late phases of universes dominated by the kinetic energy of changing alpha(t), radiation, dust, curvature, and lambda, respectively. We show that after leaving an initial vacuum-dominated phase during which alpha increases, alpha remains constant in universes such as our own during the radiation era, and then increases slowly, proportional to a logarithm of cosmic time, during the dust era. If the universe becomes dominated by a negative curvature or a positive cosmological constant then alpha tends rapidly to a constant value. The effect of an early period of de Sitter or power-law inflation is to drive alpha to a constant value. Various cosmological consequences of these results are discussed with reference to recent observational studies of the value of alpha from quasar absorption spectra and to the existence of life in expanding universes.