Recent spectropolarimetric observations of Ap and Bp stars with improved sensitivity have suggested that most Ap and Bp stars are magnetic with dipolar fields of at least a few hundred gauss. These new estimates suggest that the range of magnetic fluxes found for the majority of magnetic white dwarfs is similar to that of main-sequence Ap-Bp stars, thus strengthening the empirical evidence for an evolutionary link between magnetism on the main sequence and magnetism in white dwarfs. We draw parallels between the magnetic white dwarfs and the magnetic neutron stars and argue that the observed range of magnetic fields in isolated neutron stars (B-p similar to 10(11)-10(15) G) could also be explained if their mainly O-type progenitors have effective dipolar fields in the range of a few gauss to a few kilogauss, assuming approximate magnetic flux conservation with the upper limit being consistent with the recent measurement of a field of B-p similar to 1100 G for theta Orion C. In the magnetic field-rotation diagram, the magnetic white dwarfs can be divided into three groups of different origin: a significant group of strongly magnetized slow rotators (P-rot similar to 50-100 yr) that have originated from single-star evolution, a group of strongly magnetized fast rotators (P-rot similar to 700 s), typified by EUVE J0317-853, that have originated from a merger, and a group of modest rotators (P-rot similar to hours-days) of mixed origin (single-star and CV-type binary evolution). We propose that the neutron stars may similarly divide into distinct classes at birth, and suggest that the magnetars may be the counterparts of the slowly rotating high-field magnetic white dwarfs.