The performance of four passive optical network topologies in implementing multi-user quantum key distribution is compared. The networks considered are the passive-star network, the optical-ring network based on the Sagnac interferometer, the wavelength-routed network, and the wavelength-addressed bus network. An analysis of the quantum bit-error rate and sifted key rate for each of these topologies is used to determine their suitability for providing service to networks of various sizes.