Overlay networks are becoming widely used for delivering content, since they provide effective and reliable services that are not otherwise available. However, overlay management systems face the challenges of increased complexity and heterogeneity due to the numerous entities that are involved in realizing overlay services. We believe that autonomic management is a key solution for dealing with the complexity of overlay management. In this paper, a management architecture for service specific overlay networks is proposed. Overlays are viewed as a dynamic organization for self-management in which self-interested nodes can join or leave according to their goals. The objective of this architecture is to create autonomic overlays that are driven by different levels of policies. Policies are generated at different levels of the autonomic management hierarchy and enforced on the fly. The proposed autonomic management dynamically adapts the behavior of the overlay network to the preferences of the user, network, and service providers. A description of our novel architecture that addresses these challenges is presented.