During recent years, swarm-based peer-to-peer streaming (SPS) mechanisms have become increasingly popular for scalable delivery of live streams over the Internet. The performance of SPS mechanisms depends on the overall effect of several factors including the connectivity of the overlay, the details of packet scheduling scheme and environment settings (e.g., peer and source bandwidth). Prior studies often presented overall performance of their proposed techniques in terms of delivered quality to all peers at a particular setting without demonstrating their inherent performance bottlenecks. Therefore, it is difficult to determine whether and how the reported performance of an SPS mechanism might change as a function of available resources or the connectivity of the overlay. In this paper, we present a simple yet effective methodology for performance evaluation of SPS mechanisms. Our methodology leverages an organized view of an overlay coupled with a two-phase notion of content delivery in SPS mechanisms to derive a set of metrics that collectively capture the behavior of each phase of content delivery. Therefore, the collection of our metrics can be viewed as the “signature of content delivery” of a given SPS mechanism. We also present the signature of a well-performing SPS mechanism that can be used as a reference for assessment of other mechanisms. To demonstrate the ability of our proposed evaluation methodology in identifying performance bottlenecks of SPS mechanisms and their underlying causes, we conduct two case studies: (1) assessing the performance of a set of candidate packet scheduling schemes; and (2) examining the effect of overlay localization on the performance of SPS mechanisms. In addition to illustrating the use of our methodology through examples, our case studies shed an insightful light on the performance bottlenecks in our target scenarios.
