Detecting performance anomalies in large-scale software systems using entropy

Large-scale software systems (LSSs) are composed of hundreds of subsystems that interact with each other in an unforeseen and complex ways. The operators of these LSSs strictly monitor thousands of metrics (performance counters) to quickly identify performance anomalies before a catastrophe. The existing monitoring tools and methodologies have not kept in pace with the rapid growth and inherit complexity of these LSSs; hence are ineffective in assisting practitioners to effectively pinpoint performance anomalies. We propose two methodologies that use entropy measure to assist practitioners/operators of LSSs in quickly detecting both system-wide and underlying localized subsystem anomalies. Our performance tests conducted on an open-source benchmark system reveal that the proposed methodologies are robust in pinpointing anomalies, do not require any domain knowledge to operate, and avoid information overload on practitioners.