Various. intrusion tolerant certification authority (CA) systems have been recently proposed to provide attack resistant certificate update/query services. However, it is difficult to compare them against each other directly due to diversity in system organizations, threshold cryptography schemes, protocols and usage scenarios. We present a framework for intrusion tolerant CA system evaluation, which consists of three components, namely, an intrusion tolerant C,4 model, a threat model and a metric for comparative evaluation. The framework covers system organizations, protocols, usage scenarios, period of certificate validity, revocation rate and mean time to recovery (MTTR). Based on the framework, four representative CA systems. are evaluated and compared in three typical usage scenarios, producing reasonable and insightful results. The inter-dependency between usage scenarios and system characteristics is investigated, providing a guideline to design better systems for different usage scenarios. The proposed framework provides an effective method to evaluate intrusion tolerant C,4 systems quantitatively. Moreover, the comparison results offer valuable insights to further improve the attack resilience of intrusion tolerant CA systems.
