Assessing the validity of indicators of the quality of maternal and newborn health care in Kenya

Background The measurement of progress in maternal and newborn health often relies on data provided by women in surveys on the quality of care they received. The majority of these indicators, however, including the widely tracked "skilled attendance at birth" indicator, have not been validated. We assess the validity of a large set of maternal and newborn health indicators that are included or have the potential to be included in population-based surveys. Methods We compare women's reports of care received during labor and delivery in two Kenyan hospitals prior to discharge against a reference standard of direct observations by a trained third party (n = 662). We assessed individual-level reporting accuracy by quantifying the area under the receiver operating curve (AUC) and estimated population-level accuracy using the inflation factor (IF) for each indicator with sufficient numbers for analysis. Findings Four of 41 indicators performed well on both validation criteria (AUC>0.70 and 0.75<IF<1.25). These were: main provider during delivery was a nurse/ midwife, a support companion was present at birth, cesarean operation, and low birthweight infant (<2500 g). Twenty-one indicators met acceptable levels for one criterion only (11 for AUC; 9 for IF). The skilled birth attendance indicator met the IF criterion only. Interpretation Few indicators met both validation criteria, partly because many routine care interventions almost always occurred, and there was insufficient variation for robust analysis. Validity is influenced by whether the woman had a cesarean section, and by question wording. Low validity is associated with indicators related to the timing or sequence of events. The validity of maternal and newborn quality of care indicators should be assessed in a range of settings to refine these findings.