Graph Construction and Visual Analysis: A Comparison of Curriculum-based Measurement Vendors

Curriculum-based measurement (CBM) represents a critical strategy for data-based decisionmaking within educational settings. Visual analysis is frequently used to analyze CBM data; thus, CBM vendors often automatically generate graphs based on student data to facilitate analysis. Differences in graph formatting are apparent across CBM vendors, and currently, the extent to which these differences may influence rater interpretation of data is unknown. As such, the current study sought to evaluate whether there were differences in rater evaluation of presence and magnitude of an intervention effect when identical data were plotted on graphs of four CBM vendors: AIMSweb, FastBridge Learning, easyCBM, and DIBELS Next. Results of the study indicated that visual analysts rated. Results of the study found that probability of identifying the presence of a treatment effect was similar across vendors. However, differences emerged with respect to ratings of magnitude of effect, with raters indicating the greatest magnitude of intervention effect for easyCBM and DIBELS Next graphs. As data graphs generated by CBM vendors may be utilized by school personnel to make decisions regarding the continuation, discontinuation, or alteration of intervention procedures, the results of the study indicate the need for increased consideration in the manner in which data are presented visually such that accurate decisions are made regarding student progress and response to intervention.