Measuring Personalization, Embodiment, and Congruence in Online Learning: A Validation Study

PurposeInstructional designs promoting personalization (conversational style and visible author), embodiment (human-like mannerisms and presence), and social and cognitive congruence have been shown to enhance learning. However, measurements of these instructional design features are lacking. The authors sought to develop and validate an instrument to measure personalization, embodiment, and congruence in an online course, collecting evidence of content, internal structure, and relationships with other variables. MethodThrough literature review, expert review, and pilot testing, the authors developed a 15-item, 5-domain instrument. Participants completed this instrument along with assessments of motivation (Instructional Materials Motivation Survey and Motivated Strategies for Learning Questionnaire), knowledge, and course satisfaction immediately after a 30-minute online multimedia learning module comprising slides with voiceover and (for some participants) short video clips, from January to April 2021. Participants included pharmacists, physicians, and advanced practice providers at academic and community hospitals. Confirmatory factor analysis (CFA) was used to examine model fit, followed by structural revisions guided by exploratory factor analysis (EFA) and tested in a new CFA using a split-sample approach. The authors evaluated associations between the revised instrument scores and motivation, knowledge, and satisfaction. ResultsThere were 406 participants. CFA of the original 15-item instrument revealed poor model fit. EFA using 214 randomly selected participants suggested a 2-domain structure. CFA of the revised structure using the other 192 participants revealed good fit for a 9-item, 2-domain instrument: the Congruence-Personalization Questionnaire (CPQ). Linear regression confirmed positive, statistically significant associations for most hypothesized relationships (R-2 >= 0.12 and P < .001 for 14 of 18 associations with motivation and satisfaction). Associations with knowledge were weak and nonsignificant (R-2 <= 0.004, P > .23). ConclusionsEvidence of content, internal structure, and relationships with other variables support the validity of CPQ scores for measuring personalization and congruence in an online module.