Graders as Meta-Reviewers: Simultaneously Scaling and Improving Expert Evaluation for Large Online Classrooms

被引：14

作者：

Joyner, David A. ^{[1
,2
]}

Ashby, Wade ^{[1
]}

Irish, Liam ^{[1
]}

Lam, Yeeling ^{[1
]}

Langston, Jacob ^{[1
]}

Lupiani, Isabel ^{[1
]}

Lustig, Mike ^{[1
]}

Pettoruto, Paige ^{[1
]}

Sheahen, Dana ^{[1
]}

Smiley, Angela ^{[1
]}

Bruckman, Amy ^{[1
]}

Goel, Ashok ^{[1
]}

机构：

[1] Georgia Inst Technol, Sch Interact Comp, Atlanta, GA 30332 USA

[2] Udacity Inc, Mountain View, CA 94040 USA

来源：

PROCEEDINGS OF THE THIRD (2016) ACM CONFERENCE ON LEARNING @ SCALE (L@S 2016) | 2016年

关键词：

Peer review; online education; PEER ASSESSMENT; HIGHER-EDUCATION; FEEDBACK; STUDENTS;

D O I：

10.1145/2876034.2876044

中图分类号：

TP39 [计算机的应用];

学科分类号：

081203 ; 0835 ;

摘要：

Large classes, both online and residential, typically demand many graders for evaluating students' written work. Some classes attempt to use autograding or peer grading, but these both present challenges to assigning grades at for-credit institutions, such as the difficulty of autograding to evaluate free-response answers and the lack of expert oversight in peer grading. In a large, online class at Georgia Tech in Summer 2015, we experimented with a new approach to grading: framing graders as meta-reviewers, charged with evaluating the original work in the context of peer reviews. To evaluate this approach, we conducted a pair of controlled experiments and a handful of qualitative analyses. We found that having access to peer reviews improves the perceived quality of feedback provided by graders without decreasing the graders' efficiency and with only a small influence on the grades assigned.

引用

页码：399 / 408

页数：10

共 28 条

[1]

[Anonymous], 2013, P 6 INT C ED DAT MIN

[2] Design-based research: Putting a stake in the ground [J].

Barab, S ;

Squire, K .

JOURNAL OF THE LEARNING SCIENCES, 2004, 13 (01) :1-14

[3]

Basu S., 2013, T ASS COMPUTATIONAL, V1, P391, DOI [DOI 10.1162/TACL_A_00236, DOI 10.1162/TACLA00236]

[4] Working inside the black box: Assessment for learning in the classroom [J].

Black, P ;

Harrison, C ;

Lee, C ;

Marshall, B ;

Wiliam, D .

PHI DELTA KAPPAN, 2004, 86 (01) :8-21

[5]

Boud D., 2001, Peer learning in higher education: Learning from and with each other

[6] ENHANCING AND UNDERMINING INTRINSIC MOTIVATION - THE EFFECTS OF TASK-INVOLVING AND EGO-INVOLVING EVALUATION ON INTEREST AND PERFORMANCE [J].

BUTLER, R .

BRITISH JOURNAL OF EDUCATIONAL PSYCHOLOGY, 1988, 58 :1-14

[7] STUDENT SELF-ASSESSMENT IN HIGHER-EDUCATION - A METAANALYSIS [J].

FALCHIKOV, N ;

BOUD, D .

REVIEW OF EDUCATIONAL RESEARCH, 1989, 59 (04) :395-430

[8]

Falchikov N, 2000, REV EDUC RES, V70, P287, DOI 10.2307/1170785

[9]

Georgia Tech Fact Book, 2014, GRAD ENR COLL ETHN G

[10]

Goel A., 2014, CS7637

← 1 2 3 →