Evaluating Test Suite Effectiveness and Assessing Student Code via Constraint Logic Programming

被引：3

作者：

Dewey, Kyle ^{[1
]}

Conrad, Phillip ^{[1
]}

Craig, Michelle ^{[2
]}

Morozova, Elena ^{[1
]}

机构：

[1] UC Santa Barbara, Santa Barbara, CA 93106 USA

[2] Univ Toronto, Toronto, ON, Canada

来源：

ITICSE'17: PROCEEDINGS OF THE 2017 ACM CONFERENCE ON INNOVATION AND TECHNOLOGY IN COMPUTER SCIENCE EDUCATION | 2017年

关键词：

D O I：

10.1145/3059009.3059051

中图分类号：

TP301 [理论、方法];

学科分类号：

081202 ;

摘要：

A good suite of test inputs is an indispensable tool both for manual and automated assessment of student submissions to programming assignments. Yet, without a way to evaluate our test suites, it is difficult to know how well we are doing, much less improve our practice. We present a technique for evaluating a hand-generated test suite by comparing its ability to find defects against that of a test suite generated automatically using Constraint Logic Programming (CLP). We describe our technique and present a case study using student submissions for an assignment from a second-year programming course. Our results show that a CLP-generated test suite was able to identify significant defects that the instructor-generated suite missed, despite having similar code coverage.

引用

页码：317 / 322

页数：6

共 20 条

[1]

[Anonymous], 2006, BUGS FOUND JSFUNFUZZ

[2]

[Anonymous], 2015, Proc. 46th ACM Tech. Symp. CSE, DOI 10.1145/2676723.2677226

[3] Fuzzing the Rust Typechecker Using CLP [J].

Dewey, Kyle ;

Roesch, Jared ;

Hardekopf, Ben .

2015 30TH IEEE/ACM INTERNATIONAL CONFERENCE ON AUTOMATED SOFTWARE ENGINEERING (ASE), 2015, :482-493

[4] Automated Data Structure Generation: Refuting Common Wisdom [J].

Dewey, Kyle ;

Nichols, Lawton ;

Hardekopf, Ben .

2015 IEEE/ACM 37TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, VOL 1, 2015, :32-43

[5]

Dewey Kyle, 2014, P 29 ACMIEEE INT C, P725

[6]

Diaz D., 2000, Proceedings of the 2000 ACM Symposium on Applied Computing - Volume 2, V2, P728, DOI DOI 10.1145/338407.338553

[7] Feedback Generation for Performance Problems in Introductory Programming Assignments [J].

Gulwani, Sumit ;

Radicek, Ivan ;

Zuleger, Florian .

22ND ACM SIGSOFT INTERNATIONAL SYMPOSIUM ON THE FOUNDATIONS OF SOFTWARE ENGINEERING (FSE 2014), 2014, :41-51

[8]

Holler C., 2012, P 21 USENIX SEC S, P445

[9]

Ihantola P., 2006, Proceedings of the 6th Baltic Sea Conference on Computing Education Research: Koli Calling 2006, P87, DOI [DOI 10.1145/1315803.1315819, 10.1145/1315803.1315819]

[10]

Ihantola Petri, 2010, Proceedings of the 10th Koli Calling International Conference on Computing Education Research, P86, DOI DOI 10.1145/1930464.1930480

← 1 2 →