Extracting and analyzing time-series HCI data from screen-captured task videos

被引:20
作者
Bao, Lingfeng [1 ]
Li, Jing [2 ]
Xing, Zhenchang [3 ]
Wang, Xinyu [1 ]
Xia, Xin [1 ]
Zhou, Bo [1 ]
机构
[1] Zhejiang Univ, Coll Comp Sci, Hangzhou, Zhejiang, Peoples R China
[2] Nanyang Technol Univ, Sch Comp Engn, Comp Sci, Singapore, Singapore
[3] Nanyang Technol Univ, Sch Comp Engn, Singapore, Singapore
关键词
Screen-captured video; Video scraping; HCI data; Online search behavior; INFORMATION;
D O I
10.1007/s10664-015-9417-1
中图分类号
TP31 [计算机软件];
学科分类号
081202 ; 0835 ;
摘要
Recent years have witnessed the increasing emphasis on human aspects in software engineering research and practices. Our survey of existing studies on human aspects in software engineering shows that screen-captured videos have been widely used to record developers' behavior and study software engineering practices. The screen-captured videos provide direct information about which software tools the developers interact with and which content they access or generate during the task. Such Human-Computer Interaction (HCI) data can help researchers and practitioners understand and improve software engineering practices from human perspective. However, extracting time-series HCI data from screen-captured task videos requires manual transcribing and coding of videos, which is tedious and error-prone. In this paper we report a formative study to understand the challenges in manually transcribing screen-captured videos into time-series HCI data. We then present a computer-vision based video scraping technique to automatically extract time-series HCI data from screen-captured videos. We also present a case study of our scvRipper tool that implements the video scraping technique using 29-hours of task videos of 20 developers in two development tasks. The case study not only evaluates the runtime performance and robustness of the tool, but also performs a detailed quantitative analysis of the tool's ability to extract time-series HCI data from screen-captured task videos. We also study the developer's micro-level behavior patterns in software development from the quantitative analysis.
引用
收藏
页码:134 / 174
页数:41
相关论文
共 80 条
[1]  
Ammar N., 2012, 2012 19th Working Conference on Reverse Engineering (WCRE), P367, DOI 10.1109/WCRE.2012.46
[2]  
[Anonymous], P 9 USENIX S OP SYST
[3]  
[Anonymous], 2005, WWW '05
[4]  
[Anonymous], 2012, P SIGCHI C HUMAN FAC, DOI [DOI 10.1145/2207676.2208311, 10.1145/2207676.2208311]
[5]  
Bandura A., 1986, Social foundations of thought and action: A social cognitive theory. PrenticeHall series in social learning theory, V1, P617
[6]  
Banovic N, 2012, UIST'12: PROCEEDINGS OF THE 25TH ANNUAL ACM SYMPOSIUM ON USER INTERFACE SOFTWARE AND TECHNOLOGY, P83
[7]  
Bao L, 2015, P 30 IEEE ACM INT C
[8]   scvRipper: Video Scraping Tool for Modeling Developers' Behavior Using Interaction Data [J].
Bao, Lingfeng ;
Li, Jing ;
Xing, Zhenchang ;
Wang, Xinyu ;
Zhou, Bo .
2015 IEEE/ACM 37TH IEEE INTERNATIONAL CONFERENCE ON SOFTWARE ENGINEERING, VOL 2, 2015, :673-676
[9]  
Bao LF, 2015, 2015 22ND INTERNATIONAL CONFERENCE ON SOFTWARE ANALYSIS, EVOLUTION, AND REENGINEERING (SANER), P399, DOI 10.1109/SANER.2015.7081850
[10]   The origin of bursts and heavy tails in human dynamics [J].
Barabási, AL .
NATURE, 2005, 435 (7039) :207-211