Analyzing Knowledge Gain of Users in Informational Search Sessions on the Web

Web search is frequently used by people to acquire new knowledge and to satisfy learning-related objectives, but little is known about how a user's knowledge evolves through the course of a search session. We present a study addressing the knowledge gain of users in informational search sessions. Using crowdsourcing, we recruited 500 distinct users and orchestrated real-world search sessions spanning 10 different topics and information needs. By using scientifically formulated knowledge tests we calibrated the knowledge of users before and after their search sessions, quantifying their knowledge gain. We investigated the impact of information needs on the search behavior and knowledge gain of users, revealing a significant effect of information need on user queries and navigational patterns, but no direct effect on the knowledge gain. Users on average exhibited a higher knowledge gain through search sessions pertaining to topics they were less familiar with. Our findings in this paper contribute important ground work towards advancing current research in understanding user knowledge gain through web search sessions.