Web search is frequently used by people to acquire new knowledge and to satisfy learning-related objectives, but little is known about how a user’s knowledge evolves through the course of a search session. This paper presents a study addressing the knowledge gain of users in informational search sessions. Using crowdsourcing, 500 distinct users were recruited and orchestrated real-world search sessions spanning 10 different topics and information needs. By using scientifically formulated knowledge tests the knowledge of users before and after their search sessions was calibrated, quantifying their knowledge gain. The impact of information needs on the search behavior and knowledge gain of users was investigated, revealing a significant effect of information need on user queries and navigational patterns, but no direct effect on the knowledge gain. Users on average exhibited a higher knowledge gain through search sessions pertaining to topics they were less familiar with. The findings in this paper contribute important groundwork towards advancing current research in understanding user knowledge gain through web search sessions.
While prior work has focused on improving the learning experience and efficiency during search sessions, the measurement of a user’s knowledge gain through the course of an informational search session has not yet been addressed. The importance of learning as an outcome of web search has been recognized. Yet, there is a lack of understanding of the impact of web search on a user’s knowledge state. This is a vital cog in the wheel, if web search engines that are currently optimized for relevance can be molded to serve learning outcomes.
Link to the pre-print: http://www.l3s.de/~gadiraju/publications/gadiraju_chiir2018.pdf.