The use of tags and tag clouds to discern credible content in online health message forums

Background: Web sites with health-oriented content are potentially harmful if inaccurate or inappropriate medical information is used to make health-related decisions. Checklists, rating systems and guidelines have been developed to help people determine what is credible, but recent Internet technologies emphasize applications that are collaborative in nature, including tags and tag clouds, where site users 'tag' or label online content, each using their own labelling system. Concepts such as the date, reference, author, testimonial and quotations are considered predictors of credible content. An understanding of these descriptive tools, how they relate to the depiction of credibility and how this relates to overall efforts to label data in relation to the semantic web has yet to emerge. Purpose: This study investigates how structured (pre-determined) and unstructured (user-generated) tags and tag clouds with a multiple word search feature are used by participants to assess credibility of messages posted in online message forums. The targeted respondents were those using web sites message forums for disease self-management. We also explored the relevancy of our findings to the labelling or indexing of data in the context of the semantic web. Method: Diabetes was chosen as the content area in this study, since (a) this is a condition with increasing prevalence and (b) diabetics have been shown to actively use the Internet to manage their condition. From January to March 2010 participants were recruited using purposive sampling techniques. A screening instrument was used to determine eligibility. The study consisted of a demographic and computer usage survey, a series of usability tests and an interview. We tested participants (N = 22) on two scenarios, each involving tasks that assessed their ability to tag content and search using a tag cloud that included six structured credibility terms (statistics, date, reference, author, testimonial and quotations). MORAE Usability software (version 3.1) was employed to record participants' use of the study environment. The surveys were analyzed using SPSS version 17. Interviews with participants were transcribed, coded and analyzed using thematic text analysis with the aid of NVivo8. Findings: Most participants had experience with Internet resources. However, less than one quarter of this sample had seen or used tags or a tag clouds. The ways in which participants used tags to label the content posted in the message forums varied. Some participants were tagging the information for their own subsequent use, whereas others viewed this process from the perspective of others: they tagged the content in ways that they thought other users would find beneficial. Many participants did not use the structured credibility tags when asked to search for credible content. The interviews corroborated these findings by confirming participants were not considering credibility foremost when tagging. Conclusion: Many participants in this study focused on assessing whether the information was relevant to their current circumstances, after which they would proceed to determine its credibility by corroborating with other sources. The use of structured tags to label information may not be a useful way to encourage the use of tagging, or to indicate credibility in this context. Current applications used in the semantic web automate this process. Therefore it may be useful to engage consumers of online content, in particular health-related content, to be more directly involved in the annotation of this content. (C) 2011 Elsevier Ireland Ltd. All rights reserved.