Can the impact of grey literature be assessed? An investigation of UK government publications cited by articles and books

Grey literature encompasses a range of relatively informal textual outputs that are not indexed in citation databases. Although they are usually ignored in research evaluations, it is important to develop methods to assess their impact so that their contributions can be recognised, and successful types of grey literature can be encouraged. This article investigates the extent to which 97,150 UK government publications were cited by Scopus articles and Google Books during 2013-2017 in eleven broad subject areas. A method was used to semi-automatically extract citations to the UK government publications from articles and books with high recall and precision. The results showed that Scopus citations are more common than Google Books citations to UK government publications, especially for older documents, and for those in the broad grey literature areas of Healthcare, Education and Science. Since the difference is not huge, both may provide useful grey literature impact data. Precision estimated in two separate ways for both methods appears to show acceptable levels of error for the automated method when compared to a sample with known citation counts.