Have you ever wondered if your research has influenced policies? Now, there is a free tool that allows you to check.
The Importance of Cited Research in Policy Reports
Cited research in policy reports can help scientists demonstrate their impact in promotions. The Sage Policy Profiles tool, a web-based tool launched by Sage Publishing, aims to empower researchers to showcase their influence or impact on policies when completing grant applications or during promotion and tenure interviews, according to its creators.
How the Sage Policy Profiles Tool Works
Launched on December 7, the tool allows any researcher to register and search for their name or unique ORCID ID. The tool then scans a database containing approximately 10.2 million policy documents – the largest database of its kind in the world – compiled by the UK startup Overton. The tool returns a summary of the policy documents that cite the researcher’s papers or mention their name, along with a map showing the source of the citations and when they occurred. Users can export this data into a spreadsheet or PowerPoint file.
Improving the Sage Policy Profiles Tool
Robin Haunschild, a bibliometrician at the Max Planck Institute for Solid State Research in Stuttgart, Germany, who tried out the Sage Policy Profiles tool at Nature’s request, said: “It’s a good first version of the tool. It’s a good service from Sage for the community. I think the tool should be improved over time to be more useful for users.”
Challenges and Limitations
Ioan Aide, founder and director of Overton, based in London, acknowledges that the company’s database mainly contains documents published in the last decade, relies on papers available online for free, and tends to have a Western bias. This means Overton may face gaps in the Global South, especially in countries where policy documents are published in local languages. Aide notes that about 60% of the policy reports in Overton’s database are in English.
A 2021 study found that only about 6% of the studies listed in the Scopus database between 2008 and 2016 were cited by policy documents in Overton’s database, but the percentage varies significantly between disciplines. Aide notes that just because a study is cited, it does not mean it has impacted policy – it may be mentioned for background information.
Overton also has a broad definition of what constitutes a policy document, including reports, guidelines, and working papers written for policymakers and published by sources such as governments, NGOs, intergovernmental organizations, and research centers.
Improving the Sage Policy Profiles Tool
Haunschild believes it will be difficult to use Sage’s results alone to demonstrate impact in interviews or grant applications. He says, “It could be helpful if Sage decided to include some context for these numbers, for example, how many policy citations occur on average for researchers in the same field?”
Aide states that the team chose not to create a single digital metric and settled on broader quantitative indicators because it is difficult to determine the significance of a particular paper to a piece of legislation it cites without more detailed investigation. “Things become very personal very quickly,” he says.
References
Yu, H. et al. Scientometrics 128, 6247–6266 (2023).
Haunschild, R. and Bornmann, L. Scientometrics 110, 1209–1216 (2017).
Pinheiro, H., Vignola-Gagné, E. & Campbell, D. Quant. Sci. Stud. 2, 616–642 (2021).
Leave a Reply