The value and impact of Europe PMC were recently evaluated by the independent policy research and consulting organisation, Technopolis, to assess how the platform has been serving the research community. The evaluation was based on document review, surveys, user data and interviews.
Who uses Europe PMC, what for and why?
Europe PMC has a global reach with nearly 12 million unique devices accessing the site in 2018 alone. The users are predominantly researchers, students and health professionals, who mainly use Europe PMC as one of several tools to find scientific publications. In the surveys, 87% of users found the platform useful, the preferred features being ease of use, good coverage in terms of both content and volume, and free access.
What is the economic value of Europe PMC?
Although we know it is positive, it is difficult to put a quantitative value on the benefit of integrating several services in one place. Surveys showed that individual users were willing to pay on average about $28 per year for Europe PMC while librarians proposed to pay an average fee of around $1 per user per year. Monetising the time spent by users on Europe PMC has to account for the fact that many Europe PMC functionalities could be replicated by other services. This redundancy relates to the inherently ‘open’ nature of much of the data and content that Europe PMC is built on, which can be freely reused by other tools and services. The final calculation resulted in values between $6 and $31 per user per year (estimating between 95% and 75% of Europe PMC that could be replicated elsewhere). While these valuations do not represent a genuine market price and the approaches used may be subject to biases, they certainly suggest that Europe PMC presents significantly higher value to users relative to the moderate running cost, which is equivalent to about $0.33 per user per year. The value thus returned to the user community for every dollar invested in Europe PMC is shown in the figure below.
How does Europe PMC benefit the biomedical research community?
Europe PMC combines several functionalities under one umbrella, such as ORCID linking, a grant finder tool, data behind a paper, open citation counts, a manuscript submission system and a number of APIs and bulk download options to access the content. It has helped further the Open Science agenda by integrating freely available research content, from full text open access research articles to preprints, books, and clinical guidelines. Finally, it provides a mechanism for funders to more readily meet their open access commitments. As such, it could play an important role in the context of Plan S. An additional key benefit of Europe PMC is its autonomy, providing a level of immunity from potential political and commercial priorities. This enables the platform to be genuinely independent and respond quickly to user needs.
Loss of Europe PMC is likely to disproportionately impact under-resourced users (e.g. researchers in developing countries and citizen scientists) who largely rely on open access to keep abreast of the latest research findings. The 29 international funders who use the platform as their open access repository would also incur additional costs in locating and using alternatives. In addition, many third-party developers who use the Europe PMC APIs would have to substantially rebuild their applications, which would require additional time and money.
Take home message
Overall, Europe PMC delivers excellent value – in both monetary and non-monetary terms – to funders as well as the global research community. Loss of such a platform would be a blow to the emerging Open Science agenda which is gaining significant momentum in recent years.
The full report can be accessed here. Please send questions related to the evaluation process to the Open Research team at Wellcome at openresearch@wellcome.ac.uk.
Europe PubMed Central is indeed a very valuable component of the open science infrastructure. The quality and breadth of its data, and the comprehensive nature of its user help pages are among its most important features. The services we provide at OpenCitations are highly dependent upon it, and would be severely impacted were Europe PubMed Central suddenly to be closed. It provides excellent value for money, and the international funders that support it deserve the thanks of the entire scientific community.