By Tilo Mathes
Like for most people, numbers and metrics have played a major role in my career: I used numerical models and analyses to investigate hypotheses for my research, to report on the impact of my research in grant and academic job applications, and to help assess the value of products that I built as a product manager at ResearchGate.
Metrics are appealing to work with because they appear clean, objective, and comparable, and they can easily illustrate progress: you can crunch numbers without understanding every detail of what’s beneath, and you can work with them efficiently at scale. But simplicity often removes important context so that numbers and statistics can be easy to misunderstand and misinterpret if not communicated responsibly. In their own research, scientists are usually the first to point out potential faults with their methods and the need for further research to back up their conclusions.
It’s quite surprising, therefore, that the scientific community often relies strongly on metrics for the very difficult task of assessing scientific merit and impact — particularly when it comes to career-defining moments, such as grant applications and career progression. Because of that, it’s not so surprising, though, that such metrics and measures have become targets for researchers to work towards to succeed in their careers. And thus (hello, Goodhart's law!), unintended consequences and unhealthy behaviors have crept into scientific practice. We see this in “publish or perish” anxiety, “salami-slicing” and cherry-picking of results, publishing bias towards positive findings, p-hacking, insufficient documentation of methods that makes research difficult to replicate, and in funding streams that support safe bets in research over risky but innovative projects … just to name a few.
Several funders and community initiatives - DORA, Wellcome, Swiss Science foundation, GRC, Bibliometrics: The Leiden Manifesto for research metrics, just to name a few - have recognized the need for more scrutiny and color for research(er) assessment practices and even suggested excluding certain numerical indicators altogether. Simple indicators do allow for reducing the overhead of research assessment, which is also done to a large extent by researchers themselves and unfortunately not sufficiently acknowledged by funders and institutions. But the community pays a high price for this widespread failure to holistically assess research and scientific merit — both in terms of research integrity and diversity (or lack thereof) of science and scientists (The mental health of PhD researchers demands urgent attention, What researchers think about the culture they work in).
As a community platform, ResearchGate provides a lot of numbers and metrics as well, some of which are used in research impact assessment these days (e.g., citations, H-index, …). I share the mixed feelings our community has reported to us about some of those. Being an extremely well-adopted platform for scientists across the globe, ResearchGate clearly has an opportunity — and, I would argue, a responsibility — to incentivize good scientific practices.
I’m very happy that ResearchGate embraced a framework for establishing platform metrics for responsible use and is committed to painting a more detailed picture of a researcher’s contribution and the interest in their work (see update on the removal of the RG score). I’m convinced that taking advantage of the diversity of metrics that are out there, such as the ones provided through ResearchGate and other platforms and services, can indeed support drawing a more complete picture of and provide different perspectives on scientific merit and impact. I believe this is particularly relevant for earlier career researchers, who may want to try out new ways of scholarly communication, establish new ways to contribute to the science community, and we should make sure to foster their innovative spirit.
Generally, I also believe that there is an immense potential for researchers to utilize metrics to understand their own visibility within their community and beyond, learn how to create visibility for themselves, connect with the community, and get a well-deserved sense of recognition for their work. I also believe that, although one might argue that view metrics are vanity metrics, just being able to see that your work is actually read (Academics Write Papers Arguing Over How Many People Read (And Cite) Their Papers) and seeing that it’s being read by relevant colleagues may incentivise researchers to follow open science practices and make more of their research — including data, negative results, etc — available to the community. Moreover, we shouldn’t underestimate the positive impact on emotional health when seeing a rise in such an indicator, particularly considering that researchers essentially live in a world of continuous uncertainty about how their work is contributing to science plus the precarious job situation that most of them are in.
ResearchGate can do its best to provide helpful tools for researchers to become more efficient or successful and make sure to support good scientific practice. But in the end, it’s all about how we as a global community — and particularly funders and institutions — define scientific merit and impact and how it should be assessed in those career critical moments for members of the science community. Whether we want it to or not, the evaluation criteria we set will become a goal and influence the way researchers work. Let’s make sure these criteria support creating a healthy, diverse, equitable, and impactful science community.
