Brief Intro to Responsible Metrics

1. What are ‘Responsible Metrics’?
Senior Manager: (Library) Research Services
https://tinyurl.com/DULC-Research

2. What do we mean by metrics?
When we talk about ‘metrics’ in this context, we are
usually referring to quantitative measure of research
‘impact’.

3. They might be used
to measure…

4. They might be used
to measure…
A research output
(article, book etc.)

5. They might be used
to measure…
An individual researcher’s
output

6. They might be used
to measure…
An organisational units research output (e.g. all
publications from a Research Group,
Department, Institute or the whole University.

7. Percentile Benchmark
Usually based on citation or
publication counts…

8. Percentile Benchmark
Usually based on citation or
publication counts…
A researcher may use them to help
make important decisions… such as
where to publish their work, or
who to hire or collaborate with.

9. Percentile Benchmark
Usually based on citation or
publication counts…
Or a publisher, funder or
institution may use them
to assess their research
output…
A researcher may use them to help
make important decisions… such as
where to publish their work, or
who to hire or collaborate with.

10. Several University Rankings
also include citation and
publication metrics as part
of their methodology.

11. What is the move to ‘responsible metrics’ about?
‘Responsible Metrics’ recognises that a quantitative
metric - a number - only reveal part of a wider story
of the impact research has made.

12. Many research metrics are
not consistent and should
not be used to compare
two articles, researchers or
institutions…
… but they often are.

13. Some research metrics are
susceptible to being
manipulated (e.g. through
self-citation)…
… and sometimes are.

14. Many metrics fail to account
for, or obscure, inherent bias
in the system.

15. Some metrics can be misinterpreted, can produce different results, and
may be unstable or inconsistent…
... but how they are used does not always take this into account.

16. A ‘Responsible Metrics’
approach recognises that
metrics are often used
inform important decisions…
… recognises the full picture
may not always be clear...
… and seeks to use the
appropriate metrics
alongside other measures.

17. To provide a more complete view
of the research impact.

18. What are a few quick tips to know?
There is lots of information out there, but there are
three key documents to be aware of…

19. The San Francisco Declaration on Research
Assessment (2012)
The Leiden Manifesto for research metrics
(2015)
The Metric Tide: Report of the Independent
Review of the Role of Metrics in Research
Assessment and Management (2015)

20. Whilst stated principles vary,
common themes are shared:

21. Whilst stated principles vary,
common themes are shared:
Recognise the value of both
qualitative and quantitative
measures for research
assessment.

22. Whilst stated principles vary,
common themes are shared:
Recognise the value of both
qualitative and quantitative
measures for research
assessment.
Ensure the accuracy and
appropriateness of data used
to calculate any metric.

23. Whilst stated principles vary,
common themes are shared:
Recognise the value of both
qualitative and quantitative
measures for research
assessment.
Ensure the accuracy and
appropriateness of data used
to calculate any metric.
Build in openness and
transparency of the metrics
used, so that they can be
checked and verified.

24. Whilst stated principles vary,
common themes are shared:
Recognise the value of both
qualitative and quantitative
measures for research
assessment.
Ensure the accuracy and
appropriateness of data used
to calculate any metric.
Build in openness and
transparency of the metrics
used, so that they can be
checked and verified.
Account for variation
including, but not limited to,
disciplinary field, career stage
and publication type.

25. Whilst stated principles vary,
common themes are shared:
Recognise the value of both
qualitative and quantitative
measures for research
assessment.
Ensure the accuracy and
appropriateness of data used
to calculate any metric.
Build in openness and
transparency of the metrics
used, so that they can be
checked and verified.
Account for variation
including, but not limited to,
disciplinary field, career stage
and publication type.
Respond to changes in
circumstance and purpose
and review and update use
over time.

26. What are ‘Responsible Metrics’?
Senior Manager: (Library) Research Services
https://tinyurl.com/DULC-Research