Responsible Metrics

Research metrics are used in the assessment and evaluation of research across the sector, including for the purpose of external Institutional assessment (REF, league tables, funding agencies); to measure School performance (KPIs) and to evaluate individual performance (appointment, PDR, promotion).

The University uses quantitative and qualitative indicators to assess information about research activities (income, outputs, research degrees supervision) as well as research staff (recruitment, promotion and performance).

Edinburgh Napier University commits to the use of quantitative methods alongside qualitative indicators when assessing the quality of research, which includes appreciation by peers, impact, scale, originality, rigour and significance of the research.

DORA
The San Francisco Declaration on Research Assessment (DORA) was published in 2013 and intends to halt the practice of correlating the journal impact factor to the merits of a specific scientist's contributions as this practice creates biases and inaccuracies when appraising research. It also states that the impact factor is not to be used as a substitute to "measure of the quality of individual research articles, or in hiring, promotion, or funding decisions".

The University signed up to DORA in May 2018. This has influenced our statement on responsible metrics.

Timeline of DORA implementation:

Jan 2019: Leader of Research Information started putting a case together for the University Research and Innovation Committee (RIC) to approve the signing of DORA

May 2019: RIC approved signing up to DORA and implementing the requirements to meet DORA

June 2019: University signs up to DORA

July 2019: Blog post to researchers on signing up to DORA and responsible metrics
https://blogs.napier.ac.uk/rio/using-metrics-to-illustrate-research-impact

July-Sept 2019: Leader of Research Information and HR colleagues worked on how to include DORA into the Framework for… and the promotions and appointments process.

Consultations with senior staff to understand how research and researchers are currently being ‘assessed’ through promotion or appointments

Drafting of University responsible metrics statement

Nov 2019: Responsible metrics statement approved by RIC

Jan 2020: DORA requirements around research outputs and metrics included in the promotions round. This is communicated via the briefing information to applicants either in documents or discussion

Autumn 2020 – review of promotions applications to understand how responsible metrics was presented in promotions applications to provide better guidance for next years promotions round and to inform wider comms across the University.

Dec/Jan 2020 – Invitations across the schools to set up academic lead DORA/metrics working group

April – June 2021 – Monthly DORA working group meetings covering areas of concern for academic disciplines, career stages and feedback around Academic framework and promotions and appointment process

May 2021 – Briefing/training on responsible research metrics to HR colleagues

Sept 2021 – Feedback from DORA groups and suggested improvements to University academic assessment presented to University promotions lessons learned evaluation group

Nov 2021 – Ongoing - Academic Promotions : How to Stand Out workshops covering topics such as academic framework, responsible metrics and using narrative, mentoring and preparing to apply for promotion

Nov 2021 – Appointment of Phiona Stanley as Head of Research (Research Culture) who has joined the DORA working group.

Dec 2021 – Ongoing – Communication strategies across the University around research culture and responsible assessment