Research Analytics

Research analytics can be used to evaluate research, whether it is the evaluation of the entire university or a smaller unit.

Responsible research analytics

The use of research analytics in various evaluation contexts has become more prevalent in the 21st century. The widespread availability and advancement of databases have made research metrics readily accessible, and their production is no longer as laborious, with an abundance of ready-made metrics available. The use of research metrics has been more cost-effective and time-efficient than conducting qualitative evaluations.

Over the past decade, several declarations and recommendations have been published aimed at promoting the responsible use of research metrics and responsible evaluation of researchers and research.

The Ģ��ֱ��, along with several other Finnish universities and research organizations, has signed the DORA Declaration and is committed to adhering to the national recommendation for responsible researcher evaluation.

Guidelines and declarations

Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland was published in 2020. These recommendations have been written for the context of individual researcher assessment, but the same principles are applicable to assessments of larger entities.

The overarching principles of the recommendation are:

transparency
integrity
equity
competence
diversity

The national recommendation provides guidelines to various topics in researcher assessment:

construction of the assessment procedure
assessing research outputs
identifying the diversity of research activities
researcher participations in the process.

Organizations committed to the recommendation are required to create policies to implement it, and monitoring procedures to follow realized outcomes.

The recommendation includes guidelines for responsible publication metrics. Publication metrics can be used to support assessment, but assessment should focus on qualitative evaluation by experts. It is important to ensure that the quantitative metrics are based on relevant data, and that both the methods and the results of analysis are scientifically sound and transparent.

DORA can be summarized in this simple central tenet:

Do not use journal-based metrics, such as Journal Impact Factors, as a surrogate measure of the quality of individual research articles, to assess an individual scientist’s contributions, or in hiring, promotion, or funding decisions.

It then further explains this, giving guidelines how research organizations, funders, individual researchers, publishers, and metrics providers should implement this principle.

Research organizations should be explicit about the criteria used to reach hiring, tenure, and promotion decisions, clearly highlighting that the scientific content of a paper is much more important than publication metrics or the identity of the journal in which it was published. Organizations should also consider the diversity of research outputs (including e.g. datasets and software), and the diversity of relevant impact measures, such as influence on policy and practice.

Individual researchers, when involved in committees making decisions about funding, hiring, tenure, or promotion, are reminded to make assessments based on scientific content rather than publication metrics. Shaping the academic culture is every individual's responsibility: citing primary literature rather than reviews in order to give credit where credit is due, using range of metrics and indicators beyond (and preferably instead of) those based on journal name and prestige in one's own impact statements and CV, challenging practices and narratives that rely inappropriately on Journal Impact Factors, and promoting and teaching best practice focusing on the value and influence of specific research outputs.

The Leiden Manifesto for research metrics consists of 10 principles to guide research evaluation:

Quantitative evaluation should support qualitative, expert assessment.
Measure performance against the research missions of the institution, group, or researcher.
Protect excellence in locally relevant research.
Keep data collection and analytical processes open, transparent, and simple.
Allow those evaluated to verify data and analysis.
Account for variation by field in publication and citation practices.
Base assessment of individual researchers on a qualitative judgement of their portfolio.
Avoid misplaced concreteness and false precision.
Recognize the systemic effects of assessment and indicators.
Scrutinize indicators regularly and update them.

is an independent review of the role of metrics in research assessment and management in United Kingdom. The core recommendations emerging from this are:

Robustness: basing metrics on the best possible data in terms of accuracy and scope
Humility: recognising that quantitative evaluation should support – but not supplant – qualitative, expert assessment
Transparency: keeping data collection and analytical processes open and transparent, so that those being evaluated can test and verify the results
Diversity: accounting for variation by field, and using a variety of indicators to support diversity across the research system
Reflexivity: recognising systemic and potential effects of indicators and updating them in response

Researcher evaluation shapes and directs research. The entire research community should take responsibility for the principles and practices of researcher evaluation.

Good practice in researcher evaluation. Recommendation for the responsible evaluation of a researcher in Finland, 2020.

Information sources and tools

introduces the main concepts of publication metrics and methods for analysing different subjects based on publications. The guide presents a wide range of different indicators, data sources and tools.

The guide also highlights the metrics that are most closely related to publication metrics from other perspectives. In addition, the responsible use of publication metrics plays a central role in the content of the guide, which, in addition to its own chapter, is embedded in other chapters of the guide as a perspective or as highlights.

In addition to discovery of research outputs by keyword search or other methods, the citation databases usually also feature some publication metrics. Records of individuals publications show how many times they have been cited, and sometimes also more refined citation metrics normalized by time and research filed. Often the databases also feature researcher profiles, which list all research outputs authored by a person, and some aggregate metrics for the person.

Ģ��ֱ�� has purchased full access to the following citation databases. Access these by navigating to the resource from within the university network or via VPN, or by logging in using your university credentials if the resource supports SSO (i.e. when you log in with your @jyu.fi email address, the page bounces you to JYU domain page to enter your username and password. Do not enter your password directly on any page outside jyu.fi -domain.)

is a citation database provided by Digital Science. In addition to research publications, Dimensions indexes grants and patents and links between all of these. The basic features are free, but Ģ��ֱ�� has also subscribed to Dimensions Analytics (log in using your JYU credentials), offering more tools to explore and analyze sets of entities.

Dimensions Analytics also allows JYU users machine access to the Dimensions data using , a very powerful and versatile tool for conducting publication metrics analyses at large scale. Access using the link above, from within JYU network or VPN, or by logging in at using you JYU email address.

is a citation database provided by Elsevier. Coverage across fields of research varies, being most comprehensive for natural sciences, technology and medical and health sciences. Scopus contains some analysis and visualization tools itself, for more see SciVal below. Access using the link above, from within JYU network or VPN.

is an analysis platform based on Scopus data. Ģ��ֱ�� has purchased a license to this, so you can access and use it from within the campus network or using VPN. Publication coverage starts from 1996. Note that there are large differences in coverage for different fields of research.

SciVal features analysing publications by organization, by country, by research topic or topic cluster, or by publication platform. You can also define your own units of analysis, for example groups of researchers or publications.

is a citation database provided by Clarivate Analytics. Overall WoS has more narrow coverage than Dimensions or Scopus, excluding a large number of research outputs which the other databases do index. Coverage across fields of research varies, being most comprehensive for natural sciences, technology and medical and health sciences. Access using the link above, from within JYU network or VPN.

is a relatively new data source linking research publications and researchers to citations and mentions in policy documents (e.g. government publications, intergovernmental and non-governmental organization reports, think-tank articles) around the world, in multiple languages. The database can be used to discover, analyze and understand the societal impact of research, beyond just citations in other academic articles.

Tools and services

Training and support available at Open Science Centre

Feel free to contact information specialists at Open Science Centre for help on using and understanding research analytics.

University course catalogue also includes general training offered by Open Science Centre covering some topics in responsible publication metrics. Contact us to arrange tailored lectures and hands-on workshops for your department or research group.

Open Science Centre provides customized support for finding candidates and assessing research merits of applicants to tenure-track positions. Please contact us in as early stage of your process as possible