Skip to main content

Researcher Support Services: Research evaluation

Citation analyses

Citation analysis is a method for measuring the impact and visibility of research. Citations can be considered a kind of a peer review. Using citations as an indicator for impact means evaluating how much other researchers use the research results produced in the unit being surveyed, as a source in their own publications.

In a citation analysis, the impact of a unit's research can be compared to the international level on each field of science.

In a similar fashion, the following things can be found out

  • cooperation and networking: Which researchers/organizations/countries are publishing together?
  • scientific publishing practices: Which researchers and which organizations cite which and whom and when?

The productivity and quality of research

Publication activity analysis methods can be used to examine the productivity of a research (the amount of publications) and, when combined with e.g. Publication Forum level, quality can also be measured. The number of publications can also be examined by type, on a national/domestic scale, input-output-analysis etc.

At TUT, the publication activity analyses have been based primarily on determining the productivity of publication activity, i.e. institution and researcher based statistics have been based on the amount of publications. TUT library produces statistics of publication activities for different needs. The completion of Publication Forum in 2012 has brought a possibility to attach a qualitative dimension to quantitative numbers. These methods can not be used to compare the quality of TUT research to the international level on each field of science.

Publication forum

Quality classification for scientific journals, conference and book series and publishers. The goal of the publication forum is to cover the essential scientific publication channels of every field of research and to mark those of highest quality. The publication channel classification is a useful tool for evaluating research when handling large publication numbers, not when comparing fields of science or individual researchers or research groups.

Bibliometric visualization tools

Publication practices, co-authors and reference relations, along with operators in the field, fields of science, organizations and networks can also be presented through various graphs and visualizations. Citation databases Web of Science (WoS) and Scopus provide tools for analyzing and visualizing search results. Other databases also have these tools available.

There are also other web tools that are free to use, which can be used to visualize references. These tools include VOSviewer, Network WorkbenchGephi and Pajek.

You can use for example WoS InCites as an analysis tool.

What is bibliometrics used for?

How is your research being evaluated?

Publications are an essential part of scientific communication and important products of research. Bibliometrics means statistical analysis of publication activities. Bibliometric analysis can be used as one method when determining the productivity and quality of a unit's research output, in addition to it's impact and visibility. Citation analysis can be used to compare a unit's research in comparison to the international level in each field of science.

Source: Priem, J.; Taraborelli, D.; Groth, P.; Neylon, C. Altmetrics: a manifesto. 8.8.2013.  http://altmetrics.org/manifesto/

When articles were only published in printed journals, they gained visibility mostly through the journal in which they were published. But nowadays, because research results are published online, they spread through different web services and social media tools and the articles are evaluated individually. Some publishers (including PLOS One, Nature, Wiley-Blackwell, Biomed Central), in conjunction with the articles, publish information on their webpage about the amount of downloads, references, how many times the article has been saved and conversations (viewed/cited/saved/discussed) on social media. These article-level metrics are called altmetrics.

Bibliometrics can be used when evaluating publications, researchers or organizations. A researcher's online visibility can be evaluated with altmetrics.

Altmetrics

The visibility and impact of publication activities on web services and social media has been started to be tracked with a new metric called altmetrics. It focuses on article level evaluation and represents the article's impact after its publication. There are a few service providers that offer article level analysis services (such as Impactstory, Altmetric, PlumX).

Bibliometrics

Impact Factor

CC BY John R. McKiernan, Why open research?

Bibliometric indicators

In addition to the number of publications and citations, there are numerous other bibliometric indicators that can be used to evaluate publication activity.

H-index

Can be calculated straight from the publication catalog, when the number of citations for articles is known. Available in international citation databases (WoS and Scopus).

Researcher's H-index is value H, if the researcher has H number of articles, that have been cited at least H times.

Along with H-index, it is necessary to present the amount of citations, for example the total amount or citations per publication.

Hirsch, J. E. (2005). "An index to quantify an individual's scientific research output". PNAS 102 (46): 16569–16572. doi:10.1073/pnas.0507655102.

Tutorials on how to define H-index in WoS and Scopus.

Impact Factor (IF)

Impact factor is used to evaluate the popularity and esteem of journal. It expresses the average amount of citations in published articles. IF values are related to the WoS -database and they are calculated by finding the number of citations accumulated by the journal's articles within a given time frame and dividing it with the amount of articles published in the journal within the same time frame. Journal Citation Report -journal catalog includes impact factors of 2 and 5 years.

Source Normalised Impact per Paper (SNIP)

When calculating the indicator, original articles, review articles and conference articles are taken into account, unlike in calculating JIF, where all publication types are counted. Different citation practices in different fields of science have been taken into account in the indicator calculation and it can be used when comparing different fields, unlike JIF values. When choosing a suitable publication channel, in addition to JIF, you can also use the Scopus journal catalog and Leiden university CWTS journal indicators. Both are based on information from Scopus.