Georgia CTSA's Publication Impact Factor: Research that Reaches
For ten years, the Georgia Clinical & Translational Science Alliance (Georgia CTSA) served as Atlanta’s home for clinical and translational research as a city-wide partnership between Emory University, Morehouse School of Medicine, and Georgia Institute of Technology (under the name Atlanta Clinical & Translational Science Institute [ACTSI]) and is one of a national consortium striving to improve the way biomedical research is conducted across the country. The consortium, funded through the National Center for Advancing Translational Sciences as one of the National Institutes of Health’s Clinical and Translational Science Awards (CTSA), shares a common vision to translate laboratory discoveries into treatments for patients, engage communities in clinical research efforts, and train the next generation of clinical investigators.
An important way of illustrating impact and progress is to measure the publication output resulting from Georgia CTSA resources. Since 2007, Georgia CTSA has supported the science behind nearly 2,400 projects, leading to almost 2,300 peer-reviewed publications that have been cited almost 74,000 times. Supported publications span a wide breadth of journals and scientific disciplines, reaching audiences from basic scientists, to medical professionals, to community health researchers. Georgia CTSA’s Evaluation & Continuous Improvement program undertook a longitudinal analysis of these publications with the goal of characterizing the progress and reach of both the Georgia CTSA and the national CTSA program’s research productivity and influence.
Nicole Llewellyn, PhD, manager of Evaluation & Continuous Improvement, with the support of Emory’s Life Sciences Informationist Kimberly Powell, and public health intern Jamie Adachi, used Web of Science (WoS) to measure national CTSA’s publication impact, including a longitudinal and comparative analysis. The team assessed the WoS Category Normalized Citation Impact (CNCI) scores for all Georgia CTSA-supported publications. The CNCI is an index of citation impact, adjusted for publication year and research category. Georgia CTSA’s average CNCI score is 2.35, meaning that Georgia CTSA publications are cited 2.35 times as often as comparable papers from the same year and discipline, with some papers accumulating hundreds to thousands of citations. Llewellyn explains, “Georgia CTSA-supported publications averaged 29 citations per paper, and one cancer research publication has been cited more than 2,000 times, so far. That is over 129 times the expected number for a paper from that year (2012) and that field (immunology).” Georgia CTSA’s score is higher than those of Emory, Morehouse School of Medicine, and Georgia Tech, suggesting that the Georgia CTSA partnership is impactful beyond what each institution is doing on their own.
Georgia CTSA research is also published in highly respected journals. The average Journal Impact Factor for Georgia CTSA articles is 6.15 and averages in the 77th percentile, indicating that research is published in journals ranking in the top quartile for their field. “The yearly average CNCI scores and Journal Impact Factors have remained remarkably stable from 2007-2016, showing that supported articles are consistently high in impact and consistently published in highly cited journals,” said Llewellyn. Supported articles are frequently published in widely reaching, multidisciplinary journals such as PLOS One and the Proceedings of the National Academy of Science, as well as prestigious, highly impactful medical journals such as the New England Journal of Medicine and the Journal of the American Medical Association. In addition, many are published in journals focusing on research in pediatrics, endocrinology/diabetes, internal medicine, mental health/neuroscience, and infectious diseases, including HIV/AIDS. To date, Georgia CTSA research has been published in more than 800 journals, covering 110 WoS designated subject areas. This demonstrates the comprehensive scope of topics covered and audiences reached by Georgia CTSA-supported research.
Big Picture & Method Sharing
Llewellyn explained that the Evaluation & Continuous Improvement program started with internal Georgia CTSA results for publications and citations, before moving on to a big picture evaluation of all 64 CTSAs operating at large academic medical centers across the country. Results revealed that Georgia CTSA is a particularly strong CTSA hub in terms of publication output and citation impact. The team also explored preliminary evidence that multi-institutional hubs, like Georgia CTSA, have especially high publication impact. This idea supports findings from an earlier evaluation of Georgia CTSA’s Pilot Grants program, which found that multi-institutional pilot projects were cited at a significantly higher rate than single-institution projects. “When researchers collaborate they create high-quality science. This is another reason that our cross-institutional collaboration is vital to the success of Georgia CTSA.” said Llewellyn. This work was recently published in Academic Medicine and explains that the CTSA program has collectively supported over 66,000 publications, which have been cited over one million times as of early 2017. The overall CTSA’s average number is 2.08, meaning that CTSA-supported publications are cited more than twice as often as comparable papers from the same year and discipline.
Excitingly, these innovative evaluation methods can be exported across the CTSA consortium and to other large grants. “We have recently established a national bibliometric workgroup among the CTSA-wide Evaluation & Continuous Improvement group charged with creating useful metrics for all CTSAs based upon our methods,” said Eric Nehl, PhD, Director of Georgia CTSA’s Evaluation & Continuous Improvement program, and Assistant Research Professor in the Department of Behavioral Sciences and Health Education at Emory University’s Rollins School of Public Health.
The Georgia CTSA is a statewide partnership between Emory, Morehouse School of Medicine, Georgia Tech, and University of Georgia and is one of over 50 in a national consortium striving to improve the way biomedical research is conducted across the country. The consortium, funded through the National Center for Advancing Translational Sciences (NCATS) and the National Institutes of Health’s Clinical and Translational Science Awards, shares a common vision to translate laboratory discoveries into treatments for patients, engage communities in clinical research efforts, and train the next generation of clinical investigators.