A summary of the main panel discussion points of the November 19th Research Town Hall “Counteracting Perverse Incentives in Academic Research”
The topic of our November research town hall, “Counteracting Perverse Incentives in Academic Research”, was inspired by an article published in 2017 by Marc A. Edwards and Siddhartha Roy, “Academic Research in the 21st Century: Maintaining Scientific Integrity in a Climate of Perverse Incentives and Hypercompetition”. The concept of perverse incentives defines the flipside of the quantitative measures instituted in Academia; incentives intended to boost growth, productivity and academic or research impact lead to increased competition for grant funding, publications, citations or rankings and develop perverse effects on research practices, culture, and research environment. In other words, instead of focusing on values and quality, academics can develop a mentality focused on scientific output. For example, that paper noted that the increased number of publications, a measure intended to enhance research productivity, also led to increased number of low standard papers. “The competition for grant funding led to questionable research practices, such as overselling positive results, in addition to the intended effect of promoting research productivity.
Dr. Adrian Hernandez, Vice Dean and DCRI Executive Director: “The index factor does not capture the entire picture of somebody’s impact on education, science, policy and society”
Over 170 members of the community attended the town hall where Duke researchers and leaders discussed how these quantitative metrics have been impacting our research culture, research quality, research integrity and how the institutional leadership can help our community counteract their perverse effect. Dr. Adrian Hernandez, Vice Dean and Executive Director of the Duke Clinical Research Institute talked about the need to balance quantitative and qualitative measures. “We are taught to think in numbers from an early age. In Academia this does not get better. The index factor does not capture the entire picture of somebody’s impact on education, science, policy and society,” Hernandez added. In other words, being unable to provide information about other key aspects of academic impact (such as mentoring, contributions to society and communities, or leading policy initiatives) the h index becomes a disincentive focused on volume rather than true impact.
Beth Gifford, PhD, Margolis Center for Health Policy: “Research on rural populations, for example, would be very difficult to do on a short career trajectory”
Professor Beth Gifford, of the Duke-Margolis Center for Health Policy, an Associate Professor in Pediatrics and Sanford School of Public Policy talked about the impact of funding on the type of research that is carried out; and therefore, how the pressure for funding may automatically push researchers to discriminate relevant topics. “Research on rural populations, for example, would be very difficult to do on a short career trajectory. It is expensive and slow because it requires more time with sampling and other initial phases”. Associate Vice President for Research, and Vice Dean of Scientific Integrity, School of Medicine Dr. Geeta Swamy, found these insights relevant for institutional efforts to think about what research resources we need to identify in order to advance research in areas aligned with our community development goals and values.
Gifford and Swamy commented on how the wide variety of practices across disciplines makes these quantitative measures confusing, hard to understand and ponder their true value. For example, the standard number of publications per year, the number of citations or the number of authors per paper are very different from one research area to another. While in social sciences, it is common to have 2-4 authors per paper; in biomedical research, the number of authors can be much higher. As a result, the workload per author can be far greater in social sciences.
Dr. Swamy then commented on how the goals for publishing may differ from one discipline to another. For instance, the goal of publishing a paper in a high-profile journal may lead to “too good to be true scenarios”, when researchers submit for publication papers with results that are not backed up by data.
So how do we address these challenges given this variety of standards and practices? Gifford mentioned a recent practice as a solution that could be implemented more widely: some funders ask researchers to emphasize on their proposals the importance of cross-disseminating research results to their non-academic communities. This practice pushes researchers to think creatively of the true social impact of their work. Community dissemination of research impact could serve as a qualitative indicator to balance the quantitative measures. In response to the same problem, Dr. Beth Sullivan, Associate Dean for Research Training, Professor of Molecular Genetics and Microbiology, added that some biomedical journals instituted a practice in that each author would describe their contribution on the paper or project, to promote transparency and fairness.
Dr. Beth Sullivan, Associate Dean for Research Training: “We tend to measure productivity and not impact. There tends to be less focus on scholarship and more on transactional work”
The second part of the town hall centered on the effect of perverse incentives on trainees and how graduate programs have been impacted by the quantitative indicators; how the competition and pressures impacted the graduate and post-graduate school/program culture. Paula McClain, Dean of the Graduate School and Vice Provost for Graduate Education, opened the conversation. “PhD students are not only expected to have their degrees but also publications and not only with the advisor, but independent publications.” She commented on the relationship between pressures and questionable practices or research misconduct like the one of Michael LaCour at UCLA years ago, when a graduate student in political science co-authored a paper that falsely claimed that a 15-minute conversation with gay canvassers about gay marriage led to massive shift in voters’ views on the topic.
“This is one of those events when we need to ask ourselves, what do we need to do to support graduate students?” To give the graduate students the space to alert us if something is wrong”, said Dr. McClain. Beth Sullivan, mentioned that in the postdoctoral space, perverse incentives affect individuals with not only competing but different interests. While PhD students want to obtain their degree, postdocs are preparing for the job market and need to distinguish themselves. The pressure to obtain high profile research results and publish in elite journals is high. “We tend to measure productivity and not impact. There tends to be less focus on scholarship and more on transactional work”, said Dr. Sullivan who emphasized the need to have mentors celebrating scholarship, uniqueness and teach trainees what is valuable and truly impactful. Dr. Sullivan is now asking her graduate students to write a statement about how their research moves the field forward in the effort to convey the idea that the value of scholarship is beyond the journal impact factor.
Paula McClain, Dean of the Graduate School and Vice Provost for Graduate Education: “If our students do the work, they need to get the credit and we, the senior researchers, need to be at the end. There should never be courtesy authorship. For some of our colleagues this is tricky because they think they always need to be first authors.”
“Bias is built in the system”, commented Dean McClain who mentioned that including qualitative measures such as student interaction and student communication would lead to a more holistic view of faculty, researchers, and trainees. The key goal beyond the performance indicators should be “building professionalism, integrity, and fairness.” The speakers also talked about the need to train faculty mentors on how to best support trainees to meet their goals and receive more credit for the work they are doing during their postdoctoral fellowship. “Typically, first author is the senior author. We do not need to be first authors. If our students do the work, they need to get the credit and we, the senior researchers, need to be at the end. There should never be courtesy authorship. For some of our colleagues this is tricky because they think they always need to be first authors,” continued McClain.
Assistant Dean Johnna Frierson and Associate VP for Research Geeta Swamy commented on the need to empower graduate and post-graduate students to encourage a cultural shift towards research quality, and to engage them in the process of redefining incentives in Academia
Johnna Frierson, Assistant Dean for Graduate and Postdoctoral Diversity and Inclusion, School of Medicine, talked about the need to empower graduate students to encourage a cultural shift towards research quality, fairness and integrity, as well as a mindset focused on doing research for communities and society overall. “We can incorporate in our RCR courses for trainees how authorship works and if something is not in line with the values we uphold to, then this need to be reported”, suggested Sullivan. She added that not only trainees need to be educated, but also the mentors, the senior researchers. “Faculty need to have a mindset of continuous growth”, said Frierson, while McClain emphasized mentors responsibility to “take seriously what students say”. In addition to all these suggestions, Dr. Swamy mentioned the idea of embedding mentorship training, mentoring awards and recognition among the qualitative incentives for faculty as a way to counteract the perverse incentives in Academia. Swamy also recommended that trainees and researchers need to be engaged in the process of re-defining incentives in Academia, as these incentives have direct impact on their work and professional trajectory, as well as the future of universities, research and society overall.
Many thanks to our distinguished speakers for participating in this event:
Panel I: The PI Perspective
Geeta Swamy, MD, Associate Vice President for Research and Vice Dean for Scientific Integrity, School of Medicine
Adrian Hernandez, MD, Vice Dean and Executive Director, Duke Clinical Research Institute
Beth Gifford, PhD, Duke-Margolis Center for Health Policy, Associate Professor in Pediatrics and Sanford School of Public Policy
Panel II: The Graduate trainer/trainee perspective
Paula McClain, PhD, Dean of The Graduate School and Vice Provost for Graduate Education, James B. Duke Distinguished Professor of Political Science
Johnna Frierson, PhD, Assistant Dean for Graduate and Postdoctoral Diversity and Inclusion, School of Medicine
Beth Sullivan, PhD, Associate Dean for Research Training, Professor of Molecular Genetics and Microbiology