Book Review

Difficult Choices Between What is Best for Science or Best for Our Career

RIGOR MORTISHow Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions
By Richard Harris
288 pages, Perseus Books Group, List Price $28

In “Rigor Mortis: How Sloppy ScienceCreates Worthless Cures, Crushes Hope, and Wastes Billions,” Richard Harris provides compelling evidence through a series of stories and statistics that medical research is plagued by unnecessary errors despite our technology, effort, money, and passion to make a positive impact. This review takes the perspective of a graduate student in health sciences with the aim of assessing the value of Rigor Mortis for the next generation of scientists.  While the book focuses more on sloppy biological science, his concerns are equally valid in the areas of data science and disease modeling.

Richard Harris, a journalist at NPR who writes about science, has started an important conversation about the broad impact of our current scientific culture: we are publishing too many scientific studies which may have false or unreproducible results. Graduate students in health science research or related fields should not be surprised by Harris’s premise. The pressure to produce a large quantity of publications, instead of fewer and higher quality papers, weighs on every grad student in the world.

In 2017, the CHOICE Institute asked its members to read Rigor Mortis and discuss its implications for our field. One emerging theme was that trainees need to be able to report unethical behaviors without fearing adverse consequences. While required annual courses from the University of Washington Biomedical Research Integrity Series challenge students to reconsider their own personal conflicts of interest in publishing research, this remains a difficult ideal to implement in the face of other pressures. Around the lunch table in our grad student lounge, the book sparked an uncomfortable conversation about multiple testing during regression model fitting, and the long stretch of grey area between a dusty, pre-specified analysis plan and our shiny, new hypothesis-generating exploratory findings.

Harris’s storytelling reminded me of a book I love by David Quammen called “Spillover.” Both Rigor Mortis and Spillover are written by distinguished journalists about very complicated and technical problems. Using New York Times reader-friendly language,  both authors include conversations with scientists from around the world to share their stories so that the layperson can understand.

Both books highlight a common dilemma in academia: Should I do what is best for science or what is best for my career? Further, is this an incentives problem or a system problem? The current structure and business of research guide us to make choices that will enhance our career, while science is still often perceived as an altruistic pursuit for the greater good. The book offers a challenge to academic researchers: who among us can claim “no conflict of interest”?

Applying the book’s messages to health economics and outcomes research

I experienced this dilemma when deciding whether to share my HIV disease model. Scientific knowledge and methodology should be completely transparent, yet software coding to implement these techniques is intellectual property that we should not necessarily give away for free. My dilemma isn’t unique. Disease modelers everywhere struggle with this question: should we post our Excel spreadsheet or R code online for others to review and validate and risk having our discovery poached?

This is just one example of tension Harris highlights in his book, and why it is so complex to change our current scientific culture. Scientific advancement is ideally a collective good, but individuals will always need personal incentives to innovate.

Key book takeaways for young scientists

1. Use valid ingredients
2. Show your work
3. No HARKing (Hypothesizing After the Results of the study are Known)
4. Don’t jump to conclusions (and discourage others from doing this with your results)
5. Be tough. People may try to discredit you if your hypothesis goes against their life’s work, or for any number of reasons.
6. Be confident in your science.
7. Recognize the tension between your own achievement and communal scientific advancement

Further discussions for fixing a broken system

1. If money is being wasted in biomedical science and research, how do we fix the system to save money without sacrificing incentives to produce valuable innovations? One of our CHOICE Institute graduates, Carrie Bennette, asked this very question in cancer research and you can read about her findings here.
2. Incentives need to be changed. Academic promotions should not be dependent on the number of our publications but the quality and impact of our contributions. Can we change the culture obsessed with impact factor and promote alternatives such as the H-index or Google Scholar metrics?
3. Academic tenure systems are antiquated. How do we balance the trade-offs between hiring a post-doc or hiring a permanent staff scientist? Post-doc positions train the next generation and are cheap, however they result in workflow discontinuity from frequent turnover. Permanent staff scientists are trained to stay for a longer period of time, but would disrupt an engrained academic pipeline.

Conclusion

I think all students in any science-related field would benefit from reading this book. Cultural and systematic change will happen faster when we have uncomfortable conversations at the table with our colleagues and mentors. Additionally, we need to take the baton Richard Harris has passed us and start running with our generation of colleagues toward finding and implementing solutions. As our influence in our respective fields grows, so too does our responsibility.

Rigor Mortis is available in hardcover on Amazon for $18.65 and Audible for $19.95.

SaveSaveSaveSave

SaveSave