Month: June 2018

Three members of the Innovation and Value Initiative (IVI) recently published a paper entitled “Open-Source Tools for Value Assessment: A Promising Approach” in the Journal of Clinical Pathways. This paper lays out, in brief, some of the ways that open-source models can contribute to the challenging environment in which value assessment operates in the US.

Unlike many nations where cost-effectiveness analysis is widely used and accepted, the US has a highly decentralized healthcare system. Even when up to date US-based models are available, they are likely not applicable to every patient population. This matters because not only does treatment response vary between populations, but so does the conception of value.

Meanwhile, healthcare decision makers must assess what evidence on value exists while simultaneously trying to assess its applicability to their patients, all without robust guidance on how to adapt the conclusions of modeling studies.

IVI has tried to change this by releasing an open-source microsimulation model for rheumatoid arthritis – a common disease whose treatment with biologics has become a significant driver of drug costs for many payers. This model is extremely flexible and speaks to the needs of healthcare decision makers by allowing for modification of treatment sequences, elements considered in the definition of value, and even whether results are formatted as a cost-effectiveness analysis or a multi-criteria decision analysis. Better still, this software is released as both a convenient web-app and as an R package with fully open code.

This is a tremendous step forward for value assessment in the US and sets a new standard for openness in modeling. Still, I can’t help but wonder how this transition from proprietary, closed models to open models will be funded. After all, IVI is in a unique position, with funding from many large pharmaceutical companies and industry organizations. If every consulting company had to organize a consortium to fund its open-source modeling initiatives, this would quickly become very burdensome.

As the “Open-Source Tools” paper points out, IVI took its inspiration for its rheumatoid arthritis model from open-source software, and we can do the same in thinking about how open-source modeling efforts could be supported. Some companies who develop open-source software support themselves by offering paid support plans for their products. A typical example here would be Canonical, which develops the Ubuntu Linux distribution. While it offers its operating system for free to anyone who wants it, it also offers paid plans that include help with deployment and maintenance.

It’s hard to know whether the scale of a typical model’s distribution would allow for this source of income, though. While Linux users number in the millions, a typical value model may have just dozens of users. Competition is likely to be important to motivate the timely development and updating of models, but the question of funding needs to be solved before more developers can take part.

The real value of an open source model depends too on the data it uses. To truly customize a model to a patient population, more granular data on patient response needs to be made available from clinical trials and disease registries. Until this happens, the conclusions of models may be based on estimated shifts in response from small samples.

The shift toward open-source modeling is an important means of responding to the challenges presented by the US healthcare market. However, many problems remain unsolved that for now still prevent more models from being developed in an open and flexible way.

Meng Li is a recent graduate of The CHOICE Institute who defended her dissertation, “The Real Option Value of Life and Innovation,” on May 8, 2018. In addition to her dissertation research, she has published on a wide range of topics including the cost-effectiveness of liver transplantation in organic acidemias, risk-sharing agreements involving indication-specific pricing, and the acceptability of cervical cancer screening in rural Uganda. Her website is mlinternational.org.

What was your dissertation about?

My dissertation was about the real option value of medical technologies. The real option value of a technology refers to the value of enabling patients to live longer to be able to take advantage of future breakthroughs.  In my dissertation, I answered two closely related questions: (1) do patients consider real option value when they make their treatment decisions; and (2) how should real option value from technology advancement be incorporated in economic evaluations of medical technologies and what is the potential impact? To answer the first question, I studied metastatic melanoma patients from 2008 to 2011 and examined if they changed their treatment decisions after the announcements of the results of the breakthrough drug ipilimumab’s phase II and phase III clinical testing. The idea behind this was that, if patients considered real option value – the value of taking advantage of future innovations – they might be more likely to undergo treatments that can extend their lives. In my analysis, I found that in anticipation of ipilimumab’s arrival, metastatic melanoma patients were more likely to undergo surgical resection of metastasis, which was consistent with my hypothesis. To answer the second question, I developed methods to project likely future approvals in metastatic melanoma and potential future improvement in mortality in this patient population and incorporated them into a cost-effectiveness analysis. In my analysis, I found that the incremental QALYs increased by about 5-7% after accounting for future innovations, and the ICERs decreased by about 0-2%.

How did you arrive at that topic? When did you know that this is what you wanted to study?

I have long been interested in pricing and value assessment of medical technologies. In the short proposal phase, I brainstormed with Lou Garrison (my dissertation committee chair) and explored several topics in that area. Around that time, ISPOR also started to form a special task force to study US value assessment frameworks and Lou was the chair of the task force. As a result, a lot of our discussions back then were around the economic foundations of cost-effectiveness analysis, and benefits of a technology that are usually not accounted for in a conventional cost-effectiveness model. Real option value is one of those benefits, and it is intuitive conceptually, and relatively easy to operationalize.

What was your daily schedule like when you were working on your dissertation?

I did not have a fixed schedule when I was working on my dissertation. I usually had several research projects going on at the same time, and I tried to balance doing side projects with working on my dissertation. However, instead of spending a couple hours a day, I liked to have a few uninterrupted days when I can immerse myself in my dissertation research.

In what quarter did you submit your final short proposal and in what quarter did you graduate/defend? What were some factors that determined your dissertation timeline?

I started brainstorming ideas for dissertation in the summer of 2015, after my second year, and submitted my short proposal in March 2016, defended my dissertation proposal in February 2017, and defended my dissertation in May 2018. I would say my dissertation had a relatively slow start, as there was little existing research on the particular topic that I was studying. A lot of time was spent on thinking through the theory and developing the research plan. I probably could have finished sooner, but I decided to follow the school cycle and graduate in the Spring. I also wanted to be well prepared for my general and final exams and submit my dissertation papers to journals before I graduate.

How did you fund your dissertation?

My dissertation was not funded by any particular grant. However, I was one of the TAs for the Certificate Program in Health Economics & Outcomes Research in the Department of Pharmacy, which supported me financially during my last year in graduate school. Before that, I was the research assistant on various research projects, which provided me with financial support in the first four years of my graduate school.

What will come next for you (or has come next for you)? What have you learned about finding work after school?

I will be joining the Schaeffer Center for Health Economics & Policy at the University of Southern California as a postdoctoral scholar. My search for jobs in the past few months was mostly focused on postdoc positions. I searched job boards (Indeed and LinkedIn), career pages of professional societies (ISPOR and AcademyHealth), and websites of the groups that I am interested in working with. My advisors and colleagues also connected me with groups where there might potentially be opportunities.

There was recently a Twitter trend of people trying to describe programming in five words. The responses ranged from funny to puzzling to inspiring.

At our celebration of the end of the academic year, students, post-docs and faculty of the CHOICE Institute decided held a similar contest at our weekly seminar, and instructed attendees to “describe health economics and outcomes research (HEOR) in five words.” Here are a few of my favorite entries:

• “How to hurt less, cheap.” (Samantha Clark)
• “Math predicting value of medicine.” (Blythe Adamson)
• “Examining well-being trade-offs and technology.” (Doug Barthold)
• “Yo, treat sick people cost-effectively.” (Shuxian Chen)
• “To each his own evaluation.” (Nobody claimed this one, unfortunately!)
• “What is beyond opportunity cost?” (Enrique Saldarriaga)

As researchers in HEOR, the challenge of trying to explain what we do to outsiders is familiar to us all. That’s why it was really fun to try to encapsulate our field into just a few words.