Exit interview: CHOICE alumnus Solomon Lubinga

Editor’s note: This is the second in an ongoing series of interviews we’ve planned for the students graduating from the CHOICE Institute where we’ll get their thoughts on their grad school and dissertation experiences.

Solomon Lubinga is pharmacist and an applied health economist. After graduating with his PhD from the Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute in 2017, he became a senior fellow at the CHOICE Institute at the University of Washington, working with Dr. Josh Carlson in collaboration with the Institute for Clinical and Economic Review (ICER). He is interested in decision modelling, value of information/implementation research as well as the econometric and health policy applications of discrete choice models.

lubinga_solomonFor more details on Solomon’s work, check out his personal webpage at http://www.jonslo.com

  • What was your dissertation about?

In my dissertation, I abstracted from

well-known economics and socio-psychology

decision theories to study the incentives that drive the uptake of medical male circumcision (MMC) for HIV prevention in Uganda. My hypothesis was that a model that would combine factors from both decision theories would not only more accurately predict MMC decisions, but also be a very powerful tool to predicting the potential impacts of different MMC demand creation strategies.

  • How did you arrive at that topic? When did you know that this is what you wanted to study?

I became interested in the intersection of economics and psychology early on in the PhD program. I suppose this was because of my own proclivity to act irrationally even though I considered myself a well-informed person. This led me to ask why individuals in lower income countries in general do not value preventive health interventions. This specific topic built on a prior contingent valuation study estimating willingness to pay (WTP) and willingness to accept payment (WTAP) for safe MMC among men in high-HIV-risk fishing communities in Uganda. The results of this analysis indicated low demand (WTP) and high potential incentive value (WTAP) for MMC, suggesting that a high WTAP (a defacto increase in MMC price) may result in an unfavorable incremental cost-effectiveness or benefit-to-cost ratio for MMC. I was therefore interested in studying the relative roles of economic and psychological incentives on demand for MMC.

  • What was your daily schedule like when you were working on your dissertation?

I never had a set schedule while I worked on my dissertation. I was also a teaching assistant (TA) for the online health economics course offered by CHOICE. I spent a lot of time in Uganda collecting my data. I would spend the day in the field (8:00am – 5:00pm) and the evenings (7:00pm – 11:00pm) performing my TA duties. It turned out that this was convenient (but challenging) because of the time difference between Uganda and the west coast. I also travelled to the UK twice for a choice modelling course, which was a great help with my dissertation. When I was in Seattle, I generally combined work on my dissertation with my teaching assistant responsibilities at the UW, with no set schedule. I simply gave what was more urgent the priority.

  • If you are willing to share, what was the timeline for your dissertation? And what determined that timeline?

I submitted my short proposal sometime in April, 2015. I defended my dissertation in August, 2017. Two major factors determined my timeline. First, the death of a close family member motivated me to take some personal time. Second, although I was fortunate to receive funding for my data collection activities, it took almost 8 months (between December 2015 and October 2016) to receive international clearance for the data collection activities.

  • How did you fund your dissertation?

As I mentioned, I was fortunate to receive funding for my data collection activities through a grant awarded to my dissertation advisor.

  • What will come next for you (or has come next for you)? What have you learned about finding work after school?

I am interested in academic positions in universities in the US, or other quasi-academic institutions (e.g., research institutes or global organizations that conduct academic-style research). As an international student, the main lesson I have learned is “to synchronize the completion of your studies with the job market cycle, especially if you are interested in academic positions in the US”.

Is there still value in the p-value?

not sure if significantDoing science is expensive, so a study that reveals significant results yet cannot be replicated by other investigators, represents a lost opportunity to invest those resources elsewhere. At the same time, the pressure on researchers to publish is immense.

These are the tensions that underlie the current debate about how to resolve issues surrounding the use of the p-value and the infamous significance threshold of 0.05. This measurement was adopted in the early 20th century to indicate the probability that the observed results are obtained by chance variation, and the 0.05 threshold has been with it since the beginning, allowing researchers to declare as significant any effect they find that can cross that threshold.

This threshold was selected for convenience in a time when computation of the p-value was difficult to calculate. Our modern scientific tools have made calculation so easy, however, that it is hard to defend a 0.05 threshold as anything but arbitrary. A group of statisticians and researchers is trying to rehabilitate the p-value, at least for the time being, so that we can improve the reliability of results with minimal disruption to the scientific production system. They hope to do this by changing the threshold for statistical significance to 0.005.

In a new editorial in JAMA, Stanford researcher John Ioannidis, a famous critic of bias and irreproducibility in research, has come out in favor of this approach. His argument is pragmatic. In it, he acknowledges that misunderstandings of the p-value are common: many people believe that a result is worth acting on if it is supported by a significant p-value, without regard for the size of the effect or the uncertainty surrounding it.

Rather than reeducating everyone who ever needs to interpret scientific research, then, it is preferable to change our treatment of the threshold signaling statistical significance. Ioannidis also points to the success of genome-wide association studies, which improved in reproducibility after moving to a statistical significance threshold of p < 5 x 10-5.

As Ioannidis admits, this is an imperfect solution. The proposal has set off substantial debate within the American Statistical Association. Bayesians, for example, see it as perpetuating the same flawed practices that got us into the reproducibility crisis in the first place. In an unpublished but widely circulated article from 2017 entitled Abandon Statistical Significance [pdf warning], Blakely McShane, Andrew Gelman, and others point to several problems with lowering the significance threshold that make it unsuitable for medical research.

First, they point out that the whole idea of the null hypothesis is poorly suited to medical research. Virtually anything ingested by or done to the body has downstream effects on other processes, almost certainly including the ones that any given trial hopes to measure. Therefore, using the null hypothesis as a straw man takes away the focus on what a meaningful effect size might be and how certain we are about the effect size we calculate for a given treatment.

They also argue that the reporting of a single p-value hides important decisions made in the analytic process itself, including all the different ways that the data could have been analyzed. They propose reporting all analyses attempted, in an attempt to capture the “researcher degrees of freedom” – the choices made by the analyst that affect how the results are calculated and interpreted.

Beyond these methodological issues, lowering the significance threshold could increase the costs of clinical trials. If our allowance for Type I error is reduced by an order of magnitude, our required sample size roughly doubles, holding all other parameters equal. In a regulatory environment where it costs over a billion dollars to bring a drug to market, this need for increased recruitment could drive up costs (which would need to be passed on to the consumer) and delay the health benefits of market release for good drugs. It is unclear whether these potential cost increases will be offset by the savings of researchers producing more reliable, reproducible studies earlier in the development process.

It also remains to be seen whether the lower p-value’s increased sample size requirement might dissuade pharmaceutical companies from bringing products to market that have a low marginal benefit. After all, you need a larger sample size to detect smaller effects, and that would only be amplified under the new significance thresholds. Overall, the newly proposed significance threshold interacts with value considerations in ways that are hard to predict but potentially worth watching.

Exit interview: CHOICE alumna Elisabeth Vodicka

Editor’s note: This is the first in an ongoing series of interviews we’ve planned for the students graduating from the CHOICE Institute where we’ll get their thoughts on their grad school and dissertation experiences.

This first interview is with Elisabeth Vodicka, who defended her dissertation, “Cervical Cancer in Low-Income Settings: Costs and Cost-Effectiveness of Screening and Treatment,” on January 24th, 2018.

  • What’s your dissertation about?

evodickaMy dissertation focused on the economics of integrating cervical cancer screening and treatment into existing health systems in East Africa (Kenya and Uganda). Although cervical cancer is preventable and treatable if detected early, screening rates are low in the region (3-20% depending on regional characteristics). One strategy for improving access to potentially life-saving screening is to leverage the fact that women engage with various health systems platforms for other types of care, like for family planning, taking their children in for vaccinations, tuberculosis and HIV-treatment, etc. Since these programs are already funded and staffed, screening could be offered to women in these settings via service integration for potentially low marginal costs.

To understand the economic impact of offering cervical cancer screening and treatment to women when and where they are already engaging with the health system, I conducted costing, cost-effectiveness and budget impact analyses of integrating screening into two health care settings. First, I collected primary data and conducted a micro-costing analysis to determine direct medical, non-medical and indirect costs associated with integrating screening services into an HIV-treatment center in Kenya. For my subsequent aims, I conducted economic evaluations evaluating the potential value created in terms of cost per life year saved and budget impact to the Ministry of Health of integrating screening and treatment into HIV-treatment centers in Kenya and routine childhood immunization clinics in Uganda.

  • How did you arrive at that topic? When did you know that this is what you wanted to study?

I have long been passionate about women’s health issues and improving access to care in low-resource settings. During my second year in the program, I was offered an opportunity through the University of Washington’s Treatment, Research, and Expert Education program to conduct a micro-costing study to identify the costs associated with providing cervical cancer screening to women attending receiving HIV treatment at Coptic Hope Center for Infectious Diseases in Nairobi, Kenya. This opportunity presented a perfect overlap of my interests in women’s health, access, and health economics methods. After conducting the primary data collection, I began exploring the possibility to continue this line of research through my dissertation.

  • What was your daily schedule like when you were working on your dissertation?

To be honest, I can’t say that I had a daily schedule that was consistent over the years of working on my dissertation. While I developed my short and long proposals, I was still in classes, working as an RA, and doing consulting work on the side. These commitments often dictated the time I had to focus on preparing my dissertation proposals, which I often worked on late at night. Once I passed my general exam, my time was much more flexible. I tend to be most productive at night, so took advantage of time flexibility to make the most of my productive hours. Often this meant exercising and taking care of other tasks during the day and then leveraging my peak brain time to work on my dissertation in the evenings.

Additionally, creating regular social opportunities and accountability for my dissertation progress were key success strategies for me. My cohort and I created a dissertation writing group that met weekly to create accountability. Toward the final months of the dissertation (crunch time!), I joined a co-working space, used an online co-working app, and recruited friends and family to work together virtually and in-person to maximize accountability and meet my goals for each dissertation aim.

  • If you are willing to share, in what quarter did you submit your final short proposal and in what quarter did you graduate/defend? What were some factors that determined your dissertation timeline? 

I submitted my short proposal in Fall Quarter 2015, and I defended in Winter Quarter 2018. Together, my chair and I developed a timeline that mapped out each stage of the dissertation process from the short proposal to final defense.

  • How did you fund your dissertation?

Ongoing funding was received through work as an RA and TA. The TREE program generously supported my in-country work in Kenya. I also received additional financial and travel support through internal funding within CHOICE (e.g., Reducing Barriers for the Ambitious Fund, Rubenstein Endowment, etc.).

  • What comes next for you? What have you learned about finding work after school?

Currently, I am continuing to work as a freelance consultant on projects related to expanding access to care in low- and middle-income settings. This allows me the time and flexibility to target my employment search within groups that are an excellent fit – both organizationally and culturally – for my research interests and professional goals.

In terms of finding employment after school, the most important lesson that I have learned is to start early and network broadly. During my first year in the program, I set a goal to reach out to one new person in the field every month working on topics or in organizations that interested me. Over time and many networking coffees later, I learned about the types of organizations that might be a good fit for my interests, work style and personality, and developed positive relationships with other like-minded individuals.

Statements of purpose: a view from the admissions committee

I recently had the opportunity to serve on a graduate program admissions committee and had a few reflections on the application process that I wanted to share. In particular, I wanted to give some advice to applicants on what makes a good statement of purpose and how you can use that document to put your best foot forward in the application process.

Graduate programs in health economics and outcomes research (HEOR) recruit from a number of undergraduate and master’s-level fields. This intersection of disciplines, in addition to the lack of undergraduate programs specific to HEOR, creates a unique set of considerations that applicants need to keep in mind when writing their statements of purpose. Here are a few brief tips on writing a successful statement of purpose in your application to HEOR programs:

Tip 1. Establish your connection to the field

First, it needs to be abundantly clear to the reader that the applicant understands what HEOR is. While some students – such as those with a previous master’s degree – may have been able to engage directly with the field, others have to demonstrate their familiarity more explicitly. This advice is especially applicable to clinicians, who bring an invaluable perspective to HEOR studies but are rarely able to participate in research during their clinical training.

Writers can demonstrate an understanding of HEOR in their statements of purpose by talking about specific classes they have taken or mentors who have talked to them about the field. It’s also vital for applicants to demonstrate an understanding of current issues in the field. Keeping up with HEOR blogs and journals is a great way to gain this understanding! The Academic Health Economists’ Blog and Healthcare Economist are among my favorites.

Tip 2. Clarify your goals

Next, the admissions committee will be curious of what your goals are and how a graduate degree in HEOR might help get you there. This means writing about what you plan to do after graduate school. Nobody is going to hold you accountable to what you say in your statement of purpose, but discussing goals demonstrates to the admissions committee that you have reflected on your future and that you can be somewhat self-directed should you enter the program. Graduate school should not be a way of postponing your entry into the “real world.”

Tip 3. Demonstrate your knowledge of the specific program

It’s important to demonstrate abundantly that you have researched your target programs thoroughly. Especially for smaller programs, it’s helpful to know how you heard about the program. If a mentor suggested you apply, mention this! With the HEOR community being relatively small, there’s a good chance that someone on the admissions committee will know your mentor, which is likely to work in your favor.

Tip 4. Polish your writing

Graduate training is a massive investment for both the school and the student. When making decisions about who to admit, the admissions committee uses the statement of purpose to assess your writing skills, the rationale for your decision to apply to their program, and your suitability for a career in HEOR. A well-written statement can pique the interest of everyone on the committee, while a poorly written one can make a candidate seem less interesting, even when every other part of their application looks ideal. Polish your statement into a masterpiece that showcases your enthusiasm for the field and you’ll be a step ahead in the admissions process.