Human Challenge Studies

Lessons learned from the COVID-19 vaccine rollout Benjamin Hall October 2021

---

Photo: iStock.com/imaginima

Risks were elevated in 2020 due to the COVID-19 pandemic. This obvious understatement demands the attention of actuaries if we are to be the leading professionals in finding ways to manage risk. Risk is opportunity for actuaries, but we are a data-based profession—and a novel disease meant data was in short supply. This article will argue there is a method that could have quickly obtained key data on mitigating the pandemic while remaining laser-focused on mitigating risk, and many deaths could have been avoided as a result. This method should attract the attention of actuaries.

Consider this remarkable truth: The antidote to the COVID-19 pathogen was available before the first confirmed American COVID-19-related death. The Moderna vaccine was designed on Jan. 13, 2020, following the sequencing of the novel coronavirus genome.¹ Pfizer had developed candidate vaccines by Jan. 25, 2020.² Of course, no one knew in January that the Moderna and Pfizer vaccines would each prove to be 95 percent effective at preventing COVID-19, let alone if they were safe. Recommending widespread vaccinations without evidence of effectiveness and safety would have been foolish.

But Operation Warp Speed (the name of the public-private partnership to develop the COVID-19 vaccines) shattered the previous record, which was the four-year development of the mumps vaccine.³ After completing Phase I trials, the Pfizer vaccine began Phase II/III trials on July 27, 2020, and received emergency authorization from the U.S. Food and Drug Administration (FDA) on Dec. 11, 2020. Similarly, the Moderna vaccine published Phase I trial results on July 14, 2020, and received emergency authorization from the FDA on Dec. 18, 2020.

In both cases, nearly six months elapsed between the determination that the vaccine was well-tolerated and the approval stating the vaccine was effective. In a world where COVID-19 was spreading steadily, this time lapse came at the expense of a couple hundred-thousand lives. Could we have more effectively managed risks by obtaining the necessary safety and efficacy data faster?

The answer is yes: Once vaccine safety has been established, there is a method to arrive at rigorous decisions about vaccine effectiveness in short order. This method is called “human challenge studies” and involves intentionally exposing study participants to the contagion. This may sound dangerous, but potential harm can be extremely minimal, and the payoff can be enormous.

Human Challenge Studies Overview

In a challenge study, the placebo group intentionally becomes exposed/infected within a matter of days. Many of these individuals will show some symptoms. The test group that receives the vaccine is also intentionally exposed to the contagion. But if the vaccine works, most of the test group will not become infected/symptomatic. This has two implications:

1. Not nearly as many participants are needed
2. Once participants are recruited, only a fraction of study time is required compared to traditional studies

To see how a challenge study can work more efficiently than a traditional one, consider the actual studies that were implemented. The Pfizer study involved more than 43,000 individuals who were followed for nearly four months. Of the nearly 22,000 individuals in the placebo group, only about 2.5 percent contracted COVID-19 during the study time period. Less than 0.5 percent of the nearly 22,000 in the vaccine group contracted COVID-19, and nearly all of those were within the first week of the first dose, before the vaccine had time to take effect.⁴

After the Pfizer vaccine had a week to take effect, it had near-perfect results. But to attribute COVID-19 prevention to the vaccine, based on the data, we needed to see the placebo group contract the disease. The 2.5 percent of placebo-group individuals who developed COVID-19 provided that evidence. Yet it took more than three months after the vaccine group received the second dose for even 2 percent of the placebo group to show symptoms.

To attribute a small difference (such as 2 percent) to the vaccine and not simply chance requires a sample size of thousands. The large sample size provided the necessary statistical power. In contrast, challenge studies generate statistical power by increasing the effect size. The challenge study effect size is larger because the placebo group infection rate is intentionally very high while a vaccine protects the test group. An effect size of 50 percent can be attributed to the vaccine with less than 100 study participants. Since any differences between the vaccine and placebo groups show up nearly immediately in a challenge study, we need less than 100 participants for a challenge study—opposed to the more than 40,000 in the Pfizer study.

Hypothetical COVID-19 Human Challenge Study Benefits

A challenge study could have eliminated more than three months of waiting time. Plus, it takes less time to recruit a smaller pool of participants and administer the injections. Each stage could be administered to a small group in a single day, while the study of thousands took weeks to roll out each dose. The actual Pfizer study began on July 27. A challenge study could have started a few weeks earlier (due to the need to recruit far fewer participants), with the second dose administered around Aug. 1, since the protocol was two doses separated by 21 days. Researchers might have wanted to wait another week for the vaccine to take full effect. Both the vaccine group and the placebo group could have been intentionally exposed to COVID-19 around Aug. 8.

By the middle of August, it would have been apparent that the vaccine was more than 90 percent effective. Distribution of the vaccine could have begun the first week of September. The manufacturing of additional doses also could have been launched at this time, so that by the end of the year, a significant portion of the population could have been vaccinated.

The actual Pfizer and Moderna vaccine studies were carefully performed. Randomized studies are the gold standard for providing scientific evidence. Tens of thousands of individuals were assigned to each of the participant and placebo groups, and the vaccine group exhibited statistically significant reductions in COVID-19 cases. The evidence is overwhelming that these two vaccines are both generally well-tolerated and clearly effective. But mathematically, it is equally clear that a challenge study with far fewer study participants would have come to this same conclusion far earlier.

It is difficult to say precisely how many lives could have been saved with a four-month head start on the vaccine rollout, but it would surely have been at least 200,000 to 300,000. The actual December vaccine launch led to 20 percent of Americans being vaccinated within three months. If the launch had been in August, we could have had 20 percent of Americans vaccinated before Thanksgiving.

Addressing Human Challenge Study Concerns

Although the general concept of challenge studies has some detractors due to ethical concerns, medical researchers already were conducting studies using this method before COVID-19. Physicians have recognized that well-designed challenge studies are ethically comparable to living organ donors and do not necessarily violate the “do no harm” principle. The FDA issued the first U.S. approval for a vaccine based on a challenge study in 2016, and researcher and manufacturer interest with respect to challenge studies has since skyrocketed.⁵

In recent years, the Walter Reed Army Institute of Research has intentionally infected study participants with dengue fever. To mitigate potential harm, the studies have kept participants inpatient or at hotels to monitor their fevers, headaches, rashes and joint pain.⁶ During the COVID-19 pandemic, researchers argued a necessary next-generation pertussis vaccine will require the use of challenge studies due to circumstances specific to that disease.⁷

Despite the availability of multiple COVID-19 vaccines, the United Kingdom launched several small-sample COVID-19 challenge studies in 2021 with the goal of improving treatment and understanding ideal vaccine dosage. In one study, 64 healthy individuals who survived the original COVID-19 strain will be intentionally re-exposed and monitored for 17 days to estimate the risk of reinfection.⁸

How safe should a human challenge study be to pass ethical muster? There is no clear consensus. One set of proposed guidelines recommends that volunteer participants face no more risk than the normal activities of life, specifically of riding a bicycle. However, the risk of riding a bicycle varies based on factors, such as how many miles the cyclist travels and under what conditions.⁹

A group of Nobel laureates who encouraged COVID-19 challenge studies pointed out that for the young and healthy, the risks from participating in a COVID-19 challenge study were similar to living kidney donors.¹⁰ If society allows volunteers to give a kidney to help someone else, what logic forbids them to face a similar risk in a COVID-19 study that could potentially help millions? Actuaries are uniquely positioned to weigh in on such risk comparisons and are needed voices in these ethical conversations.

Conclusion

Hindsight offers improved vision. Some vaccine producers were willing to consider challenge studies, and a challenge study may have been the best way to save more lives from COVID-19. As we ask what we could have done differently in the face of the novel coronavirus, a human challenge study should be on the list.

Actuaries can also consider business-related contexts where the challenge study model may be useful. Use cases could be any situation where the effectiveness of mitigation efforts on an adverse experience needs to be determined. As long as the adverse experience can be ethically created on a small sample (for example, deteriorating benefit offerings), then a challenge study allows for strategic testing of the intervention strategy. However, the most important challenge study use cases will continue to involve the development of vaccines, both for known diseases and for the next novel pathogen.

Statements of fact and opinions expressed herein are those of the individual authors and are not necessarily those of the Society of Actuaries or the respective authors’ employers.

Copyright © 2021 by the Society of Actuaries, Schaumburg, Illinois.