Extraordinary Evidence Requires Extraordinary Hypotheses

Hypothesis: repetitively arrayed vaccines could save the world. Now if only we could please be allowed to just do the experiment

This article was originally posted on Medium in April 2023

Grim experience has taught us that vaccines against SARS-CoV-2 don’t reliably offer durable sterilizing immunity against Covid. A reasonable best-guess might be that each booster dose provides a 50/50 chance of sterilizing immunity that might last as little as six months — especially if major new variants emerge. In stark contrast, a single dose of the HPV vaccine provides a near-guarantee of sterilizing immunity that lasts at least a decade and effectively cross-protects you against related HPV variants that weren’t even in the vaccine. What accounts for the difference between these two vaccines? And why aren’t other vaccines more like the HPV vaccine?

To understand the answer, we should first discuss the so-called Sagan standard: extraordinary claims require extraordinary evidence (ECREE). In school, I was taught that ECREE is a bedrock principle of the scientific method — but several decades of life as a workaday scientist have taught me it’s actually anti-scientific. In my experience, ECREE is mostly used by privileged authorities who have fallen into the trap of mindlessly defending some hallowed bit of dogmatic orthodoxy. Dogma “everybody knows” but that nevertheless sometimes turns out to be disastrously wrong. The actual bedrock foundation of the scientific method is simple: claims require evidence, in a way that’s pretty much upside-down from ECREE. It’s equally important to require evidence for ordinary claims — such as the old claim that everybody knows the Earth is at the center of the cosmos, or the modern claim that everybody knows decorative cloth masks will protect us against Covid.

A fundamental problem with ECREE is the subjectivity of the term extraordinary — particularly on the evidence side of the ledger. In 1808, Thomas Jefferson rejected the extraordinary claim that the type of rocks we now call meteorites fall from the sky. Jefferson’s error was in his failure to recognize that Jean-Baptiste Biot’s comprehensive 1803 survey of several thousand meteorites could reasonably have been viewed as extraordinary evidence. Similarly, climate denialists have spent the past half-century incrementally moving the goalposts on what might constitute sufficiently extraordinary evidence of the inconvenient claim that human-made climate change is occurring.

Since the time of Galileo, the central error-correction mechanism of the scientific method has been the replication and extension of key findings by independent investigators. This is good, because other possible error-correction methods — such as peer review — are themselves highly error-prone. In 2009, peer reviewers for Science missed glaring clues that a retrovirus called XMRV might be a lab artifact. Conversely, ECREE thinking led peer reviewers at Journal of Virology to reject the initial discovery of the actual human retrovirus HTLV. Independent experiments quickly uncovered the XMRV error and quickly confirmed the truth of HTLV. It’s a good thing we don’t rely on peer review to do all the heavy lifting.

A boots on the ground view of the modern scientific method. ECREE is where science goes to die.

I find peer review highly valuable — but only in the same way I find collegial critiques valuable in every other step of the scientific method. What I find dangerous is when the collegial feedback comes with unilateral veto authority. I’m an enthusiastic supporter of the new publishing model at eLife, which explicitly removes veto authority from the peer review process. Obviously, I also support the veto-less environment here in Substack.

---

The extraordinary claim that the HPV vaccine is a one-and-done affair has its roots in an extraordinary hypothesis originally proposed by Bachmann and colleagues more than 30 years ago. The basic idea is that rigidly repetitive molecular structures — such as the repetitive surfaces of some types of virus particles — can reliably provide an exceptionally strong trigger for antibody-producing B cells. The exceptional performance of the HPV vaccine, which is composed of empty virus-like particles self-assembled from 360 copies of the HPV coat protein, is a spectacular fulfillment of Bachmann’s prophecy.

A repetitively arrayed nanoparticle vaccine, as conceived of by Bing Image Creator

So why don’t Covid vaccines use this proven form of magic? The HPV vaccine community has spent more than a decade doing its utmost to spread the repetitively arrayed vaccine gospel far and wide, but the message has vexingly never attracted widespread attention. To some extent, I suspect this reflects the perception that the success of the HPV vaccine is an extraordinary claim — so our audiences keep passively waiting for extraordinary evidence that the magic can be applied to other vaccines. Evidence that polyomavirus vaccines are even better than the HPV vaccine apparently still isn’t extraordinary enough.

Fortunately, the repetitively arrayed vaccine gospel has won a few converts and several second-generation Covid vaccine efforts are making effective use of the Bachmann magic. Unfortunately, funders and regulatory authorities have egregiously dropped the ball on these important efforts.

In early 2021, I attempted to put my arm where my mouth is by enrolling in a Phase I (safety) trial of a vaccine developed by my friend Kayvon Modjarrad at the Walter Reed Army Institute of Research. The Walter Reed vaccine arrays SARS-CoV-2 Spike proteins on a repetitive platform called ferritin. I was ejected from the trial when FDA officials dictated that people who had received the newly released Moderna or Pfizer vaccines couldn’t be allowed to volunteer. I found the regulatory edict incomprehensible. The key question a Phase I trial of a second-generation Covid vaccine needs to answer is whether the vaccine is safe to use as a booster dose. Excluding previously vaccinated individuals made the trial less scientifically useful — and it took many months to recruit heroically brave volunteers who were willing to refuse the approved mRNA vaccines and instead roll the dice on possibly getting a placebo in the trial. From news reports, I infer that the Walter Reed effort is currently stalled out at the stage of attracting the private venture capital required to proceed to Phase II/III (efficacy) trials. It’s deeply regrettable that Operation Warp Speed was terminated prematurely. I hope the new Project Next Gen can help get Kayvon’s team back in the saddle.

This past January, I again attempted to vote with my arm by asking to be enrolled in a Phase III trial of ABNCov-2, which uses a repetitive SpyCatcher platform Bachmann helped develop. Two weeks prior to my attempt to enroll in the trial, regulators inexplicably dictated that only people over age 65 could be eligible.

Another repetitively arrayed vaccine, SKYCovione, was designed by Neil King and David Baker at University of Washington. SKYCovione was authorized for use in South Korea in June 2022, but Korean regulators devastatingly hobbled it by only allowing it to be used as a first dose. There’s basically nobody left in that category in Korea, so only a few thousand people received the improved vaccine.

---

Although carefully conducted clinical trials have clearly demonstrated that SKYCovione is safer and more effective than first-generation Covid vaccines, the FDA continues to enforce strict prohibition on the obviously superior product. Why? I suspect the insidious hand of ECREE is at work here too. The FDA generally relies on private for-profit companies to provide proof that the medicine they wish to sell is safe and effective. As illustrated by the trials I couldn’t enroll in, the agency has the power to micromanage minute details of trial design in a secretive behind-closed-doors process that’s ultimately enforced by the agency’s power to deny licensure. The FDA has apparently judged the evidence of the SKYCovione trials to be insufficiently extraordinary — perhaps because the trials were conducted in Korea, where the agency couldn’t micromanage their development. There’s no mechanism for appealing or overriding this de facto FDA veto — or even to simply get a clear public accounting of how the decision was reached. It’s pretty much the dictionary definition of “unaccountable.”

Relying on regulatory bureaucracies to dictate vaccine decisions to us with no checks or balances has clearly been a catastrophic failure — and not just during the pandemic. We need to find a more democratic solution.

Current law requires the FDA to offer a yes/no decision either endorsing or prohibiting a medicine. We would be better off if there were also a “definite maybe” category that would enable the agency to acknowledge scientific uncertainty. The new category would make it possible to rule that a given vaccine has a well-documented safety profile and has clear evidence supporting the possibility that it realistically might be effective. People would then be allowed to choose the definite maybe product based on their own personal judgment. The new category would also be useful for things like the controversial new Alzheimer’s medicine aducanumab. But I digress. This article is about vaccines.

In essence, the FDA is only chartered to serve as a veto-wielding peer reviewer. In an ideal scenario, the FDA budget would be boosted to a level where the agency could also serve the normal error-correction function of the scientific method — by independently testing vaccine products and conducting post-market monitoring. Having a Consumer Reports-like testing program would empower people to make better evidence-based decisions about which medicines they put in their own bodies. Some consumers might make foolish decisions, but that isn’t a reasonable justification for denying everybody a fundamental form of autonomy — and it certainly isn’t a reasonable excuse for grinding the development of improved Covid vaccines to a disastrous crawl.

I hope like hell at least one of these next-generation Covid vaccines can make it through the existing yes/no regulatory gauntlet to become the one-and-done boost we desperately need. And not just because the improved vaccines will finally put an actual end to the effin’ pandemic, but also because the success should finally provide a level of extraordinary evidence that will irresistibly set a consensus high standard for all future vaccines.

It’s too bad we didn’t start betting on Bachmann’s extraordinary hypothesis 30 years ago. We would have been in an extraordinarily better place right now.

---

Update: In April 2023, a trial of a vaccine where the “stalk” domain of influenza HA is repetitively arrayed on ferritin provided yet another convincing confirmation of the breadth and durability of antibody responses to Bachmann immunogens (Widge et al). Believe it or not, I was also rejected from the ferritin-flu trial. At the entry screening, my systolic blood pressure was 127, which is pretty good for a 50 year-old man. The research nurse scared me when she told me my diastolic pressure, which was 87, indicated I was too unhealthy to meet FDA guidelines for participation in the trial. To my relief, a quick internet search revealed the guidelines are nonsense. One of the top hits is a review entitled, “Is systolic blood pressure all that matters? Yes.”

I guess it might sound like I’m idly complaining about a series of trivial personal disappointments, but the fact that I’ve been bounced from three different vaccine trials for three different BS reasons over the past three years is telling us something. Nonsensical enrollment criteria needlessly retard vaccine development. Every day a couple hundred Americans die of Covid. This isn’t just the trivial whining of one disappointed scientist.