What’s the math behind proving a vaccine works? on the news they said double blind study, do they just say it works if less people get infected?


I was listening to the radio and they were talking about covid vaccines undergoing trials and depending on the result they do or don’t get approved. How do they tell if it’s good. Do they just use a null hypothesis test? are there other options?

In: Mathematics

A double blind study just means that nobody-not even the people running the experiment- know which patients are getting the actual vaccine. This is meant to eliminate any chance of bias in the data.

In a nutshell what a drug study tries to do is test a drug candidate (like a vaccine) in a group or multiple groups of patients that are representative of the general population. Then you wait to see if a statistically significant fraction of them getting the vaccine/drug do not contract the disease, or meet other markers such as blood antibody levels that indicate the response you’re looking for. These endpoints are all determined ahead of time and are used to judge the efficacy of the experiment.

This is an extremely complex process- drug trials can go on for years and involve thousands of patients before being approved for use.

If you have a group of 20 000 people and you select people by random so 10 000 get the vaccine and 10 000 only get a placebo likely a saline injection. You then instruct them to behave as normal.
Wait some time lest say 3 months and then compare the number that got the infection in each group.


If zero that got the vaccine got infected and 500 in the placebo group get the infection then you can be quite sure taht the vaccine works.

If 50 that got the vaccine to get infected and 500 in the placebo group you can still be quite sure it is very effective as the infection rate is 1/10

If you 500 that got the vaccine to get infected and 500 in the placebo group you can be quite sure that the vaccine has no effect.

Even if 250 that go the vaccine got infected the vaccine still has an effect. But at some point even if the number is nor equal the effet start to approach zero.

It is all based on the idea if you select people by random it is likely that with no vaccine both groups would be infected at the same rate. It would not be 500 and 500 but perhaps 463 and 532 so you need to do a real statistical evaluation of the results.

The large test also has another fundamental function and that is to detect any side effects of the vaccine. That is a hard part because people that got the vaccine will be diagnosed with a new medical condition and so will the placebo group because we are humans and humans get sick. So you need to determine that it do not happen to a higher degree than normal.

All medical intervention has some risk so the result is an evaluation of side effects vs how effective the vaccine is and risk from the infection if no vaccine is used. So it is a risk vs benefit analysis.

If the vaccine would make everyone that got it immune to the infection but 1% died that got inkjet it is as dangerous as the infection and would never be used.
If there is almost no serious side effect but only 30% get immune it might be a good idea to use if there is not a better vaccine.

In a double blind study, there are people who get the vaccine and people who get a placebo. The decision of who gets which is completely random. Not even the people administering it know.

The reason for this is that they want to control for all other factors. If they gave a placebo but told people what it was, they might not take the same precautions going about their daily lives as if they were told they got the real thing. As a result, you might see the people with the placebo taking less risk, thus catching Covid less naturally, and polluting the placebo test.

The brunt of what you are asking is done, like you said by statistics and the likelihood the results were seen due to chance (lower is better, and generally less than 5% chance is accepted). Further, you could quantify your results (and then use statistics) by saying what percentage reduction in infection do we see –> this is how we might determine the efficacy of the drug, or what percent reduction in severity, and so on.

However, another major reason for these larger trials is to test for their safety, to find uncommon side effects, as smaller trials are sufficient to determine efficacy (or at least good enough efficacy for the vaccine to be worth it. What I mean by this is if in a trial of 40 people, you can’t tell if the vaccine is efficacious, it means it has a low efficacy and then it’s probably not worth it).