29

I came across a study that makes the following claim (emphasis mine):

Two doses (300 μg/kg/dose in a gap of 72 hours) of ivermectin chemoprophylaxis reduced COVID-19 infection by 83% among HCWs for one month. Ivermectin is a safe and effective strategy to prevent COVID-19, in the containment of pandemic alongside vaccine. Further research is required to guide the frequency of chemoprevention, acceptability, and cost-effectiveness in the community setting.

Is this a robust study and are the conclusions supported by the data?

F1Krazy
  • 3,722
  • 5
  • 21
  • 30
John
  • 407
  • 1
  • 4
  • 4
  • 25
    This seems to have no blinding, such that the subjects actually choose whether to receive none, 1 or 2 doses. I think this qualifies it as not robust. – User65535 Feb 01 '22 at 12:55
  • 21
    Cureus seems to be well-regarded by some as a publication where peer review and publication are quick and trouble free for publishers. However, the peer review process has been compared to "crowdsourcing," with a window of a few days where readers comment on it (anyone can join in). That's not particularly robust. It has an impact rating of 1.15, on a scale where 10 is excellent and 3 is considered adequate. – Cristobol Polychronopolis Feb 01 '22 at 15:38
  • 5
    @User65535: I'm going to tell you a story about blinding. It became necessary to test whether or not the measles vaccine caused autism. This was done without blinding on purpose. The test was done by gathering the entire data on the population of Wales for who was vaccinated and who wasn't and who got autism and found and publish a small negative correlation: if anything, the measles vaccine prevents autism. Why not blind? Easy. You chose to participate in the control group by not getting the measles vaccine. They didn't sample. They used the whole population. – Joshua Feb 02 '22 at 01:09
  • 16
    @Joshua That tells you that the the measles vaccine and autism have a correlation, it tells you nothing about the direction of causality. It could be that the measles vaccine prevents autism, or that autistic people are more likely to avoid the vaccine, or that there is a genetic component and parents of children who become autistic are less likely to get their children vaccinated. – User65535 Feb 02 '22 at 09:11
  • 4
    We can only judge whether ivermectin has a notable effect by looking at all the studies and carefully considering the detailed methods and results. Whether this study says ivermectin works is almost irrelevant. – matt_black Feb 02 '22 at 12:59
  • 12
    There is also the theory floating around that all the studies that showed positive effects are in places where parasites are somewhat common, so that effect could be attributed to the Ivermectin clearing out parasites that the patients unknowingly had, which would reduce the stress on their immune system and lead to a better outcome. – Simon Richter Feb 02 '22 at 16:26

2 Answers2

57

Summary: The conclusions are not generally applicable because the study did not isolate the role of ivermectin in the outcomes, and the process was not a randomized controlled trial.

First, this August 2021 report of a September-to-November 2020 study the sort of study that suggests "this may be promising, more research is needed." As the authors note, "Further research is required to guide the frequency of chemoprevention, acceptability, and cost-effectiveness in the community setting." (Result so far have been dubious: see Meta-Analyses Do Not Establish Improved Mortality With Ivermectin Use in COVID-19 in the Jan.-Feb. 2022 issue of American Journal of Therapeutics)

This study of 3,532 doctors, nurses, students, administrators and other staff at All India Institute of Medical Sciences (AIIMS) Bhubaneswar has some shortcomings that undercut its conclusiveness, some as noted in the comments above:

  • Administration of the ivermectin was not double-blind and was not randomized. Drug studies typically compare the results of two randomly-assigned populations, one taking the drug, the other taking a placebo or other control, with neither the researchers nor the participants knowing which they were getting. In this case, not only did the participants know they were taking the drug, they were also the people who chose to take the drug. For example, the authors note that "Ivermectin prophylaxis uptake was better with increasing age and among males."
  • We don't know how the behavior of the subject group differed from the general population given that they were people who chose to take ivermectin, or how their behavior might have changed knowing they were taking ivermectin or by participating in the study. They may have taken more or fewer risks to avoid infection, or otherwise altered their activities, their diet, or their other medication. The authors suggest "we had a strong institutional policy in place related to COVID-19-appropriate behavior in the workplace, which may have avoided the possible bias," but we don't know if that policy was enforced; there have been many studies about doctors misestimating how often they wash their hands, for example. There were also reports of ivermectin side effects such as nausea and headache among 42 participants; while side effects are commonly over-reported, we don't know if those drug-takers were so sick they missed work and had reduced exposure, for example.
  • We don't have conclusive evidence about how many health care workers (HCWs) in the overall study were infected. The authors note: "The major limitation is that we only tested HCWs who either developed symptoms or who were direct or high-risk contacts of positive patients. This was done in keeping with the Government strategy for COVID-19 testing in India. However, this precludes us from including the HCWs who may have been asymptomatic or mildly symptomatic and chose not to get tested."
  • Some unknown number of the hospital employees had already contracted covid in September 2020, which is what prompted the study. "We noticed an increasing number of HCWs getting infected with SARS-CoV-2 infection in early September 2020 at our hospital, which was negatively impacting the healthcare services we had to provide." We don't know how many of those employees were back at the hospital or how their participation in the study, other behavior, or chance of getting reinfected might have been affected by their personal experiences.
jeffronicus
  • 5,427
  • 3
  • 31
  • 27
  • 54
    "Later higher-quality studies and meta-analyses indeed suggested little to no benefit from taking ivermectin, so I won't rehash that information." I think it would be worth rehashing this because a lot of people believe it anyway, for other reasons. – Laurel Feb 01 '22 at 17:40
  • 5
    @Laurel Adding a link to a fairly critical new overview of the current set of meta-analyses of ivermectin. – jeffronicus Feb 01 '22 at 18:06
  • 2
    Thanks, @jeffronicus Appreciate you taking the time to do this. – John Feb 01 '22 at 20:33
  • 15
    Also worth emphasizing, the employees who took ivermectin did so as part of the hospital's response to rising covid infections; it seems likely that the people willing to take an experimental treatment to avoid infection (two-thirds of the staff) would have been a part of the employee population more inclined to take extra action to reduce their risks. Presumably handing out ivermectin wasn't the only thing the hospital was doing that was different; it was just the only thing the hospital was studying. – jeffronicus Feb 01 '22 at 23:11
  • 1
    @Laurel: The meta-analysis is completely broken, but the answer still is No! There is no possibility that ivermectin prevents infection. This is the astounding world we live in: over half of peer reviewed research papers are wrong. – Joshua Feb 02 '22 at 01:01
  • 7
    "this may be promising, more research is needed." This phrase is found in almost every publication. It does not necessarily mean what laymen might understand with it, but more like 'we keep working on this'. – pinegulf Feb 02 '22 at 07:07
  • Minor quibble - "the other taking a placebo" - this is not necessarily accurate; it could be they compare those taking the drug to those using some other drug - often it is compared to the "standard of care", for example in cancer drug trials they're not comparing to nothing/placebo, but they're comparing to people using the regular chemotherapy/etc. that is normally used. – Joe Feb 02 '22 at 17:57
  • It's also notable that 3,532 is a fairly small sample size. Normally a study will have tens of thousands of participants to make sure the results are as accurate as possible. Also, with the wide variety of exposure the employees saw, it could skew the results with an admin not being at risk as much as a surgeon or nurse. Not to mention that many could have had immunity from already having had covid previously, which prompted this study. It's also possible that participants under reported symptoms in order to help "prove" it worked, since it wasn't a blinded study. – computercarguy Feb 03 '22 at 19:33
  • This study doesn't even mention how many people were vaccinated or compare the results of vaccinated vs unvaccinated rates of infection. It only mentions vaccines in the conclusion, so it stating that Ivermectin works in conjunction with the vaccine is a pretty far stretch. India is still only 52% vaccinated. While vaccination rates might be higher specifically for HCWs, it's still not mandated for anyone, so with the study not mentioning vaccination rates or even using the word vaccine until the conclusion means there's significant data missing. – computercarguy Feb 03 '22 at 19:41
  • 1
    @computercarguy The study was conducted a few months before vaccines became available in India in January 2021. – jeffronicus Feb 03 '22 at 21:50
  • I appreciate the level of detail in this post, but indeed it is long. Can you give a 1-line summary at the top? – user1271772 Feb 03 '22 at 22:00
  • 1
    How does point 4 affect the study, when both employees who took and didnt take the drug could have been previously infected? – Pablo Feb 03 '22 at 22:22
  • @user1271772 Added. – jeffronicus Feb 03 '22 at 22:42
  • @Pablo That's not clear. The chart of cases among HCWs they includes shows about 250 cases among employees in the two weeks before ivermectin distribution began, and another 150 or so in the two weeks afterward. Whether those employees were quarantined, treated at the hospital or returned to work and/or received ivermectin isn't clear. – jeffronicus Feb 03 '22 at 22:53
  • @jeffronicus nice! So basically "the study did not isolate the role of ivermectin in the outcomes" which means that the 83% risk reduction could have been due to things other than ivermectin? – user1271772 Feb 03 '22 at 23:05
  • @user1271772 I believe so. The authors say the hospital did not use a randomized controlled trial -- which would have isolated those effects by giving only part of the subject group the real medication -- due to ethical concerns, presumably over potentially withholding helpful medication. But if you are sufficiently motivated -- by fear, by hospital guidance, whatever -- to take the medication that your institution's own doctors thinks is helpful, what else will you be willing to do differently? Was this the only policy change or practice the hospital implemented at the time to combat covid? – jeffronicus Feb 03 '22 at 23:36
  • @jeffronicus Good point, I wouldn't be surprised if a hospital implemented more than one policy change to help combat COVID. Probably people were in general more careful (wearing masks, keeping good airflow in the rooms, and minimizing close-contact?) in the hospital, which would reduce COVID transmission, and these are factors that are hard to isolate from any study on the effectiveness of something at reducing COVID risk? I haven't read the article yet, nor much more on the topic, as this week has been the busiest in a long time, hence why I asked you for the 1-line summary :) – user1271772 Feb 03 '22 at 23:50
  • @jeffronicus, considering when the vaccine was developed compared to this study, you'd think they would have at least mentioned that it hadn't been used in India yet. IMO that would clarify some of the unknowns in the study and makes me think this is another reason why the study wasn't well done, or at least well documented, due to all the missing data points. – computercarguy Feb 04 '22 at 00:00
22

This recent Cochrane review did consider the Behera study mentioned by the OP, but rejected it on the grounds that it wasn't an RCT (randomised controlled trial), along with eight other studies for the same reason.

Out of interest, there are other reviews of ivermectin for covid. This one is from last year, and this one is very recent. Both found significant issues in some of the studies they considered.

The first two reviews I linked to were mentioned by @jeffronicus in their excellent answer. However, the third wasn't, and maybe the most relevant finding in that review was how strongly the effect of ivermectin decreased as more problematic studies were excluded from the meta-analysis.

See Figure 1 here, which I've also posted below. The effect of ivermectin is strong when all studies are included, but the p-value is very low (0.01). However, when studies with any degree of concern are excluded, the effect of ivermectin is marginal (a 4% increase in terms of improvement in survival, i.e. 1.04, 95% CI, 0.56–1.66; p = 0.90).

Edit: there is also the PRINCIPLE trial, run by the University of Oxford. They are currently investigating ivermectin for covid treatment. For context, they're also testing favipiravir and the "usual standard of NHS care". They've already looked at inhaled budesonide, azithromycin, doxycycline and colchicine, of which only budesonide showed any benefit ("shortens recovery time by a median of three days in patients with COVID-19 who are at higher risk of more severe illness and are treated in the community").

Mentioning this as ivermectin results will presumably be released in the near future.

enter image description here

Edit 2: the results of the PRINCIPLE trial for ivermectin have been published (as a preprint, as of 04/04/24), see here. Ivermectin is BS.

Here's a summary: "In this largely vaccinated population of participants with confirmed cases of COVID-19, there was no differences in hospital admission, a modest reduction in first-reported time to recovery (from 16 to 14 days), and no impact on work or studies at three, six and 12 months. The analysis included 2,157 participants receiving ivermectin plus usual NHS care, compared with 3,256 participants receiving usual NHS care alone."

paddyr
  • 482
  • 3
  • 7
  • 3
    If you are going to refer to the higher quality reviews of ivermectin, it would be worth quoting their explicit conclusions on what the reliable evidence says. This would be a useful answer to the question which is far broader than the reliability of a single paper. – matt_black Feb 02 '22 at 12:21
  • 2
    @matt_black: Thanks, good point, I've made an edit to reflect your suggestion. – paddyr Feb 02 '22 at 12:47
  • 1
    Worth stating that the Cochrane review you link to considered ivermectin both for treatment and for prevention. On the latter it says "Ivermectin compared to no treatment for prevention of SARS‐CoV‐2 infection: We found one study. Mortality up to 28 days was the only outcome eligible for primary analysis. We are uncertain whether ivermectin reduces or increases mortality compared to no treatment" – Henry Feb 03 '22 at 11:42
  • 3
    This is a (common) misuse of the p-value. The p-value is the probability of observing the data/observations given your hypothesis is true. NOT the probability your hypothesis is true given the data (if you want this you need to convert using Bayes rule). – Cole Feb 03 '22 at 12:11
  • 3
    This tells us the low-bias studies do report anything that would be surprising if ivermectin had no effect, NOT the probability that ivermectin has no effect given the clinical data. (e.g. P(street is wet | raining) =/= P(raining | street is wet) (| = "given"), one is ~100% the other likely depends on how often your neighbor runs their sprinklers) This is somewhat a nitpick but I thought it was worth pointing out. – Cole Feb 03 '22 at 12:17
  • @Cole: Not nit-picking at all, thanks! I am not qualified to comment, please feel free to edit my answer. – paddyr Feb 03 '22 at 12:24
  • Two years later, the mentioned PRINCIPLE Trial doesn't seem to have published about ivermectin. – Oddthinking Feb 20 '24 at 15:04