Playback speed
×
Share post
Share post at current time
0:00
/
0:00
12

The Gross Misapplication of Evidentiary Standards

How the rigid formulas for assessing evidence defy common sense and corrupt science
12

The main reason for rewriting this is that it (hopefully) conveys a useful skill, to be able to unwind and identify the arguments and propaganda elements in an article that superficially looks legitimate and objective.

One of the things I am most frequently asked about by people through my social circles, is some form of “please explain the following article” that seems to make a compelling case for one of the establishment’s narrative assertions.

I don’t know how representative of people my experience is generally, but it is a critical skillset to have regardless1.

I am indebted to Dr. Kory for giving me a boost “out of the gate” by strongly endorsing it on the weekly FLCCC zoom (above video).

(I apologize to the approximately 30 of you who already read last year’s version, please don’t hold it against me 😅)


Someone sent me an article entitled Ivermectin, COVID-19, and making sense of scientific evidence, by science writer Liam Mannix. The central goal of the article was to provide a framework for analyzing scientific evidence and apply it to the studies and data regarding Ivermectin. Mr. Mannix of course concluded that the evidence base for Ivermectin is essentially a collection of unreliable and corrupted studies/trials.

His article however was well written, and comes across as fairly convincing and tightly argued, a lethal combination for a propaganda piece. (The lack of malicious intent by the author would not change the objective character of his article.)

Additionally, I think that this article succeeds in articulating the central epistemological axioms of the mainstream academic community (at least for the portion of whom still care about intellectual honesty despite their massive blind spot to the corruption of the scientific community). This provides the opportunity to rebut the entire edifice underlying their methodology for determining factual truth, specifically regarding assessing the reliability of source material/data/studies.

One note before we begin: obviously, there are some premises that I am assuming are true a priori, such as the endemic corruption of the scientific community to some degree, that Liam Mannix would not grant. However, the primary purpose in writing this response is to provide rational people with an explanation of how those of us who are staunchly and unreservedly pro-Ivermectin understand the evidence base, and to expose the logical flaws in his arguments2.

Below is a paragraph-by-paragraph rebuttal of the entire article. I recommend first reading the article straight through in full, because it will give you a better sense for how it comes across and how it superficially seems to be very well argued and very compelling.

The assertion that “nearly all the evidence on Ivermectin has been of such low quality etc” is categorically false. This will be dealt with along the way.

A Cochrane review is not some magical method for determining absolute truth in a way that is free from human judgement, which is the only truly pertinent fact here - anything subject to human judgement is by definition subject to human error, and corruption.

A Cochrane review essentially is a evidence review that follows a specific collection of standards and tests dictating the evidentiary parameters we apply to characterize a data point or collection of data based on attributes such as the source’s size, sampling, biases, protocols, and so on. It also provides a series of statistical methodologies by which one can assign a value to data or data sets, and combine data from different sources. In simple language, a model for ranking evidence.

There are at least two glaring flaws in this model:

  1. Ultimately, the application of Cochrane standards relies upon the research ability/skill and personal integrity of whomever is performing the review to accurately document all of the pertinent characteristics. Obviously, a failure to do so will compromise the results of the review. To take an obvious but extreme example, suppose an RCT was fraudulently conducted but written up in a manner that successfully eliminates any trace of the fraudulent activity? Continuing with our ‘absurd’ illustration, suppose they gave the drug being tested to the placebo group as well in order to ensure that the trial fails to observe a benefit in the tested drug. Obviously, a reviewer’s ignorance of this will lead him to categorize the study as high-certainty evidence per the Cochrane review standards.

  2. Cochrane reviews stipulate a hierarchy of evidence that even if it is generally able to accurately capture an effect’s significance and certainty (a bold assumption), it does so by assigning values to characteristics that are often situationally unwarranted. This method therefore can sometimes falsely portray evidence as either weaker or stronger than it is. The prime example of this is probably the consideration of RCT’s as the ultimate “gold-standard” of studies, but there are plenty of other assumptions by the Cochrane standards that are similarly unsound or unpredicated. To put this more succinctly: any formal, rigid formula used to assess evidence by definition lacks the flexibility to cope with situations where the available evidence will not conform to the exacting specifications demanded by such standards, even though the evidence might be obviously compelling and demonstrative to anyone with common sense.

These flaws, when combined with broad systemic corruption in the scientific community, result in a cult-like insistence of ignoring an avalanche of real-world experience & evidence in favor of following the officially prescribed method of evidence assessment, which is what has occurred regarding Ivermectin.

Here is a “theoretical” example to illustrate this:

Suppose there was a pandemic where the world was caught flat-footed. Doctors and scientists scrambling to find an effective treatment rapidly began testing different drugs to see if any seemed to be of help. Harried doctors on the front lines, noticing a potential signal of a potential drug that might be efficacious, quickly organize ad hoc trials, where they essentially give a group of people, usually either health care workers or patients, the option to take or refuse the new drug. These trials show insanely huge reduction in mortality, disease severity and prophylaxis. As word starts to spread around the world, a few countries engage in mass distribution of this drug to a substantial portion, or all, of their population, with a reproducible tight correlation between mass distribution and near eradication of the pandemic virus.

None of the evidence in this “hypothetical” would be acknowledged by a formal, proper Cochrane review, because these are either “low quality” studies, or determined and judged to be nullities, due to things like a lack of adequate procedural controls, proper pre-registration, and similar technical specifications that are necessary to dot all the “i’s” and cross all the “T’s”.

Thus the phenomenon of meta-analyses claiming to survey the entire evidence base that somehow omit most of the studies from the meta-analysis of “all” the evidence.

With that introduction, let’s turn to the substance of this article.

Mannix comically claims that being empaneled by a government is somehow a positive attribute vis-à-vis adjudicating the panel’s credibility, regarding a controversial political topic no less. A government panel is controlled by the government, either directly or indirectly via some combination of legal, social, financial, and professional peer pressures, usually all of the above. Thus, you can be sure that the governments wishes and priorities will not be lightly disregarded.

Next, Mannix claims that such scientists are “independent”. The virtue of independence is meant to say that the scientist is unencumbered by any external influences in performing his scientific analysis. The most powerful corrupting influences on scientific enquiry emanate from the scientific community itself in addition to the government as mentioned above. (Even if the panel had not been assembled by the government, the government would still hold tremendous clout over its activities.) Thus, at a minimum, unless a scientist is demonstrably acting independently of the scientific establishment he is presumed to be not independent of the crushing peer/social/govt pressure as is the case here, where in addition to the default scientific community pressure, the govt itself constituted the panel and decides what they “discover”.

That this government-constituted panel rejected a drug purchased by the government as a potential useful treatment is wholly irrelevant. There is no political cost to having purchased a drug during the chaos of a novel pandemic, even for a sizable cost, that maybe could have worked out. If anything, the citizenry - especially a citizenry already whipped into a panicked frenzy - would tend to appreciate that the govt is acting so proactively and not feel that the govt had acted irresponsibly. In any event, absurd and unreasonable govt expenditures are common and routine phenomena that neither excite the passion nor provoke the ire of the population3.

Ivermectin, on the other hand, is politically charged dynamite, as anyone remotely familiar with the current political climate and discourse surrounding Ivermectin readily appreciates.

Representation from across the medical spectrum is likewise irrelevant because the medical establishment itself is a primary corrupting influence on the scientific process. This is like saying we have “broad representation from across the spectrum of mafia & crime families” on our expert crime panel (and at least they are actual experts on the matter too). The benefit of diversity on the panel is that we get the unique input from various fields and disciplines. However, such a benefit vanishes when the ‘diverse’ array of experts are all beholden to the same corrupt influence of institutional science and medicine that will compel them to toe the line and NOT contribute their genuine unique insights.

And as stated already, everyone working under the aegis of the govt is equally subject to the corrupting whims and diktats of the politicians and bureaucrats who wield its powers. Given the political dynamics of the pandemic, it is highly likely that these ‘diverse’ members were selected primarily because they are known to be compliant and in consonance with the agendas of the dominant political factions. There is no rational basis to simply presume that they were selected because they would fearlessly articulate the highest quality expert judgement in the process of adjudicating the merits of Ivermectin (or any other potential game changing treatment) without regard for any other considerations.

This claim is ultimately only useful insofar as representative of the delusional naiveté of Mr. Mannix.

Mannix is trying to suggest that anonymity is indicative of low-quality and possibly some amount of fraud. Unfortunately, in the current political climate anonymity is necessary to protect dissenting scientists or doctors from career/professional assassination at the hands of their colleagues, govt and/or the media. Thus, contrary to what Mannix is claiming here, anonymity does not inherently imply anything about the quality of the work, as both legitimate and illegitimate actors would publish anonymously.

Their work, however, speaks for itself. If you look at their website, you can find a massive treasure trove of incredibly sophisticated analyses of the numerous trials, far more robust than anything cited by Mannix in his article. Even more compelling are the genuine world-class medical pioneers who are the driving academic force behind the Ivermectin campaign, and who hold this website and its creators in high esteem.

Mannix’s argument here suffers from a far more profound flaw though. Mannix is essentially claiming that the C19early website is the primary expert authority for the use of Ivermectin to treat covid. This is akin to attributing special relativity to some middling college physics professor while ignoring Einstein and claiming that the primary authority behind the theory is the no-name professor, and then citing the rando professor’s lack of credentials to suggest that relativity is therefore probably junk science because the primary authority espousing relativity is a nobody without any sort of reliable credentials. The FLCCC, comprised of unimpeachably credentialed brilliant medical luminaries, presents a thorough overview and analysis of the evidence base and protocols for Ivermectin on their website. It is doctors such as these who are the primary expert authority for Ivermectin.

The conundrum posed by Mannix is really just a question of which experts should you trust: a govt empaneled committee who are essentially analyzing from comfortable perches in their ivory towers, who face no real threat of sanction for denying Ivermectin’s efficacy but face very real professional perils should they buck the medical/scientific/govt establishment; or a group of brilliant doctors who have put their careers, social lives and reputations on the line while successfully using Ivermectin for a year and treated tens of thousands of patients directly and consulted for doctors worldwide who have collectively treated millions. Not much of a choice when you frame it honestly and accurately.

If you look at their references for their Ivermectin research, you will find cited Lopez-Medina et al. This study literally engaged in deliberate scientific fraud. A panel dedicated specifically and solely to grading quality of evidence that cites an obviously fraudulent study lacks even a semblance of credibility to assess evidence. Period. Their reference list is highly problematic for other reasons as well. It is difficult to discern objective criteria informing this panel’s methodology for selecting which studies to include vs exclude if you would actually examine the various Ivermectin studies in detail (Alexandros Marinos has written a number of pieces doing just that on his substack).

The one positive I can think of regarding the curious pattern of included studies is that at least they aren’t citing the Roman et al meta-analysis, which simply lied about the results of some of the studies whose results they were allegedly analyzing.

The FLCCC doctors, in a few of their weekly updates, explained the ins and outs of some of the ridiculous anti- studies. You can listen to an explanation of the evidence base from Dr. Kory here (surprisingly, YouTube has still not removed this video in late 2022), and from expert WHO consultant Dr. Tess Lawrie here.

Mr. Mannix conveniently ignores every outrageous anti-Ivermectin study - of which there are no shortage - while nitpicking on the couple of pro-Ivermectin studies that are insignificant in the context of the entirety of the evidence supporting Ivermectin.

Worse, he cites the Surgisphere fiasco, where the world’s most prestigious journal published a “peer-reviewed” sham study in order to sabotage Hydroxychloroquine, whose data was never reviewed at all (and was quite literally fabricated out of thin air), and had enough red flags to paralyze a rampaging bull with joy.

A word of advice to Mr. Mannix: Invoking the Surgisphere fraud - while certainly entertaining - is probably ill advised for reasons that should be unnecessary to elaborate upon. Suffice it to say that it would seem difficult to instill confidence in the scientific establishment by irrefutably proving the entanglement and corruption by political considerations of the peer-review and editorial processes.

The Elgazzi et al Egyptian study that was retracted following serious allegations of fraud was withdrawn by the pre-print server without affording its lead author the chance to respond to the allegations, and who Dr. Kory reported was disputing these accusations as manipulative and false, so the jury is still out on this one.

And even if the Elgazzi study turns out to be a fraudulent study, that does not affect the dozens of other studies showing clear and convincing evidence of Ivermectin’s efficacy, and most certainly is moot in the face of the clinical firsthand experience of thousands of doctors worldwide who have been successfully using Ivermectin as a covid treatment for millions of people (e.g. see: Indian state of Uttar Pradesh).

You know, like the journal that published the aforementioned unmitigated Surgisphere fraud. Points for sophisticated discernment?

In any event, this argument is rubbish, because one of the cornerstones of the current debate is the medical community’s censorship of dissenting opinions from mainstream journals and publications. Obviously, if the top journals simply refuse to publish any papers that would place the political narrative in serious jeopardy, you cannot adjudicate the credibility of a paper by where its published (or more to the point, where it is not published).

Furthermore, the simple fact of the matter is that the bigger or more prominent a journal is, the more it is financially dependent on Pharma and government, and thus must conform to the agendas of both.

A more viable approach to assess the credibility of a study by its provenance would be to look at the authors instead of the journal – if they are highly credentialed and mainstream according to a Google search starting from before March 2020 and are not conforming to the establishment narratives, then they carry far more weight than those who do not meet these criteria4.

Ultimately, there is no substitute for sensible judgement, which is an intangible quality that some people have and some people don’t.

The notion that one big, grand RCT is superior to a number of small trials is functionally illiterate. A collection of small trials all showing the same result is itself extremely high-powered evidence, because the chance of running, say, 30 independent trials - especially trials with widely ranging characteristics - and observing the same results are so infinitesimal as to be negligible. 30 small, individually underpowered studies are far more conclusive than one large trial, because while 30 trials with widely ranging characteristics can largely negate each other’s (potential) biases, one large study cannot overcome its own design flaws or outright corruption. (Yes, RCT’s can be designed quite horrifically while following every rule and standard, something which cannot be emphasized enough.)

This is emphatically the case regarding Ivermectin, although you would have to look at the individual studies to get a sense of this. Something that oh by the way the very website Mannix maliciously attacks makes very easy to do - i.e. lists all of the studies with a summary of the basic results intelligible to a layperson.

This is rather ironic, as the Dexamethasone arm of the trial referred to here was designed so poorly that the FLCCC doctors were scared that it would show negative results (such as extremely low dosing. I don’t remember all of the other criticisms off the top of my head but there were others mentioned in one of the weekly FLCCC updates).

This trial also led to the official codification and subsequent mindless adoption of the precise regiment of the trial – exact same ridiculously low dose first utilized at a later stage of the runaway inflammation than was reasonable to wait for, only Dexamethasone but no other steroids, etc – the FLCCC docs discovered long ago, for instance, that the steroid methylprednisolone was a noticeably superior alternative and at much higher doses. And a recent5 study by Dr. Robert Malone indicated that NSAID’s may be superior to steroids.

As to the parenthetically enclosed observation, this is again ridiculous naiveté. When was the last time you heard of mass distribution of Dexamethasone?? Exactly. Dexamethasone’s efficacy, especially at the comical under-dosing used in the Oxford trial, is not exactly anything stupendous. The reason that Pharma had no problem with Dexamethasone is that Dexamethasone does not pose a threat to the viability of vaccines or expensive anti-virals, since it is a mildly effective treatment only available in hospitals as a late stage intervention. Daniel Horowitz (Blaze) has done a spectacular job documenting the establishment’s war on effective treatments.

C19early.com is a compilation of every study for a number of potential covid treatment drugs including Ivermectin, by default ranked chronologically. They include every study run, regardless of the results, and regardless of the statistical significance. (They still have up the fraudulent Lopez-Medina and Roman studies for instance.) Their list is not designed to exclude/censor low quality garbage.

Mr. Mannix is ultimately being ridiculous, a competent person easily understands the nature and purposes of the website, and would use a little judgement and certainly not cherry pick the first study on a chronologically organized list as representative of all the studies on the matter.

Furthermore, as previously stated, a number of low-quality studies taken together provides quite robust evidence in sufficient numbers, because the odds of practically all the studies show efficacy if there wasn’t any are so remote as to be wholly unreasonable.

There is also a marked tendency by adherents of “evidence based medicine” (definitely deserves the quotation marks) to label any study that was run somewhat haphazardly as entirely devoid of any evidentiary value. This is, to put bluntly, moronic and disingenuous. Someone treating ill covid patients doesn’t have time to properly randomize sorting into evenly propensity score matched groups and all of the other minutiae necessary to run a “proper” RCT for a disease that will either start recovery or turn towards demise within a week or two. Delaying the start of treatment in any event increases the risk that treatment will not be effective, which is both highly unethical and reduces the chance of seeing a statistically significant result especially when you have a low study subject population to begin with.

As if we needed Mr. Linnix to expose his complete illiteracy any further. . .

Mr. Meyerowitz-Katz, for the record, seems to pop up in literally every media hit piece against Ivermectin. He is at best just another typical “expert” mindlessly toting the virtues of the official academic processes.

Anyway, this is no “masterpiece of science”. This is a pathetically designed trial so brutally horrific that the BIRD group penned a letter where they characterized the trial design as follows:

In conclusion, it is our strong belief that The PRINCIPLE Trial is a non-essential, poorly designed study that will lead to a harvest of unreliable data concerning the utility of Ivermectin in COVID-19. Any further delays in getting safe, effective, early treatments to patients will result in additional needless illness and death.

Conclusion:

The Ivermectin evidence base, ranked in order of quality (ie, most improbable results if Ivermectin doesn't work), is more accurately roughly organized as follows:

  1. The clinical experience of thousands of doctors worldwide successfully treating patients with Ivermectin & Ivermectin centered protocols. This includes a spate of court cases where judges ordered hospitals to administer (or not impede administration of by 3rd party physician of) Ivermectin to already ventilated patients for whom their hospital had no other treatments to offer and who all subsequently recovered (with the notable exception of the Mt Sinai case where the patient died amidst the inconsistently administered Ivermectin during the court wrangling).

  2. The countries that mass distributed Ivermectin to part or all of their population with tightly correlated and reproducible “flattening the curve” of every covid metric (like Mexico, Peru, India, just to name a few), some of which had a control group of an untreated population either concurrently or because of a subsequent change of policy.

  3. The numerous prospective control trials, both random and unrandom, showing Ivermectin’s profound efficacy6.

  4. The small, individually underpowered retrospective observation trials.

  5. In-vitro/in-silico demonstrations of Ivermectin wiping out covid, and everything else not mentioned above.

On the other hand, the Cochrane standards:

  • reject #1 outright,

  • severely undervalue #2 because they are lacking one or more technical requirements,

  • assign “low weight” to almost all of the studies in #3 for not following proper academic protocol or for (alleged) “risk of biases”,

  • and basically exclude #4/5 entirely as “noise”;

  • while simultaneously incorporating straight up fraudulent studies like Lopez-Medina; poorly designed studies that dose too low, treat too late, use a demographic that already widely uses Ivermectin; and meta-analyses that are rigged by excluding all of the above while including all the aforementioned poorly designed junk studies;

  • and have no mechanism for assessing the political and financial corruption of the academic process.

I think that sums it up quite nicely.

Resisting the Intellectual Illiteratti is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

1

There are two dimensions to achieving proficiency in unwinding propagandistic arguments that adroitly manipulate facts, logic and emotions to warp the objective situation into something completely disconnected but that superficially feels impossible to pinpoint where and what the deceptions are:

  1. The technical analytical skills to parse language and arguments & break them down into their constituent parts, which exposes the illegitimate pieces and identifies how they are being deployed within the argument.

  2. Accustoming yourself to instinctively apply these analytical “tools” - if you’re not used to it, it will probably take time to feel natural and easy doing so. “Practice makes perfect”, as the saying goes, and even if not perfect, practice is pretty crucial to improvement.

2

I cannot deal with every single nuance, inference, etc. here, and also probably missed a few regardless.

3

Just look at the absurd things routinely funded by our Federal government, such as jacking up hamsters on steroids and making them fight each other, or the truly edifying paper “Cocaine induces state-dependent learning of sexual conditioning in male Japanese quail”, and all sorts of fascinating science experiments that enrich scientific institutions and researchers but do not improve the lives of the citizens.

4

This does not mean that non-mainstream doctors who were outside the mainstream prior to the pandemic are presumptively illegitimate, just that even per the basic assumptions of the establishment, it is reasonable to place higher credibility in acknowledged mainstream doctors who are bucking the covid narrative. If anything, one thing we have learned from the covid saga is that the ‘mainstream’ has a nasty habit of marginalizing any dissenters and portraying them as delusional quacks.

5

It was recent when I first wrote this

6

I personally remember feeling a feeling “Whoa” when I first saw the Carvallo et al study, an unrandom control prospective prophylaxis study (the subjects chose whether to accept Ivermectin by choice). This was definitely compelling – 237/400 or so in the control contracted covid vs 0/788 on Ivermectin – ZERO!! – a result so lopsided and stunning that cannot possibly be chalked up to chance, biases or any other nonsensical pathetic attempt to invalidate the study. (The treatment arm also was using a nasal iota-carrageenan viricidal disinfectant (similar to the Povidone-Iodine suggested by the FLCCC), so the effect is really Ivermectin + Nasal spray.)

12 Comments
Resisting the Intellectual Illiteratti
Logic Checking the Fact Checkers
"Fact Checkers" is an Orwellian double speak. As a practical matter, there is no such thing as "fact checking" that is done by truly neutral arbiters on the basis of purely objective standards and facts. Rather, "fact checking" is merely another tool used by propagandists to push their propaganda and censor dissent. Their raison d'etre is to suppress the truth from the public.
Authors
Ashmedai