Research method

Most of it begins in PubMed, the public index of the United States National Library of Medicine. We pull it through the official programming interface rather than by hand, so a search is repeatable and nobody can pick along the way what goes in. Every article prints that query verbatim.

But PubMed is not everything, and pretending otherwise is the easiest way to miss something. We also look in the Cochrane Library, where systematic reviews often appear earlier and in fuller form. In ClinicalTrials.gov and PROSPERO, to see which research was registered and then never published, because that is the single largest source of bias there is. And in Google Scholar for theses, reports and literature indexed nowhere tidily.

What we cannot reach is Embase, which sits behind a paywall. That is a real limitation and we put it here rather than leave it out.

For products there is one more layer: certificates of analysis and test reports from the manufacturer, and where we can measure something ourselves, our own measurements. Those appear as a measurement and never as a verdict, with the method beside them.

Research in people. Randomised trials weigh heaviest, then systematic reviews and meta-analyses of those trials, then observational work.

Cell cultures and animal studies do not count as evidence that something works in you. We mention them sometimes to explain a mechanism, and when we do we say so explicitly.

Uncontrolled research where controlled research exists. Papers we cannot read in full when the abstract does not carry the whole result. And anything we cannot trace to an indexed source with a PMID or DOI.

This is where we differ most from everybody else. A study that found nothing is written up as fully as one that found something, and where the best research in a field found nothing, that is the conclusion.

We publish against our own shop when the evidence points that way.

We look at the strength of the design, the size, whether it replicated, and whether the effect is large enough to mean anything to you. A statistically significant difference you cannot feel is not a result.

Where reviews contradict each other we work out why, usually in the full text, and write that down instead of averaging them.

Above every article that reaches a verdict sit two labels, and they say deliberately different things.

What this rests on is a fact about the literature. Pooled human trials, individual randomised trials, observational research, laboratory and animal work only, or nothing at all on the product itself.

How solid it is is our judgement, and it comes with a rule you can check.

Strong. Randomised trials in people, not contradicted by other randomised trials.

Reasonable. Trials, systematic reviews and analyses that do not contradict each other but fall short of that bar.

Limited. Everything else.

Weak. Barely a conclusion to draw, because there is no research in people or because everything contradicts everything.

There is one exception, and it is written down rather than applied quietly. A claim about how well something predicts, rather than about what happens when you do something, cannot be randomised: nobody can be assigned an age or a blood value. There a second population takes the place of a second trial. So a measure built in one cohort and holding up in another reaches Reasonable, and never Strong.

Note what is not in there: how much research exists. Four hundred reviews that disagree rate lower than one clean trial, which is exactly why this sits beside the basis rather than replacing it.

Both labels carry a sentence saying why, because a label without a reason is a sticker. Articles that explain a subject rather than judge a claim do not get them.

We sell products and coaching. That is an interest, and it is why every claim on a product page has to point at our own reading of the literature, including the research that found nothing.

If we cannot support a claim that way it does not go on the page, however well it would sell.

We use software to search, retrieve and organise studies, including language models. Every PMID and DOI here is then checked against the source automatically.

After that Liam reads all of it himself: he checks it, changes what is wrong or not sharp enough, and only then releases it. No sentence appears on this site that he has not read.

We say this because a site asking you to check its work should be checkable about itself.

We get things wrong. When an article changes on substance rather than spelling, we change it and record the date. If you find something that is not right, tell us, and we will fix it and write down what was wrong.