When I am confronted with a new ingredient or test and am told that this is the solution to all man’s problems, of course I’m going to view that with a grain of salt. But is there a way to effectively evaluate claims?
I’m sure we all have a variety of steps we go through in evaluating date, but I have found the following three step process to be helpful to me:
- Show me the science
- Tell me the history
- Tell me a story
Actually, there’s a first step which comes before the above list, and it’s captured in the Latin phrase “prima non nocere,” the basic mantra of medicine and at the core of the Hippocratic Oath. Is this product safe? How do we know? I just finished a book about telomeres by Elizabeth Blackburn and Elissa Epel (Blackburn shared the Nobel prize in Physiology or Medicine in 2009 for her discovery of telomeres and telomerase and their role in aging). After reading their book it’s pretty hard to think of telomeres as anything but the holy grail of healthy aging, so it would be natural to think that anything that helps support telomerase activity or the lengthening of telomeres would be a good thing. But they specifically call out supplements that claim to enhance telomerase as being potentially harmful. Their point (which I actually think is well-taken), is that, since studies show elevated telomerase may be linked to increased cancer by a mechanism we don’t fully understand. So they say that “naturally” increasing telomerase, which can be accomplished in lots of different ways outlined in their book, is an important and beneficial strategy for a longer “healthspan,” any claims for supplements which include “artificially” increasing telomerase by anything other than lifestyle modification should be viewed with skepticism. I should also point out that they are not saying unequivocally that supplementing to increase telomere length (or upregulate telomerase activity) is known to be harmful; just that it hasn’t been clearly established safe. And it is incumbent upon the companies making such products (and associated health claims) that they establish safety as an important first step.
And I agree.
So let’s say we have that established (or at least the relative risk is low). The first step is “Show me the science.” That’s pretty obvious: I’ll want to review whatever published data exists on the product. I’ll read the actual studies; are they human, animal or cell line? How many studies? Where and when were they published? How large were the studies? If human, were they observational, open label, randomized control trials, epidemiological? Within this context, there’s a hierarchy, and I’m going to want to see what evidence is available and how solid it is. And while the gold standard in medicine is the randomized double-blind, crossover placebo study, but there is evidence that RCTs may not be the best way to evaluate efficacy of nutritional supplements (but that’s a topic of another post). Anyhow, the stronger the evidence (as evaluated through the lens of the scientific data provided), the better. And frankly, I shouldn’t have to go dig up the research myself; a reputable company is going to make that available to me. I may still dig a bit deeper to see if they might have cherry-picked their data, but they are the ones making the claim; it’s up to them to support their product.
Step two gets activated if there’s no science, or if the science is spotty. My next step is to ask for the history of the product. Where has this product been used? By whom? For how long? In what context? Etc., etc., etc. It’s important to point out that I don’t reject something out of hand if there’s no science for it (or if the science is spotty). For example, Traditional Chinese Botanicals frequently have little science (comparatively) from a Western perspective, but remember that some of these formulae have been used for millennia in Chinese medicine. That constitutes a pretty impressive history, in my humble opinion! Just the fact that we don’t have reams of papers published in Western peer-reviewed literature doesn’t make it less effective; it may just mean that we haven’t gotten around to studying it! But I will say that my “skepticism meter” goes up a notch or two if no studies are provided. So if it fails the “science” step, is there a history of use that I can hang my hat on?
The third step is the final one: Tell me a story about how this works. I’m going to be pretty demanding here, that this product fits in with what I know (or understand) to be true physiologically. For example, some years ago there was a lot of buzz around SOD (superoxide dismutase), a very important anti-oxidant enzyme produced in virtually every cell, because it quenches the superoxide anion radical, produced by mitochondria in WBCs as part of our defense against infectious agents. SOD was being sold as an oral supplement. A cursory evaluation of the studies provided (and there were many) showed that the test product was injected, and in fact has a half-life measured in seconds or less. So I asked the vendor what evidence he had that oral administration was effective and he acted like he didn’t hear my question; he simply referred by to the studies of SODs benefit (through injection). He was in essence asking me to simply accept that it would survive digestion and work the same as the injectable. For me that spelled no joy. And it should be noted that except for a few outriders (mostly in supplements sold in health food stores), no one carries an oral SOD product any more.
As I said, I’m sure most have some process they go through to evaluate new ideas; this one works well for me.