Whether they’re broadcast, digitized or printed on paper, the latest medical studies are a hot topic because they often address answers to serious health questions that have threatened our lives or impaired our ability to function for years.
But not all studies are created equal. And some types of studies are more likely than others to lend themselves to misinterpretation or even outright manipulation of their results. “Peer review”—whether a study has been published in a peer-reviewed medical journal—is another important measure of a study’s quality. Plenty of studies, including randomized controlled trials (see below), aren’t of good enough quality to get published in a high-quality, peer-reviewed journal. That’s why it’s important to know which types of studies are most likely to produce useful advice. Here’s how to figure that out…
Randomized placebo-controlled trials. The “gold standard” of medical studies is one in which researchers gather a group of individuals with the same health issue, then randomly assign them to one of two groups. One group will be the “control,” and receive a placebo or the current standard treatment. The other group will receive a treatment that the researchers want to test. While many studies establish an association between two factors, a randomized placebo-controlled trial can actually determine what caused something to happen.
The key words to look for in these studies are “randomized,” which means people were assigned randomly to one group or another, and “double-blind,” which means that neither the study participants nor the researchers knew who was receiving which treatment. A study that’s both randomized and double-blind prevents researchers from stacking the deck and delivering a result that’s been influenced by an outside source such as a pharmaceutical manufacturer.
Studies that show an association. Given the cost, logistics and ethical standards required for a “gold-standard” study, researchers frequently use another type of study in which people with the same health issue are divided into two groups. But instead of receiving either a fake treatment or a real one doctors want to test, the researchers simply watch both groups for an extended period of time, keep track of how they fare, then note any common threads that seem to lead to either a good or a bad outcome. This type of study cannot establish what actually caused something to happen, but it can suggest an association between various factors in something like a flu epidemic—an event in which doctors need to move quickly before the infection spreads. An example of this type of study was reported on in Bottom Line’s article “Finally, an Early Warning Sign of Pancreatic Cancer.”
Meta-analyses and systemic reviews. Researchers often like to develop studies in which they survey all the studies on a given topic. A systemic review is typically just a survey of good existing literature on a topic. A meta-analysis, on the other hand, actually analyzes the existing data from the included studies to create a new analysis. Meta-analysis studies are particularly useful at helping doctors figure out what’s causing a particular illness. Bottom Line’s article “Delirium Danger” is a good example of a meta-analysis study.
Avoid the all-inclusive package. An “ecological study” is the wolf-in-sheep’s-clothing of medical research. It may look as though it watches two groups of individuals set up by researchers and discovers a relationship, but it doesn’t. Instead it uses specific data gathered from individuals, but then compares it to a diverse and miscellaneous group of people, looks for an association, and then announces its “findings.” Consider, as a topic example, the relationship between meat consumption and colon cancer. If a study simply compares national colon cancer rates and national averages of meat consumption, there is no way to know if the individuals eating more meat are the same as those more likely to develop cancer. And without that information, the study isn’t all that helpful—except to suggest that further, more specific research is necessary. This type of study should be treated with strong skepticism.
The bottom line for consumers of health news? Always ask questions. While not every study published in a medical journal will be solid gold, many can raise important discussion points worth talking over with your doctor.
For more on interpreting headlines, check out “Are You Reading Too Much into Research Studies?”
