“A new study says…” What you should know about media reports of medical research
You often hear about new medical research results in the media, social media, or internet sources. But is this information trustworthy? Here’s how you can take a closer look at the information sources, verify the information, and consider how it applies to your health. Here’s a list of questions that you should ask when you evaluate this new information.
What is the source of the study information?
An important consideration is the source of information. You should consider if they may have a biased viewpoint. For example, information from a company or others with a financial interest in the product may be biased toward support of their claim. Most medical treatments are not a “cure.” Carefully consider anecdotal information in small numbers of people; for example, amazing testimonial results by just one person. Lastly, you should be skeptical of social media posts without a clear link or attribution to information about the study.
Credible medical discoveries are usually published in a reputable, peer-reviewed medical journal. This means that the study has been reviewed by other expert medical professions before the results are made available to the medical community and the public. The article will describe details about how the study was conducted, the criteria used to select individuals for the study, the interventions and how they were used/taken, how the study was analyzed, and a summary of the results and associated conclusions.
How was the study conducted?
The information provided should explain the type of study used to generate the results. The study design used can help us interpret the results or findings.
A randomized controlled clinical trial is a study type, or design, that compares the new treatment to no treatment (placebo) or another standard or current treatment. In this type of study, neither the patient nor study personnel know which treatment is being used. These study types are usually done to try to prove a new drug, or therapy will work to treat a condition or is better than other treatments offered. If the study is done correctly, these results can typically be used to determine whether a causal relationship exists between the treatment and the outcome.
Another study type is an observational study in which the researcher collects data from study participants or uses data that was already collected such as from medical records. Researchers do not assign a specific intervention to each participant but rather note the specific ways the participant behaves or uses an intervention and any notable changes to their health. This type of study design is usually used to make observations that may lead to more controlled studies or when conducting a randomized controlled study isn’t possible. The results found in observational studies can be used to find associations.
What are some things I should look for in the reporting of the results?
Carefully consider the results of the study. Do the results demonstrate a difference in one treatment compared to another treatment or medication? If an improvement (or decline in symptoms) is shown, it is important to consider what methods were used to verify that the results are not due to chance. Other factors may also affect the results such as accompanying treatments, length of therapy, and degree of illness.
It is important to recognize the difference between association and causation. Association is finding a relationship between two or more situations. This relationship may be coincidental, caused by other unknown factors, or may be related. Causation means that one factor has a direct effect on another. Here’s an example of a study to show the difference between causation and association. This study tracked the number of people with sunburn and ice cream sales for an entire year. Analyzing the data showed a strong positive association: days with higher ice cream sales had much higher sunburn rates. Clearly, sunburn doesn’t cause increased ice cream sales, and increased ice cream sales don’t cause sunburns, either. This example shows that confounding (external to the study) factors such as hot, sunny weather causes people to buy more ice cream, but it also causes more sunburns.
Another reporting technique is to state that some action results as a percentage reduction (or increase) between the two groups in the study. For example, a news article states that people taking a new medication would have a 50% reduction in heart attacks compared with people taking other medications. This makes the treatment sound very effective. However, the actual reduction in heart attacks between the treatments was from 1% to 0.5% ─ a much smaller effect. In this study of 2000 people, this would be the difference between 10 people out of 1000 in the control group or 5 out of 1000 people in the new treatment group having a heart attack. The actual reduction taking the new medication is 0.5% ─ which is very different finding than the reported effect of a 50% reduction. This type of mathematical comparison often greatly overestimates the benefit or risk of the treatment and does not reflect the baseline risk of the outcome event.
Does this information apply to me?
Review the information to see if it is applicable to your situation. You should verify that it is the exact same condition that you have, and that it was studied in a similar group of individuals (age, sex, race, smoker/non-smoker, degree of illness). The patients in the study should also have the same treatment (e.g., medication or procedure) as you would have.
What can I do if I still have questions?
Discuss the report with your doctor or pharmacist. It is helpful to have details or a printout of the information and source for them to review. Discuss how this information may pertain to your current medical condition as well as to consider side effects and your other medications or conditions.