Misinterpretation, misunderstanding, and purposeful misleading are common in science writing. These tips may help you identify trustworthy sources.
Originally published in the February 2019 issue of the Tufts Health & Nutrition Letter
Nutrition information (and mis-information) is all around us, in books, magazines, talk shows, news stories, or just a tap of the mouse or the touchscreen away. How do we know if the information we are getting is credible? “Interpreting research studies can be difficult, even for highly-trained researchers,” says Jeanne P. Goldberg, PhD, professor of nutrition communication at the Friedman School of Nutrition Science and Policy. “Sometimes news stories or websites simply get it wrong. Sometimes the author may have an agenda of their own, such as the desire to sell more of a particular product.” To be more confident in the information you’re getting, try following the ABC’s: does the information have Authority? Is there Bias? Is it Complete and current?
Authority
Accurate nutrition information is most likely to come from experts on the particular topic and reputable organizations or publications that specialize in that field. Here are some tips for assessing the authority of your source:
- Evaluate experience. Journalists with a science background are more likely to be able to interpret and present research in an accurate and understandable way. Internet sites should list the author and/or reviewer of an article. This not only allows the reader to find out more about the writer’s background, but also indicates the author stands behind the information presented. Many websites and book jackets offer author biographies, and a quick online search of an author or speaker can often reveal whether their degree or background qualifies them to give this health advice.
- Look for references. Good articles tell you where they got their information. Print pieces may include the journal name, year, and/or author in the text, and digital articles may have links in the text or references listed at the bottom.
- Seek out independent sources. Have you heard of this publication, and what is its reputation? Consider if the goal of the organization providing the information is to inform and educate or to sell a product or diet plan.
- Consider the studies. Research should be appropriate and well-designed. There are many different kinds of studies (see Glossary). Some show cause-and-effect, but many simply suggest a connection or association, which is important groundwork but, barring a preponderance of this type of evidence, may not be a good reason to change one’s behavior. A good article will indicate what kind of study was done. Results of a study done in a test-tube or in animals can rarely be generalized to humans. Additionally, interventional studies that include larger numbers of participants, longer time periods, and control groups typically provide stronger results.
- Check who is quoted. Many stories about research studies include quotes from scientists. There should be an indication of whether the person commenting was involved with the study or is an outside expert.
Bias
It can be blatant, or it can creep into our information sources unintentionally.
- Catch the slant. Someone trying to sell you something will happily list any research supporting the product but is unlikely to include studies that found it doesn’t work. Even many well-known publications and news sources are known to have a particular slant. Filter what you hear or read through your knowledge of the source. Some popular nutrition websites and public personalities have come under fire for pushing inconclusive evidence as fact or cherry-picking studies to sell diet programs or supplements. Consider an online search of the author or site owner to see if there have been complaints made against them.
- Check the funding source. Source of funding can affect the content of a study, article, or news story and how it is presented. Knowing who is behind a website or magazine, and what their agenda is, can help you filter the information they choose to present.
Good websites have an “About Us” page that should describe the person or organization running the site and their funding sources. Be aware that food manufacturers or trade groups are a common source of funding for nutrition studies. “The fact that a study is funded by industry does not mean it should be immediately ruled out,” says Goldberg. “There are many times when the only people who fund research are the people who have both a scientific and a financial interest in it.” To address potential bias, a writer or spokesperson can clearly identify the funding source and cite other studies that support or disagree with the findings.
“The fact that a study is funded by industry does not mean it should be immediately ruled out. There are many times when the only people who fund research are the people who have both a scientific and a financial interest in it.”
- Be aware of personal bias. Sometimes we dismiss information that goes against a long-held belief, habit, or preference. Keep your mind open to new information but remember that nutrition advice is not one-size-fits-all. “Beyond following an overall healthy dietary pattern, there is no one diet plan or approach to eating that is best for everyone,” says Goldberg. “For example, if you read that six small meals are better than three, that may be true for some people, but not necessarily for everybody.”
Completeness and Currency
Look for articles and experts that provide the whole picture.
- Seek context. Information should always be presented in the context of what is already known. Look for work that is not afraid to mention study limitations, and that presents corroborating (or contrary) information from other reputable studies. “Rarely has a single study in nutrition proved evidence of a dramatic breakthrough,” says Goldberg. “When information is presented as ‘new’ and contradicts what has been the consistent direction in the evolution of the science, it is important to regard it critically.”
- Consider timeliness. Recommendations do change, so make sure the information you’re reading isn’t out-of-date. Reputable online postings should indicate when they were last reviewed and updated.
- Expect quality. Poor spelling and grammar may indicate that a site or story is not credible. Good quality publications and websites value presenting information clearly and professionally.
If something offers a quick fix or seems too good to be true, it’s probably more hype than helpful. “People should not underestimate their own common sense,” says Goldberg. “If it sounds too good to be true, it probably is.” Before you make any major changes based on something you read or hear, you may want to run it by a healthcare professional. “And if you have a condition that requires true nutrition management,” says Goldberg, “you may find some sessions with a Registered Dietitian, especially one who is specifically trained in working with individuals with that condition, to be an excellent investment.”
“People should not underestimate their own common sense. If it sounds too good to be true, it probably is.”
Glossary of Research Terms
Animal Research Study: Addresses questions that cannot be studied in humans. In general, used to identify mechanisms of action (why a cause and effect relationship occurs). The data are frequently used to develop treatment strategies.
Case-Control Study: Researchers choose people who have an outcome of interest (e.g., cancer, obesity) and identify people with similar characteristics who have not developed the outcome. This design allows for the identification of potential causative factors.
Case Report or Series: A report of a patient or series of patients with an outcome, usually a rare outcome, of interest.
Cross-sectional Cohort (Observational) Study: A clinical research study that assesses subject characteristics and outcomes at a single time point. Because subjects are not followed over time (prospectively) this study design is not considered as reliable as a prospective cohort study but is valuable when prospective data are not available.
In vitro Study: Experiment conducted in test tubes or petri dishes using cells or pieces of tissue. Addresses questions that cannot be studied in humans. In general, used to identify mechanisms of action (why a cause and effect relationship occurs). Similar to the results from animal studies, the data are frequently used to develop treatment strategies.
Meta-Analysis: A statistical approach that combines the findings from multiple research studies, starting with a systematic review. Allows for an assessment of whether the findings from multiple studies are consistent or inconsistent.
Prospective Cohort (Observational) Study: A clinical research study in which participants are followed over time focusing on a specific outcome. Identifies associations, either positive or negative, between characteristics of the subjects (e.g., diet and physical activity) and an outcome (e.g., heart disease, diabetes), but cannot establish cause and effect.
Randomized Controlled Trial: A clinical trial that randomly (by chance) assigns participants with similar characteristics to two or more groups. In general, one group receives a placebo or no intervention (“control” group) and the other group receives some type of intervention (experimental group). This study design can establish cause and effect.
Systematic Review: An evaluation of all available studies that address a specific research question. By including all research studies on the topic, the results provide an unbiased assessment of the current state of the issue.