Levels of scientific evidence

24/02/2016 Advice for teachers, Help for Parents

With such a large amount of nutrition information available across the world’s media, it can be difficult to figure out who to trust and what advice to follow. Here at Healthy Kids, the nutrition recommendations we provide to schools and parents are founded on established scientific evidence.

We recently came across this great infographic that outlines the different levels of scientific evidence1. A story from a neighbour or the latest celebrity endorsed diet isn’t the same as advice based on scientific studies. To help explain these levels of evidence, we have created a table giving examples of each level. Generally, if there is high level evidence for a particular dietary recommendation, it takes contradictory evidence of a similarly high level for those recommendations to be revised. Conflicting evidence of a lower level isn’t generally considered strong enough to alter the advice, but may provide impetus for further research.

Healthy Kids constantly refer to the Australian Dietary Guidelines2 released in 2013. These guidelines were developed by a team of nutrition experts who assessed over 55,000 peer-reviewed journal articles. So the Australian Dietary Guidelines (ADG) and associated Australian Guide to Healthy Eating (AGHE) are based on high-level scientific evidence. Since the guidelines were released, there hasn’t been sufficient high-level evidence to contradict them, so they are considered the best advice available.

Also, because the recommendations that Healthy Kids give are for the general public, we base them on population health data. Giving advice to the general public is very different to giving dietary advice to individuals.

An example of this is the consumption of butter, which is high in saturated fat. Currently the highest level of evidence available shows convincing data that replacing saturated fats with poly-unsaturated fats lowers the risk of coronary heart disease (CHD)3,4. Therefore, public health advice is to substitute butter for poly- or mono-unsaturated fats. This is because a large proportion of the general Australia population already consume high levels of saturated fat from other sources, such as discretionary foods, so shouldn’t be consuming any more in the form of butter5There are lower levels of evidence that may make us question this recommendation and provides the motivation for further research. However, high level evidence would need to be found before current public health advice is altered. 

For individuals who already follow the recommendations of the AGHE, there may be room for a little butter in their diet because their consumption of saturated fats from other sources is low. Well-known nutritionist Dr Rosemary Stanton acknowledges that she occasionally eats butter, saying: “I eat a little butter not because I think it has any health benefits but because I like it and I think that, in the context of an otherwise healthy diet, that’s fine”.

So advice needs to be taken in the context in which it is given. Sadly, the majority of Australians do not consume enough positive nutrients (e.g. vitamins, minerals, fibre) and too much of the nutrients that are detrimental to health when consumed in excess (e.g. energy, saturated fat). If Australians focused on getting the basics right and ate according to the AGHE, the health of the nation would look much brighter. Perhaps then debating the latest ‘superfood’ or food ingredient would have a place in the context of an already healthy diet, rather than adding confusing information to a crowded debate.

Here’s our table of examples of the different levels of scientific evidence to help illustrate what each means:

 Study designDescriptionExample*
1Systematic literature reviewsSystematic literature reviews combine the findings of multiple studies relevant to the topic of interest to draw more holistic scientific conclusions. During the review process, the quality of each study is evaluated and their findings are weighted accordingly to minimise any biases. Systematic reviews provide the highest level of evidence and their results can usually be extrapolated to the broader population.A systematic literature review analysed 30 studies to determine whether the consumption of sugar-sweetened beverages (SSB) was linked to childhood obesity. The researchers found a strong, positive association between consumption of SSB and obesity in children [6].
2Randomised controlled trialsRandomised controlled trials (RCTs) randomly assign participants to an experimental group that receives the treatment being tested, or a control group. Ideally, both the participants and researchers should be ‘blinded’ so they do not know which group they have been allocated to – this reduces bias. In scientific research, RCTs are considered as the gold standard in study design.To investigate the effectiveness of a school-based intervention program in preventing childhood obesity, 2,425 primary school children were recruited. They were randomly assorted into an experimental group that received nutrition education sessions, or a control group. After three years, their BMI was calculated and compared to baseline. It was found that there was a significantly lower prevalence of overweight and obesity in the intervention group compared to control, with the control group’s BMI being 2.1 points higher, on average [7].
3Cohort studiesCohort studies follow a population over time to assess the relationship between risk factors and the development of a specific outcome or disease. Participants’ exposure level to these risk factors is identified at the beginning of the study, and the outcome of interest is measured later.Researchers wanted to examine the association between calcium intake and the risk of bone fractures. 5,022 participants, aged 18-30, completed a questionnaire to determine their baseline calcium intake. These were classified into three categories of ‘low’, ‘moderate’ and ‘high’ levels of calcium consumption. At 20-year follow up, the incidence of fractures in the cohort was assessed. It was found that the rate of fractures was significantly higher for participants with ‘low’ calcium, than ‘moderate’ or ‘high’ calcium groups. However, there were no significant differences in incidence of fractures between participants with ‘moderate’ and ‘high’ calcium intake8.
4Case control studiesCase control studies work backwards. In these studies, subjects are selected based on the presence (“cases”) or absence (“controls”) of a certain disease or outcome. The groups are then compared retrospectively to assess how frequently each group was exposed to a particular factor of interest. However, any associations between the exposure and outcome factors do not necessarily prove causation.A case control study was conducted to assess the protective effect of vegetables and legumes against prostate cancer. 1,619 men with confirmed cases of prostate cancer were identified via hospital databases, and were matched to 1,618 controls. The researchers then interviewed participants to assess their dietary consumption patterns. It was found that consumption of legumes and vegetables was inversely related to prostate cancer. That is, participants with prostate cancer were found to have consumed fewer legumes and vegetables in the past than controls without prostate cancer [9].
5Case reportsThese provide a record of individual cases, often describing unique or unknown conditions. Single case reports can only provide a low level of scientific evidence and their findings cannot be generalised to the wider population. However, they can be useful for studying rare diseases or other emerging nutrition-related issues.A recent case report detailed a rare incident of scurvy in an 11-month old infant, who was exhibiting poor growth, irritability and bone loss. The researchers discovered that the baby had severe vitamin C deficiency because he had been on a diet primarily comprised of almond milk. The mother was not informed that almond milk did not contain vitamin C, unlike breast milk or infant formula [10].
6Testimonials and expert opinionsNutritional claims made by ‘experts’ often rely on anecdotal evidence or testimonials that rarely illustrate typical circumstances. They have limited scientific credibility, as they are generally not backed up by valid research.A certain celebrity chef is an outspoken advocate for the paleo lifestyle, which excludes grain and dairy. He claims that this particular diet can cure major diseases such as autism and cancer. His fans regularly post testimonials about the improvements to their health since following the diet. But this lifestyle contradicts established scientific evidence, for example the benefits of whole grain foods in reducing the risk of colon cancer or obesity.

*Examples are based on real studies, but some details may be fictionalised and simplified for the purposes of this article.

References: 

  1. Compound Interest (2015, April 9). A Rough Guide to Types of Scientific Evidence. Retrieved from: http://www.compoundchem.com/2015/04/09/scientific-evidence/
  2. National Health and Medical Research Council (2013). Australian Dietary Guidelines.Canberra: National Health and Medical Research Council.
  3. Food and Agriculture Organization of the United Nations (FAO) & World Health Organization (WHO). (2010). Fats and Fatty Acids in Human Nutrition. Report of an expert consultation. FAO Food and nutrition paper #91. Geneva, Switzerland.
  4. National Heart Foundation of Australia. (2009). Dietary fats and dietary cholesterol for cardiovascular health. Retrieved from:http://heartfoundation.org.au/images/uploads/publications/Dietary-fats-summary-evidence.pdf
  5. Australian Bureau of Statistics. (2013). Australian Health Survey: Users’ Guide, 2011-2013. (Cat. no. 0.55.001). Canberra, Australia: ABS
  6. Malik, VS., Schulze MB., & Hu, F.B. (2006). Intake of sugar-sweetened beverage and weight gains: a systematic review. American Society of Nutrition, 84, 274-88.
  7. Jiang, J., Xia, X., Greiner, T., Wu, G., Lian, G., & Rosenqvist U. (2007). The effects of a 3-year obesity intervention in schoolchildren in Beijing. Child care, health and development,33(5):641-646
  8. Warensjo, E., Byberg, L., Melhus., H., Gedeborg, R., Mallmin, H., Wolk, A., & Michaelsson, K. (2011). Dietary calcium intake and risk of fracture and osteoporosis: prospective longitudinal cohort study. 342
  9. Kolonel, LN., Hankin, JH., Whittemore, AS., Wu, AH., Gallagher, RP., Wilkens, LR., … Paffernbarger RS. (2000). Vegetables, Fruits, Legumes and Prostate Cancer: A Multiethnic Case-Control Study. Cancer Epidemiol Biomarkers Prev., 9, 795
  10. Live Science (2016, January 18). Rare Case of Scurvy Seen in Infant Fed Almond Milk. Retrieved from: http://www.livescience.com/53402-scurvy-in-infant-fed-almond-milk.html