Are the health messages in schoolbooks based on scientific evidence? A descriptive study

Abstract

Background

Most textbooks contains messages relating to health. This profuse information requires analysis with regards to the quality of such information. The objective was to identify the scientific evidence on which the health messages in textbooks are based.

Methods

The degree of evidence on which such messages are based was identified and the messages were subsequently classified into three categories: Messages with high, medium or low levels of evidence; Messages with an unknown level of evidence; and Messages with no known evidence.

Results

844 messages were studied. Of this total, 61% were classified as messages with an unknown level of evidence. Less than 15% fell into the category where the level of evidence was known and less than 6% were classified as possessing high levels of evidence. More than 70% of the messages relating to "Balanced Diets and Malnutrition", "Food Hygiene", "Tobacco", "Sexual behaviour and AIDS" and "Rest and ergonomics" are based on an unknown level of evidence. "Oral health" registered the highest percentage of messages based on a high level of evidence (37.5%), followed by "Pregnancy and newly born infants" (35%). Of the total, 24.6% are not based on any known evidence. Two of the messages appeared to contravene known evidence.

Conclusion

Many of the messages included in school textbooks are not based on scientific evidence. Standards must be established to facilitate the production of texts that include messages that are based on the best available evidence and which can improve children's health more effectively.

Introduction

Quality of health education in the School

Health education forms a part of the compulsory education curriculum in Spain, either as explicit content within a given area of knowledge or as supplementary content in any area or didactic act [1]. In order to encourage development in this area, with regards to both explicit and supplementary health education, various didactic methodologies have been promoted, which are more unidirectional in the first instance, and more participative and interactive in the second instance [2]. This two-fold approach to health education, particularly in the case of supplementary education, has given rise to a situation wherein practically all textbooks contain messages relating to health, irrespective of their subject matter. The profuse levels of information relating to health in school books require rigorous analysis with regards to the quality and suitability of such information [3]. Moreover, such analysis is indispensable as many of the health messages are formulated as advice or recommendations addressing the children directly. Certain studies have evaluated the quality of this information, measuring the effectiveness of messages aimed at illness prevention within the school [4] or the fostering of healthy habits [5]. Extracurricular programs that have been set in motion in schools by third parties have also been evaluated [6]. On other occasions, studies have focused on identifying the health areas addressed in the books or the didactic methodology employed to expound such content. However, we have found a single study that examined the contents about sexuality of the textbooks used in Spanish secondary education analysing the scientific evidence that support them [7]. Furthermore, in view of the section of the population that is targeted by these messages and their importance in improving health awareness and healthy behaviour amongst children, we must ensure that these messages are based on the best available evidence derived from reliable scientific studies.

Scientific evidence in the health messages in textbooks

Evidence-based Medicine (EBM) is a process for the critical evaluation and incorporation of scientific discoveries into the decision-making process within the healthcare sector. This process can be applied to any type of health-related action: diagnosis, therapy or preventative measures [8]. From this perspective, health-related messages in schools should also be filtered via EBM. Determining the level of evidence of messages included in textbooks is of vital importance. The appropriateness of including messages that will be learned by children in textbooks should be determined by the level of evidence on which such messages are based.

The research questions for this study is if the health-related messages in textbooks are based on scientific evidence. The objective was to identify the scientific evidence used as a fundament for health-related messages in textbooks employed within compulsory education in Spain, namely, Primary and Secondary Education. The use of EBM methodology may prove extremely useful when selecting content to be included in school books. Textbooks should only contain the most reliable affirmations in relation to health, that is, statements that are solidly based on scientific evidence. The results of this study may aid the authors and publishers of school books to improve content. In addition, our results may also help teachers to place greater importance on the assimilation of messages that are more firmly based on scientific evidence. Finally, this study may facilitate the adoption of behaviours amongst children and the acquisition of knowledge by children that have more consistently demonstrated a capacity to improve health and prevent illness.

Methods

This is an observational, descriptive and analytical study that was carried out in the city of Granada.

Study material

Between the 1^st of March 2006 and the 1^st of June 2007, textbooks used in primary and secondary schools in a district of the province of Granada were identified. Books written in languages other than Spanish, "Teacher Guides" and books containing no health related messages were excluded. A representative sample of books was selected: a proportion with a precision of 5% was estimated along with a bilateral confidence interval of 95%.

Within the framework of the study, health-related messages refer to any statement within a school book that addresses any health-related subject in the form of a recommendation or advice. Messages relating to four areas were excluded: a) Aspects of Anatomy or Physiology that are beyond all scientific doubt were ignored (the heart has four chambers). b) Respect for the environment (preventing damage to the ozone, water saving, recycling, etc) as it is currently difficult to evaluate the scientific evidence and effects on health of recommendations in this area; c) Psychological and Social well-being relating to living in a community and mutual respect, as such considerations are not easily transformed into research questions and we were unable to find studies relating to evidence in this regard (in order to live together without conflict, we must respect one another); d) Messages relating to personal hygiene and food hygiene that emphasised norms of urbanity or courtesy (don't let your hair fall into your food, don't clean your mouth with the sleeve of your shirt, etc).

Instrument

The texts selected were reviewed in order to identify any health-related messages. We drew up a table to classify messages, recording information relating to the school year and subject matter of the textbook containing each message. In addition, the focus of the message was defined in accordance with the classification of health priorities for children and adolescents drawn up by the health authorities [9].

Procedure

Each health-related message in the school books was transformed into a research question. In order to transform the messages into interrogations, where possible, the Patient-Intervention-Comparison-Outcomes (PICO) scheme, drawn up by Richardson et al was employed. However, as these authors drew up their scheme for question construction in a clinical setting, which clearly does not apply to schools, the scheme was slightly modified for the present study. In the present study, the following aspects were considered in each question: (P) the individual or group targeted by the health-related message (boys, girls, adolescents, etc); (I) the area of health that the message focuses on, normally in the form of advice or recommendations (the consumption of fruit and vegetables, physical exercise, road safety, etc); (C) comparison with behaviour that is detrimental to health or with the failure to abide by the advice (the absence of fruit and vegetables in the diet, a sedentary lifestyle, failure to wear a helmet, etc); and (O) associated beneficial effects where the advice is followed. An example of transformation of the messages into interrogations is: Tobacco smoke produces severe diseases of the respiratory system (message) in: Does tobacco smoke produce severe diseases of the respiratory system? (question)

Finding the best available evidence in relation to the questions

To this end we employed the Trip Database (TD), one of the most important Internet resources for searches relating to evidence-based medicine. In accordance with TD recommendations, searches were carried out using search terms relating to question content in conjunction with Boolean searching, parenthesis and stemming.

From the results obtained via the TD, we only considered those documents containing information that was classified in terms of evidence, such as clinical practice guides and systematic reviews. The full texts of the documents were located and thoroughly reviewed, searching for a response to each research question. In each case, we identified the institution that published the document and the uniform resource locator (url) providing a link to the full text.

Data analysis

Results relating to the level of evidence were classified into three categories: a) Messages with a level of evidence: messages wherein the research question is answered in at least one of the consulted documents that also report on the level of evidence. b) Messages based evidence without assigned level: messages wherein the research question is answered in at least one of the consulted documents, which nevertheless fail to report on the level of evidence. c) Messages with no known evidence: messages wherein the research question is not addressed in any of the consulted documents.

In the case of the Messages with a level of evidence group, the level of evidence was informed in each case. As different institutions employ a wide variety of systems to classify the degree of evidence, this study employed its own classification system, synthesising the most widely accepted classifications. This study classifies the level of evidence into three groups - high, medium and low - which homogenise and respect the original classification hierarchies that are most widely employed. Table 1 summarises the classification system used in this study.

Table 1 Scale employed in the study to measure the level of evidence

The statistically analysis used was only descriptive. Frequencies, percentages and mean were calculated.

Results

Textbooks and messages studied

237 textbooks used in primary and secondary schools were identified. 129 of them met the inclusion criteria. A representative sample of 80 books was randomised via simple aleatory selection. All the books were manually revised and 1135 health-related messages were identified. Subsequent to the elimination of messages that did not meet the criteria for inclusion, 844 were studied.

Subject and School year of the health-messages

Table 2 displays the characteristics of all messages according to subject matter: 291 not eligible messages were found. Of these, 79 were about Anatomy and Physiology, 58 about Social and Psychological Well-being, 93 about Respect for the Environment and 61 about Norms of Urbanity or Courtesy. Balanced Diet and malnutrition was the most frequent subject of the messages (264, 31.3%), followed by Protection against environmental agents (96, 11.4%).

Table 2 Selection and randomisation of messages classified by subject.

As Table 3 illustrates, the average number of messages per book amounted to 10.6, whilst the highest concentrations were found in the 3^rd year of secondary school (37.1) and in the subject Biology and Geology (32.5). The subject Environmental Knowledge (13.5) and the fourth year of primary education (19.1) ranked second in terms of the number of messages. The 18 health-related messages found in the 2 Social Sciences text books were not included as, focusing almost exclusively on respect for the environment, they failed to meet the criteria for inclusion.

Table 3 Books, messages and messages per book according to school year and subject.

Table 3 also provides information on the number of books and classified messages according to academic year and subject matter. 59 books (73.7%) and 548 messages (64.9%) were of Primary School and 21 (26.3%) and 296 (35.1%) respectively of Secondary School. The mean of messages for book in Primary School was 9.3 and in Secondary School 15.

Scientific evidence of the health-messages

Table 4 provides information on the frequency and percentage of classified messages according to subject matter and level of evidence. As the table shows, almost 61% of the messages are based on documentary evidence, although information on the levels of evidence is not provided. Less than 15% fell into the category where the level of evidence was known and less than 6% were classified as possessing high levels of evidence.

Table 4 Messages according to subject and level of evidence.

"Balanced diets and malnutrition" is the subject matter that is most often addressed in messages and also gives rise to the highest frequency of messages based on a given source of evidence, although the level of evidence is not known. This subject, along with messages concerning "Food and Hygiene", "Tobacco", "Sexual behaviour and AIDS" and "Rest and ergonomics", is based on a source of evidence in more than 70% of cases, although, once again, the level of evidence is not specified.

"Oral health" registered the highest percentage of messages based on a high level of evidence (37.5%), followed by "Pregnancy and newly born infants" (35%).

Almost a quarter of the messages (24.6%) are not based on any known evidence. In the case of "Respiratory illnesses" and "Protection against environmental agents" more than half the messages are not based on any known evidence (57.9% and 53.1% respectively) and in the case of "Oral Health", the number of messages not based on any known evidence accounts for 40% of the total. In two instances, according to the documents consulted, the messages appeared to contravene known evidence.

Table 5 provides information on the documents - Clinical Practice Guides (98 in total) and Systematic Reviews (9 in total) - that were consulted in order to answer the research questions linked to health-related messages in school books. Table 6 provides examples of messages with different levels of evidence and identifies the two messages that contravene evidence according to the documentary sources that were consulted.

Table 5 Consulted Guides and Systematic Reviews with url to facilitate their location.

Table 6 Examples of messages with different levels of evidence

Discussion

Attention should be drawn to the considerable number of health-related messages found in school books. This can be interpreted as a symptom of the high levels of observance of the official recommendation to include health education in all areas of the curriculum. However, a shadow is cast on this optimistic view when we consider the number of messages that are not based on scientific evidence. This raises questions in relation to the criteria that are employed to select messages for inclusion in school books. The fact that almost a quarter of the messages that were studied are not based on any documentary evidence (Guides and Systematic Reviews), suggests a certain degree of arbitrariness and a lack of clearly defined criteria to evaluate the importance of a health-related message. If we assume that the inclusion of a health-related message in the sources of evidence is indicative of their importance and suitability, our results show that far too many health-related messages aimed at school children are not firmly based on scientific evidence.

Nevertheless, the presence of health-related messages in guides and systematic reviews can not be used as the only means of justification for their inclusion in school books. This is due to the fact that clinical practice guides and systematic reviews entail a long preparation period that may significantly delay the publication and circulation of health information based on evidence. Moreover, the psychology of learning suggests that directing a health-related message towards children, even where the message is not based on sufficient scientific evidence, may aid the consolidation of healthy habits. For example, consulted documents recommend that children clean their teeth two times a day, a message that is based on a high level of evidence. However, the documents do not state that the cleaning process should be carried out after each meal. Nevertheless, recommending that teeth are cleaned immediately after each meal may help children to consolidate habits and routines that entail health benefits.

This paper does not intend to recommend the exclusion, at a general level, of messages that are not clearly based on evidence. However, we do feel that it is advisable to place less emphasis on messages that are not backed up by sources of evidence with a greater degree of frequency. For example, this is the case of messages relating to protection against environmental agents.

This study presents a number of shortcomings that bear mention. Considering clinical practice guides and systematic views as the only source of evidence may limit the scope of the study. Nevertheless, the tenets of care practice consider observance of the recommendations of a clinical practice guide or high quality systematic review to represent the most efficient method of ascertaining the best clinical course to follow [10].

Nor can we consider the exclusive use of the Trip Database as an evidence search engine as a shortcoming. Other search engines and other search strategies may produce different results. However, we decided to use this pre-filtered source of evidence and thereby ensure that all documents consulted were based on evidence that has been rigorously verified.

As conclusion, almost a quarter of messages included in school textbooks have an unknown scientific evidence. These messages are included in textbooks along with messages that are solidly based on scientific evidence. Children receive all of these messages and are unable to identify the importance or lack of importance of a given message or the extent to which it is based on evidence. The results of our study suggest that there is a need to establish standards that will enable the publishers who draw up the texts and the teachers who employ such texts to select and impart messages with solid scientific evidence.

Finally, this study has some practical implications: The results of our study demonstrate that there is a need to establish a mechanism that is capable of selecting messages on the basis of the degree to which they are based on scientific evidence. The publication of books that respect the criterion of a firm basing in scientific evidence with regards to the messages that they contain will facilitate the acquisition of knowledge amongst children that has consistently proved capable of improving health and preventing illnesses.