High-quality patient information materials may help consumers make informed decisions for or against participation in CRC screening. For identification of appropriate information materials, we compiled a list of criteria via qualitative aggregation based on systematically identified recommendations and tools, and validated the list of criteria by a review process. This instrument is designed for use by persons with expertise in cancer screening. It explicitly is not a checklist for consumers to check the quality of health information. The final manual-based list of criteria contains 230 criteria in four domains. The criteria are rated on multiple dimensions (e.g., presence and correctness of information), if applicable. Free text entries (mainly verbatim quotes) were allowed to ensure rating transparency.
One main limitation of the list of criteria is the lack of a summary assessment. It may be tempting to use a numerical scoring system for simplicity, but there is no empirical evidence to support this. Consequently, as there is no justification for a scoring system that applies an arbitrary (one point per criterion, etc.) or explicit weighting system, we rejected the use of a numerical scoring system or scale. Scales for quantitative assessment of study quality were very popular until empirical evidence  and theoretical considerations  indicated that scales provide invalid results. Because of the lack of a summary assessment method, analyses with the proposed list of criteria are restricted to the level of single criteria. This is inconvenient due to the large number of criteria to be assessed. To give an overview of the results, we used a traffic light (status indicator) system. This system can indicate two dimensions (e.g., presence and correctness of information) simultaneously, and it provides detailed information on single and overall categories. Such a comparative overview is particularly useful for benchmarking purposes. The future aim is to develop a qualitative summary assessment based on ratings on the category level. A Delphi consensus process might be used to explore the importance of each criterion, as was done in International Patient Decision Aids Standard Instrument (IPDASi) development .
The list of criteria represents the maximum content of information material. Not all of the criteria are essential for high-quality information. The comprehensiveness and depth of information materials vary depending on the targets and target groups. Thus, it would be reasonable to differentially define essential criteria for short information materials like flyers and for more detailed materials like brochures. It would also be reasonable to select these mandatory, material-specific criteria in a Delphi procedure including experts and consumers. Obviously, expert and consumer opinion is needed to explore the importance of each criterion for further summary assessments and to develop specific assessment lists of criteria for short and more detailed information materials. This input could be used in further research to revise the list of criteria.
The rationale behind providing EBHI on cancer screening is to enable consumers to make informed choices for or against cancer screening. The proposed list of criteria examines whether health information materials meet EBHI standards. It cannot directly assess whether the information is suitable to support informed decision-making. There is evidence from two randomized controlled trials that decision aids on CRC screening via FOBT  and FOBT/colonoscopy  may effectively support informed choice. However, both studies compared an interactive decision booklet with an accompanying DVD  or interactive internet module  (intervention groups) against a standard governmental booklet (control group). The effect may have been mediated by the interactive components resulting in more intense study of the materials. The proposed list of criteria does not assess interactive components. Ultimately, if information materials are found to meet EBHI standards according to our list of criteria, it cannot be concluded that these materials promote informed choice. Such a claim would have to be verified in further studies.
The ethical goal of EBHI to enable as many of the target population to make an informed decision whether or not to participate in CRC screening  may be conflicting with the aim of achieving a high uptake . There is inconclusive evidence on detailed information material, it may have a positive or no effect on participation [7, 10] or may even increase non-attendance . Non-attendance based on an informed choice has to be accepted, while non-attendance arising from the EBHI itself and not from an informed choice is not desirable. EBHI especially may deter socioeconomically disadvantaged people and those with low health literacy from participating in screening  resulting in higher health inequalities. Further research is needed to explore tailored communication strategies for deprived target groups focusing on increasing knowledge and understanding to promote an informed choice-making.
To our knowledge, the proposed list of criteria is the first assessment tool designed to rate the correctness of consumer education materials on CRC screening. Many existing tools use structural quality as a surrogate for content quality, which might not always be correct. For example, it was shown that website origin does not predict content quality: the quality of university websites was not better than that of commercial websites . Website certification programs like the HON (Health on the Net) code  and MedCERTAIN (MedPICS Certification and Rating of Trustworthy Health Information on the Net) were also established to ensure the provision of reliable information. The HON code requires health information website owners to abide by eight principles: to indicate the authors’ qualifications, information sources, funding sources and advertising policy and to maintain confidentiality, etc. However, there are concerns that these criteria might not be sufficient to identify trustworthy information. For example, the HON label failed to predict the good content quality of mental health-related websites in some cases . In contrast, the DISCERN score was shown to be a content quality indicator of relatively high specificity [52, 54]. Other analyses of web-based information on depression found that content quality correlated with the DISCERN score and HON label . However, the usual tools for the assessment of content quality do not check the correctness of information and might even rate false information as being of good quality.
As the proposed manual provides the correct answers to the criteria queries, it is an essential part of the rating method. Ideally, many CRC-specific content criteria should be explored by systematic reviews, especially if related to benefits and risks. This was not feasible in this project. Our research, which was mainly restricted to selectively searched evidence from S3 guidelines, systematic reviews and HTA reports, was still very time-consuming. It will be challenging to incorporate the latest evidence in the finished manual, as knowledge changes over time. It is also unclear how often the manual should be updated. Other problems can arise from different interpretations of the evidence. In breast cancer screening, for example, experts disagree on the actual numbers characterizing the benefits and risks in a British leaflet . The National Cancer Institute (USA) took an interesting approach to providing key information, namely, by posting a one-page factsheet on lung cancer screening for doctors and patients providing numbers on benefits and risks derived from a randomized controlled trial . In Europe, aggregated uniform evidence-based factsheets on screening procedures would complement the existing comprehensive guidelines  and would offer a thorough base of knowledge for different players who provide information on cancer screening.