Gaps in detailed knowledge of human papillomavirus (HPV) and the HPV vaccine among medical students in Scotland
© McCusker et al.; licensee BioMed Central Ltd. 2013
Received: 11 October 2012
Accepted: 11 March 2013
Published: 22 March 2013
A vaccination programme targeted against human papillomavirus (HPV) types16 and 18 was introduced in the UK in 2008, with the aim of decreasing incidence of cervical disease. Vaccine roll out to 12–13 year old girls with a catch-up programme for girls aged up to 17 years and 364 days was accompanied by a very comprehensive public health information (PHI) campaign which described the role of HPV in the development of cervical cancer.
A brief questionnaire, designed to assess acquisition of knowledge of HPV infection and its association to cervical cancer, was administered to two different cohorts of male and female 1st year medical students (school leavers: 83% in age range 17–20) at a UK university. The study was timed so that the first survey in 2008 immediately followed a summer's intensive PHI campaign and very shortly after vaccine roll-out (150 students). The second survey was exactly one year later over which time there was a sustained PHI campaign (213 students).
We addressed three research questions: knowledge about three specific details of HPV infection that could be acquired from PHI, whether length of the PHI campaign and/or vaccination of females had any bearing on HPV knowledge, and knowledge differences between men and women regarding HPV. No female student in the 2008 cohort had completed the three-dose vaccine schedule compared to 58.4% of female students in 2009. Overall, participants’ knowledge regarding the sexually transmitted nature of HPV and its association with cervical cancer was high in both year groups. However, in both years, less than 50% of students correctly identified that HPV causes over 90% of cases of cervical cancer. Males gave fewer correct answers for these two details in 2009. In 2008 only around 50% of students recognised that the current vaccine protects against a limited subset of cervical cancer-causing HPV sub-types, although there was a significant increase in correct response among female students in the 2009 cohort compared to the 2008 cohort.
This study highlights a lack of understanding regarding the extent of protection against cervical cancer conferred by the HPV vaccine, even among an educated population in the UK who could have a vested interest in acquiring such knowledge. The intensive PHI campaign accompanying the first year of HPV vaccination seemed to have little effect on knowledge over time. This is one of the first studies to assess detailed knowledge of HPV in both males and females. There is scope for continued improvements to PHI regarding the link between HPV infection and cervical cancer.
KeywordsHuman papillomavirus Knowledge Vaccine Medical students Public health
Anogenital human papillomavirus (HPV) infection is the most prevalent viral sexually transmitted infection (STI) in the world today . Infection is extremely common, most prevalent among sexually active young adults and usually follows a benign course. Persistent infection with “high risk” (HR) HPV causes cervical disease that can lead to cervical cancer . HR-HPV types, of which there are 15, cause over 99% of cervical cancers .
September 2008 saw the introduction of a UK nationwide HPV immunisation programme . The vaccine is now offered routinely in secondary school, to girls aged 12 – 13 years. A three-year “catch up” campaign, delivered in GP surgeries and dedicated vaccination centres, has allowed older girls up to the age of 17 and 364 days access to the vaccine. Boys are not vaccinated. In the UK, between 2008 and 2012, the bivalent Cervarix vaccine  was used to protect against the two HR-HPV types, HPV 16 and 18, that cause around 70% of cervical cancers . The vaccine was provided in three doses over six months to elicit a protective immune response in vaccinees prior to exposure to the virus .
Vaccine uptake in the UK has been excellent, at > 90% for 12–13 year-old girls. However, uptake rates for the catch-up programme are lower at less than 60% overall . Vaccination is accompanied by targeted public health information (PHI), in the form of leaflets and/ or information sharing sessions directed by trained school nurses. Details of HPV infection and its association with cervical cancer are explained: HPV is sexually transmitted, HPV causes all cervical cancers, HPV vaccine protects against two types of HPV. Therefore, young women who have been vaccinated may be expected to have increased knowledge of HPV and its association with cervical cancer compared to the unvaccinated population. At the launch of the vaccination programme these routine information sources were supplemented by an extremely intensive media campaign as well as a specially developed Scottish website (http://www.fightcervicalcancer.org.uk/) and associated literature. The website has now been withdrawn and replaced with information on immunisation (http://www.immunisationscotland.org.uk/vaccines-and-diseases/hpv.aspx). Although these efforts were largely targeted to women, young men would also have been expected to have some exposure to the media campaign because of its intensity and range (cinema, television, radio, billboards, newspaper adverts) in 2008 .
Correct knowledge about HPV infection and cervical cancer could be important to inform decision making regarding uptake of the vaccine. Prior to vaccine roll out many studies revealed a lack of awareness of HPV (i.e. whether respondents had heard of the virus) and a lack of knowledge of the association between HPV and cervical cancer in the UK and a range of other countries [9–14]. Recent studies have demonstrated increased awareness of HPV following the introduction of the vaccine [15, 16]. However, fewer studies have investigated acquisition of detailed knowledge of HPV infection. Those that have, indicate low levels of more detailed knowledge of HPV infection and links to cervical cancer [17–20]. Such knowledge appears to be greatest in those with higher education levels .
This study was conducted at the University of Glasgow, Scotland. Prospective medical students at this university are admitted to study on the basis of interview where depth of knowledge of current topics in medicine is assessed. The HPV vaccine was one of the main topics for 2008 and 2009 so new medical students at the university could be expected to acquire some knowledge of HPV. HPV knowledge was assessed in two groups of first year medical students, mean age range 17–20 years of age. The timings of the surveys were specifically chosen. The first survey was carried out in November 2008 immediately following the roll-out of the HPV vaccine and a summer’s intensive media campaign that aimed to inform the public about HPV and the HPV vaccine. The second cohort was surveyed one year later when many more females had received the vaccine and associated PHI. The multimedia PHI campaign was sustained over this one year period. Moreover, media interest in the death from cervical cancer of the celebrity Jade Goody [21, 22], and increased discussion of HPV in the media, at the start of 2009 coincided with the timing of interviews for medical school of the 2009 student cohort.
We sought to answer three research questions. The first question we addressed was knowledge about three specific details of HPV infection that could be acquired from reading of PHI leaflets accompanying the HPV vaccination programme and from PHI in an intensive media campaign between 2008 and 2009. The second research question was whether vaccination of females in the group and a year’s PHI campaign had any bearing on HPV knowledge. In contract to policy in the rest of the UK where screening starts at age 25, from age 20, Scottish (and Welsh) women are offered three-yearly cervical screening for the detection of HPV-associated cervical disease. 73.7% of eligible women were screened in the past three and a half years (as of March 31st, 2010), which is a rise in uptake of around 4.5 percent in comparison to previous years . Arguments for and against screening before the age of 25 can be found at http://www.cks.nhs.uk/cervical_screening/management/scenario_when_to_offer_cervical_screening/. The final research question was to find out if men and women acquired similar levels of knowledge about HPV from the same or different sources. This study is among the first to compare knowledge of details of HPV of an educated mixed gender cohort at a time of an extremely intensive media campaign then exactly one year later in a similar cohort where most of the females had received at least one dose of the HPV vaccine and following a sustained PHI campaign.
A questionnaire survey was administered to first year medical students at a UK university (University of Glasgow). Medical students were chosen as a "best case scenario" of HPV knowledge acquisition among young adults because they are an educated population and may have a greater awareness of current medical issues: entry into medical school at Glasgow is by interview where prospective students are questioned on knowledge of current topics in human health. Moreover, as health professionals of the future, they may have high motivation to pay attention to and acquire current PHI. Student knowledge and opinion was assessed in 2008 in that year’s intake of medical students i.e. immediately following the implementation of HPV vaccine delivery in the UK, and then one year later in the 2009 intake of students. The same questionnaire was administered in both years.
An 8-item closed-ended questionnaire, with, for females, a 3-item data gathering section to determine stage of vaccination, intention to attend smear testing and smear testing status, was developed from a literature review and in consultation with a vaccine specialist at Health Protection Scotland and NHS public health information specialists. No studies examining detailed knowledge of HPV in the UK were reported in the literature at the time of this study so three questions relating to HPV were designed to address specific points regarding HPV knowledge discussed in leaflets delivered to vaccinees and in other PHI. The draft questionnaire was piloted on students in the final year honours virology classes of both years. The questionnaire was then reviewed by members with relevant expertise of the University of Glasgow Ethics Committee and modified according to feedback received. The survey was administered at the same time of year, in November, first in 2008 and then in 2009. This timing was chosen because it preceded taught modules on cancer, including cervical cancer. The questionnaire was completed in a ten minute period at the end of a teaching session in a lecture theatre. One of the researchers introduced the study and explained the ethical permissions and voluntary nature of participation. Students made aware of how anonymity would be ensured. Although signed informed consent was given, the consent forms were collected separately to the questionnaires so that no questionnaire answers could be traced back to an individual. Ethical approval was obtained from the University of Glasgow’s Faculty of Biomedical and Life Sciences Ethics Committee.
In 2008 the timing of the survey was chosen to be immediately following first roll out of the vaccine (September 2008) after a summer's intensive public health campaign. This included television, radio, internet, media, cinema and billboard adverts and age-directed leaflets available in public places, community and health centres and GP surgeries. Few females in the population were vaccinated in 2008. In contrast, in 2009, many of the female students had been offered the vaccine, delivered in association with appropriate PHI. Moreover, there was a sustained high level PHI campaign throughout 2009. In particular, during this period there was extensive media coverage of the illness and death due to cervical cancer of the celebrity Jade Goody [21, 22].
The study group
Age ranges of the study populations (2008: n=150, 2009: n=213) and vaccination status of female respondents
Had 1st injection
Had 1st and 2nd injections
Had all 3 injections
Do not wish vaccination
Data collation and analysis
Data were input into Microsoft Excel 11.5. Exact binomial, Randomisation and Chi-Squared tests were used to compare between group data. Differences were considered significant if p< 0.05. Analyses focused on questions pertaining to knowledge of HPV and the HPV vaccine.
Knowledge of details of HPV and its association with cervical cancer
Frequency of responses to questions in the questionnaire in 2008 and 2009: Students' knowledge of HPV and cervical cancer
Q1: HPV is sexually transmitted between males and females:
Q2: What percentage of cervical cancers do you think HPV is responsible for?
Q3: There are at least 15 types of HPV that can cause cancer. How many of these types do you think the HPV vaccine protects against?
Assessing potential impact of the vaccine on uptake of cervical smear testing
In 2008, 94% and in 2009, 87.3% of the students assessed the cervical screening programme as “very important”. In Scotland, women are invited for their first smear test at age 20. Some of the female students in the survey populations had already attended for testing: in 2008, 16.0% (n=16) of students (97% of those eligible for screening (i.e. ≥20 years of age)) had been for a smear test while in 2009, 13.3% (n=15) (68% of those eligible for screening) had been tested. In both years the majority of women answered positively when asked if they would go for a smear test when invited. Importantly, in 2009, 94% of females who had been fully vaccinated said they would attend for a smear test when invited.
Exposure to public health information on the HPV vaccine
Participants’ rating of public health information (PHI) on the HPV vaccine
Q1. How would you rate the PHI on the HPV vaccine?
Q2. Where have you heard/seen information on the current HPV vaccination programme? Tick all that apply.
Poster in a pharmacy
Word of mouth
I have not seen/heard anything
Many studies in different countries in a range of socio-economic groups have demonstrated an overall lack of awareness of HPV and its link with cervical cancer [9–11, 16, 24, 25]. This study represents one of the first reports of the knowledge of educated young people in the UK of details about HPV infection, the HPV vaccine and cervical cancer, and importantly, how knowledge changed within the first year of introduction of the vaccination programme. The surveys were carefully timed to question the students when they would have been maximally exposed to PHI and media information on HPV and the HPV vaccine. Moreover, the students may have had more motivation to pay attention to details in the PHI than most other young people of this age group because their successful entry to medical school could depend on their displaying clear knowledge and understanding of facts on HPV at interview. There are few studies comparing detailed knowledge of HPV in males and females. The young male population could have a role in encouraging their female peers to be vaccinated so both genders were questioned in each year’s cohort. The study included unvaccinated women and women who were in the age group between being offered the vaccine and being called for cervical screening. The vast majority (99%) of the students, male and female, questioned in each year indicated that they had seen or heard information on the HPV vaccine indicating some efficacy of PHI. The questionnaire assumed that the students would have some knowledge of HPV and cervical cancer and indeed it was clear that the level of knowledge of this educated population was probably greater than that of the general population  as recently suggested .
Knowledge about specific details of HPV infection
The UK PHI leaflets accompanying HPV vaccination note the sexually transmitted nature of HPV. Surprisingly, although most students seemed to appreciate this fact, in agreement with another study , the 2009 survey showed a small decrease in the percentage of students who understood that HPV was sexually transmitted. This was despite a significant step-up in the PHI campaign surrounding HPV and its link with cervical cancer from the summer of 2008 through to autumn 2009 and the media coverage early in 2009 of the death from cervical cancer of the young celebrity Jade Goody [21, 22]. However, the level of knowledge of the sexually-transmitted nature of HPV in our cohorts was much higher than that in another study of a similar age group of school students in Germany where less than 50% understood that HPV was a sexually transmitted infection . It is becoming increasingly clear from similar surveys in a number of countries that even among educated young people there is a lack of understanding of the sexually transmitted nature of HPV infection [18, 19, 29–31].
PHI leaflets clearly state that the vaccine protects against 70% of cases of cervical cancer. However, it is not made clear that the remaining 30% of cases are caused by other HR-HPV types. Consequently, anecdotal evidence suggests that the public may believe it is possible to develop cervical cancer without being infected with HPV [16, 32, 33]. Students were asked to guess what percentage of cervical cancers was caused by HPV. Only around half of men and women in 2008 understood that HPV caused between 80 and 100% of cervical cancers. There was a decrease in understanding this fact between the 2008 and 2009 surveys and a greater percentage of men responded incorrectly in 2009.
Relationship between vaccination and HPV knowledge
The vaccine PHI intended to inform the public that the vaccine protected against two types of HPV. However, in 2008 around half the survey population did not appear to appreciate this fact. Male understanding did not increase by 2009 but females gave a larger number of correct responses in that year. This could be due to the increased percentage of vaccinated female students in 2009 that correlated with an increased number who claimed to have seen or heard HPV information in formal education, as expected due to the PHI delivered to vaccinees. However, as noted above, a lower percentage of the 2009 cohort of women realised that HPV caused most cervical cancers. Therefore our evidence is equivocal that PHI delivered with the vaccine improved HPV knowledge.
Acquiring knowledge: gender differences in HPV knowledge
Although there was some evidence of gender differences in HPV knowledge, especially in the 2009 cohort, gender did not have a significant difference on the sources of information on HPV. However, a much higher percentage of male students reported receiving information from word of mouth and more males in 2009 compared to 2008 were unable to rate PHI on the HPV vaccine. Interestingly, although the males in the study displayed knowledge of HPV association with cervical cancer, the majority answered ‘no’ to a question included in the survey “Do you think males should be offered the vaccine?” This could indicate perception of HPV vaccination as a female issue.
Limitations of the study
The study has some limitations. Prior to administering the questionnaire it is possible that some students had begun to prepare for the oncology portion of their course by reading recommended materials and had thus acquired a greater appreciation of the subject prior to questioning. Some homogenisation of answers could have occurred through students comparing answers in class. The restricted time period (ten minutes) for filling in the questionnaire was designed to avoid this. In itself this may have introduced another limitation because the short time period may have led the students to provide rapid, poorly considered responses.
Public knowledge of HPV, cervical cancer and vaccination is central to ensuring good vaccination coverage in the female population and success of future HPV testing strategies that will reduce the burden of disease. This survey indicates that PHI has been somewhat successful in delivering key elements of HPV knowledge for both genders in the young educated population studied, but that there are still important knowledge gaps especially in males. Continued efforts in pressing home the main messages regarding HPV, the HPV vaccine, and cervical screening are required.
University of Glasgow Faculty of Biomedical Sciences Ethics Committee project approval number FBLS 0825 17/11/2008 and FBLS 0930 01/10/2009.
Public health information
Sexually transmitted infection
We would like to thank Dr. Susan Jamieson for kindly arranging for us to administer the survey. We thank the Year 1 students of medicine at the University of Glasgow in 2008/9 and 2009/10 who took part in the study and the final year virology students in 2008/9 on whom the study was piloted. We thank Dr Katy Sinka, Health Protection Scotland for advice on the questionnaire design and the manuscript. We are grateful to Ms Claire Scott, Senior, Public Health and Health Improvement (Cancer), NHS, Greater Glasgow and Clyde and West of Scotland Cancer Network and Ms Isabel Gavin, National Services Division, for advice on public health issues relating to HPV vaccination. We thank Prof Heather Cubie, Dr Kate Cuschieri and Dr Becky Devine for critical reading of the manuscript. Graham and Campbell are members of the Scottish HPV Investigators Network (http://www.shine.mvm.ed.ac.uk/index.shtml).
- zur Hausen H: Papillomaviruses in the causation of human cancers - a brief historical account. Virol. 2009, 384: 260-265. 10.1016/j.virol.2008.11.046.View Article
- Woodman CBJ, Collins SI, Young LS: The natural history of cervical HPV infection: unresolved issues. Nat Rev Cancer. 2007, 7: 11-22. 10.1038/nrc2050.View ArticlePubMed
- Munoz N, Bosch FX, de Sanjosé S, Herrero R, Castellsague X, Shah KV: Epidemiologic classification of human papillomavirus types associated with cervical cancer. N Engl J Med. 2003, 348: 518-527. 10.1056/NEJMoa021641.View ArticlePubMed
- published on the Department of Health. 2011, http://www.dh.gov.uk/en/Publichealth/Immunisation/keyvaccineinformation,
- Szarewski A: HPV vaccine: Cervarix. Expert Opin Biol Ther. 2010, 10: 477-487. 10.1517/14712591003601944.View ArticlePubMed
- Stanley MA: Human papillomavirus vaccines. Rev Med Virol. 2006, 16: 139-149. 10.1002/rmv.498.View ArticlePubMed
- published on the Department of Health. 2011, https://www.wp.dh.gov.uk/immunisation/files/2013/01/2900783_HPVMonthlySurvey_Dec12_acc.pdf,
- published on Health Scotland. 2012, http://www.healthscotland.com/health/topics/immunisation/HPV.aspx,
- Trim K, Nagji N, Elit L, Roy K: Parental knowledge, attitudes and behaviours towards human papillomavirus vaccination for their children: a systematic review from 2001 to 2011. Obs Gynecol International. 2012, 2012: 1-12.View Article
- Cuschieri K, Horne A, Szarewski A, Cubie HA: Public awareness of human papillomavirus. J Med Screen. 2006, 13: 201-207.PubMed
- Marlow LAV, Waller J, Wardle J: Public awareness that HPV is a risk factor for cervical cancer. Br J Cancer. 2007, 97: 691-694. 10.1038/sj.bjc.6603927.PubMed CentralView ArticlePubMed
- Klug SJ, Hukelmann M, Blettner M: Knowledge about infection with human papillomavirus: A systematic review. Prev Med. 2008, 46: 87-98. 10.1016/j.ypmed.2007.09.003.View ArticlePubMed
- Tiro JA, Meissner HI, Kobrin S, Chollette V: What do women in the U.S. know about human papillomavirus and cervical cancer?. Cancer Epiedmiol Biomarkers Prev. 2006, 16: 288-294.View Article
- Pitts M, Clarke T: Human papillomavirus infections and risks of cervical cancer: what do women know?. Health Educ Res. 2002, 17: 706-714. 10.1093/her/17.6.706.View ArticlePubMed
- Cooper Robbins SC, Bernard D, McCafferty K, Brotherton J, Garland S, Skinner SR: "Is cancer contagious?": Australian adolescent girls and their parents: Making the most of limited information about HPV and HPV vaccination. Vaccine. 2010, 28: 3398-3408. 10.1016/j.vaccine.2010.02.078.View ArticlePubMed
- Hilton S, Smith E: "I thought cancer was one of those random things. I didn't know cancer could be caught." Adolescent girls' understanding and experiences of the HPV programme in the UK. Vaccine. 2011, 29: 4409-4415. 10.1016/j.vaccine.2011.03.101.PubMed CentralView ArticlePubMed
- Licht AS, Murphy JM, Hyland AJ, Fix BV, Hawk LW, Mahoney MC: Is use of the human papillomavirus vaccine among female college students related to human papillomavirus knowledge and perception?. Sex Transm Infect. 2009, 86: 74-78.View ArticlePubMed
- Bowyer HL, Marlow LAV, Hibbets S, Pollock KG, Waller J: Knowledge and awareness of HPV and the HPV vaccine among young women in the first routinely vaccinated cohort in England. Vaccine. 2013, 31: 1051-1056. 10.1016/j.vaccine.2012.12.038.View ArticlePubMed
- Marlow LAV, Zimet GD, McCafferty KJ, Ostini R, Waller J: Knowledge of human papillomavirus (HPV) and HPV vaccination: An international comparison. Vaccine. 2013, 31: 763-769. 10.1016/j.vaccine.2012.11.083.View ArticlePubMed
- Caskey R, Lindau ST, Alexander GC: Knowledge and early adoption of the HPV vaccine among girls and young women: results of a national survey. J Adolesc Health. 2009, 45: 453-462. 10.1016/j.jadohealth.2009.04.021.View ArticlePubMed
- Metcalfe D, Price C, Powell J: Media coverage and public reaction to a celebrity cancer diagnosis. J Pub Health. 2010, 5: 1-6.
- Bell L, Searle C: The reporting of cervical cancer in the mass media: a study of UK newspapers. Eur J Cancer Care. 2010, 20: 389-394.View Article
- published on Information Services Division, Scotland. 2011, http://www.isdscotland.org/Health-Topics/Cancer/Cervical-Screening/,
- Walsh CD, Gera A, Shah M, Sharma A, Powell JE, Wilson S: Public knowledge and attitudes towards Human Papilloma Virus (HPV) vaccination. BMC Publ Health. 2008, 8: 368-377. 10.1186/1471-2458-8-368.View Article
- Das A, Madhwapathi V, Davies P, Brown G, Dearnley E, Spencer A, Williams H: Knowledge and acceptability of the HPV vaccine by school children and their parents in Birmingham. Vaccine. 2010, 28: 1440-1446. 10.1016/j.vaccine.2009.11.041.View ArticlePubMed
- Schmeink CE, Gosens KC, Melchers WJ, Massuger LF, Bekkers RL: Young adults awareness of HPV and vaccine acceptance after introduction of the HPV vaccine in the Dutch national vaccination program. Eur J Gynaecol Oncol. 2011, 32: 418-426.
- Williams K, Forster A, Marlow L, Waller J: Attitudes towards human papillomavirus vaccination: a qualitative study of vaccinated and unvaccinated girls aged 17–18 years. J Fam Plann Reprod Health Care. 2011, 37: 22-25. 10.1136/jfprhc.2010.0017.View ArticlePubMed
- Blödt S, Holmberg C, Müller-Nordhorn J, Rieckmann N: Human papillomavirus awareness, knowledge and vaccine acceptance: A survey among 18–25 year old male and female vocational school students in Berlin, Germany. Eur J Public Health. 2012, 22: 808-813. 10.1093/eurpub/ckr188.View ArticlePubMed
- Samkange-Zeeb F, Spallek L, Klug SJ, Zeeb H: HPV infection awareness and self-reported HPV vaccination coverage in female adolescent students in two German cities. J Comm Health. 2012, 37: 1151-1156. 10.1007/s10900-012-9589-1.View Article
- Juntasopeepun P, Suwan N, Phianmongkhol Y, Srisomboon J: Factors influencing acceptance of human papillomavirus vaccine among young female college students in Thailand. Int J Gynaecol Obstet. 2012, 118: 247-250. 10.1016/j.ijgo.2012.04.015.View ArticlePubMed
- Ghotbi N, Anai A: Assessment of the knowledge and attitudes of female students towards cervical cancer prevention at an international university in Japan. Asian Pac J cancer Prev. 2012, 13: 897-900. 10.7314/APJCP.2012.13.3.897.View ArticlePubMed
- Mosavel M, El-Shaarawi N: "I haven't heard that one": young girls' knowledge and perception of cervical cancer. J Health Comm. 2007, 12: 707-719. 10.1080/10810730701671985.View Article
- Lloyd GP, Marlow LAV, Waller J, Miles A, Wardle J: An experimental investigation of the emotional and motivational impact of HPV information in adolescents. J Adolesc Health. 2009, 45: 532-534. 10.1016/j.jadohealth.2009.06.003.View ArticlePubMed
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/13/264/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.