Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

A research tool for measuring non-participation of older people in research on digital health

BMC Public Health volume 19, Article number: 1487 (2019) Cite this article

Abstract

Background

Healthcare services are being increasingly digitalised in European countries. However, in studies evaluating digital health technology, some people are less likely to participate than others, e.g. those who are older, those with a lower level of education and those with poorer digital skills. Such non-participation in research – deriving from the processes of non-recruitment of targeted individuals and self-selection – can be a driver of old-age exclusion from new digital health technologies. We aim to introduce, discuss and test an instrument to measure non-participation in digital health studies, in particular, the process of self-selection.

Methods

Based on a review of the relevant literature, we designed an instrument – the NPART survey questionnaire – for the analysis of self-selection, covering five thematic areas: socioeconomic factors, self-rated health and subjective overall quality of life, social participation, time resources, and digital skills and use of technology. The instrument was piloted on 70 older study persons in Sweden, approached during the recruitment process for a trial study.

Results

Results indicated that participants, as compared to decliners, were on average slightly younger and more educated, and reported better memory, higher social participation, and higher familiarity with and greater use of digital technologies. Overall, the survey questionnaire was able to discriminate between participants and decliners on the key aspects investigated, along the lines of the relevant literature.

Conclusions

The NPART survey questionnaire can be applied to characterise non-participation in digital health research, in particular, the process of self-selection. It helps to identify underrepresented groups and their needs. Data generated from such an investigation, combined with hospital registry data on non-recruitment, allows for the implementation of improved sampling strategies, e.g. focused recruitment of underrepresented groups, and for the post hoc adjustment of results generated from biased samples, e.g. weighting procedures.

Peer Review reports

Background

More and more studies are being conducted to evaluate new digital health technologies that might be incorporated into the provision of healthcare services and also meet the needs of older people, among other user groups [1]. The results are promising and show that digital health technologies can bring benefits at individual and organisational levels [1,2,3]. However, such results are often obtained based on evaluations conducted with selected groups of individuals who are typically younger, are better educated and have higher digital skills compared with older end-users [4,5,6,7]. In many cases, older end-users are still less likely to be digitally engaged and experienced [8], and hence risk being disadvantaged by such digital shift.

The overall aim of the present paper is to introduce, discuss and test an instrument to measure non-participation in digital health studies. Applying such an instrument can generate information for improved sampling strategies in digital health studies and provide data to enhance the quality of research outcomes by post hoc adjustments of results generated from biased samples.

First, we explore the nature of non-participation and its relevance in digital health research. Second, we discuss the conceptualisation of an approach to study non-participation and the two processes behind it – non-recruitment and self-selection. Third, we introduce an instrument for the analysis of self-selection. Fourth, we present the results of a pilot study of the instrument. Finally, we consider its use and applicability and knowledge generated by it.

Digitalisation of healthcare services

European national healthcare systems have started to move towards a provision of healthcare which also includes services offered by digital means, such as scheduling medical appointments on the web, redirecting patients to web portals for health information and education, communicating remotely with care providers (e.g., e-Prescription), accessing personal health records (PHRs) online, and remote monitoring of patients’ health (e.g., telemedicine, telehealth) [9, 10]. This trend is driven by broader policy strategies that foster the creation of a digital society as a key to economic growth and sustainability [11] and has been decisively influenced by the accumulation of available technological advances [12].

Older people are expected to embrace such technological shifts in healthcare just as much as other age groups [12]. Several studies have been conducted to explore the potential benefits resulting from incorporating digital health technologies, at both individual and organisational levels, and some potentials have been recognised [1, 2].

Digital inequalities among older people

Although digital health incorporation appears to be very promising, the way individuals embrace the opportunities that digitalisation offers can vary substantially [13], as do the benefits each individual gains from it. Even though the proportion of older people using technology and the Internet has increased significantly in recent years, a high proportion of older people are still not digital users or experienced digital users [14, 15].

On the one hand, the lower engagement of older people with digital technologies may reflect several different aspects, ranging from a lack of skills and material resources [16] – which is often related to their sociodemographic and socioeconomic backgrounds [17, 18] – and a lack of exposure to technology over their lifetimes [8, 18], to intentionally choosing not to use new digital technologies [19]. Moreover, physical and cognitive impairments can reduce the use of technology among older people [20, 21]. Focusing specifically on digital health technologies, those who, for example, are in the oldest age groups, have a lower socioeconomic status, have poorer digital skills and do not perceive digital health as useful in their everyday life, are less likely to engage with such services [20, 22,23,24].

Digital health research can be a driver of exclusion if some groups of people, such as more vulnerable groups, are underrepresented and their specific needs are neglected. Such limitations of digital health research can lead to these groups of people being denied the opportunity to benefit from new digital technologies [5, 25].

Non-participation as a driver of inequalities

When conducting health-related research evaluating new digital health technologies, some individuals are less likely to be involved than others. Such a selection occurs during the process of recruiting participants, where a combination of study requirements and individual decisions to participate makes it challenging to recruit a group of study participants which is large and representative enough to reflect the heterogeneity of the population of interest [6, 25,26,27,28,29].

Health-related research often suffers from non-participation which can bias the results of the research and lead to the wrong conclusions. Non-recruitment derives from the non-recruitment of targeted individuals [30,31,32] and individual decisions not to participate [6, 26, 28]. As a result, many studies involve highly selected samples of individuals [5, 25] which do not reflect the overall population of interest.

Study participants and non-participants can differ considerably, with the former group typically being younger and in better health, and having higher digital skills and better socioeconomic conditions [7, 33].

The differences between participants and non-participants indicate that these two groups can differ in terms of their needs. Therefore, the underrepresentation of some groups of people in the research implies that some of the needs and interests of the target population are neglected and research results are biased.

Process 1: non-recruitment

In the process of recruiting study participants, medical conditions and scientific reasons can guide evaluations of eligible patients for the study and, thus, the non-recruitment of others. This represents the first process of selection, namely non-recruitment, which distinguishes eligible individuals from non-recruited individuals (Fig. 1). This often includes the non-recruitment of, for example, the oldest old, people with hearing and sight impairments, people with cognitive impairments and dementia, people with multiple comorbidities and people with poor medical compliance, who are deemed to be more likely to drop-out from a research study [30,31,32, 34,35,36].

Fig. 1
figure 1

Processes behind non-participation in the process of recruiting study participants. Legend: Non-participation results from two processes: (1) non-recruitment and (2) self-selection. (1) the non-recruitment process distinguishes eligible individuals from non-recruited individuals, within the target population, and (2) the self-selection process distinguishes study participants from those who decline to participate, among the eligible individuals

Full size image

Process 2: self-selection

In addition to non-recruitment, a further key mechanism of selection occurs for those people who have been deemed eligible and are thus invited to take part in the research: the self-selection (Fig. 1). Self-selection is based on the individual decision to participate in the study and distinguishes consenting people (i.e., the final group of study participants) from those who decline to participate.

Self-selection has often been reported in health-related and digital health research. Among patients with chronic obstructive pulmonary disease invited to participate in a controlled telemedicine trial, more than 50% declined the invitation to participate [26]. More than 75% of eligible patients declined to participate in a study testing homecare telemedicine in primary care [28], while the 80% of invited older patients to a randomised controlled trial (RCT) of home telecare did not agree to participate in the research [6].

Several predictors of self-selection in health-related research are discussed in the literature, such as socioeconomic conditions, health and quality of life (QOL), social participation and time resources. Research participants are likely to be younger than those who decline to participate [6, 26, 37,38,39,40,41]. Decliners have also been found to have lower levels of education [7, 41, 42] and lower QOL [42], worse health [37, 38, 41, 43,44,45], more cognitive impairment [45] and higher levels of perceived social support [41] compared to participants. The role of gender is less clear. Some studies show that men are more likely than women to participate in health-related research [7, 26, 39, 40], while others demonstrate the opposite [46] or no [38] effect. Lack of time and competing tasks have been described as further reasons for declining participation in health-related research [33, 38, 47,48,49], and in some cases the lack of time was due to care-related activities [37]. Participants, on the other hand, participated in the research because they hoped to improve their health and benefit from more care and support [50], and to contribute to science and society [50, 51].

Reasons for not participating when invited to participate in an evaluation of digital health technologies can often be attributed to technology, such as a lack of access to the Internet and computer devices [33], a lack of skills or of familiarity with computer and Internet use [33, 49, 52, 53], or a lack of perceived usefulness of technology (e.g., the belief they could not get any additional benefit from the technology) [28, 49].

The two processes of non-participation – non-recruitment and self-selection – work independently, but together they determine the selectivity in a sample. This affects the appropriateness of the results regarding accessibility, usability and suitability of new digital solutions. Outcomes might differ between underrepresented people and those who are more likely to participate. These considerations call for the development of an innovative research approach to understand the problem of non-participation and improve the results of digital health research accordingly.

Methods

We developed the research approach of NPART (Evaluation of Non-Participation in Digital Health Research) for measuring and addressing non-participation of older people in digital health research, within the Supporting Self-Care by Information and Communication Technology (ICT) for Older People with Long-Term Conditions (ICT4Self-care) research programme (2015–2018) funded by the Swedish National Science Council and the Swedish Research Council for Health, Working Life and Welfare (VR-FORTE) (ref. 2014–4100).

The NPART research approach

Non-participation is conceptualised as consisting of non-recruitment and self-selection processes (Fig. 1), and is studied in two phases of data collection. Following the two phases of data collection, comparisons between non-recruited, decliners and participants are conducted. Such an investigation based on hospital and survey data serves as a key basis for implementing improved sampling strategies, for example focused recruitment aiming at reaching specific underrepresented groups [54], and/or for conducting post hoc adjustments of the research outcomes, for example weighting outcomes according to the information on non-participation [55].

In a first phase, data collection must focus on the population at the recruitment site that has been defined as the target for the research, and which thus includes both the older people who will participate and those who will not due to non-recruitment or self-selection. Data consists of hospital or further registry information in combination with a recruitment log. Information based on hospital data allows for comparisons between the non-recruited, the decliners and the participants.

In a second phase, data collection must focus on those individuals who were deemed eligible for the research and were thus invited to participate. Therefore, those who will subsequently both consent and decline are approached. Data must consist of information on key aspects associated with the individual decision to participate. Data is collected through a survey questionnaire and combined with the information on (non-) participation from the recruitment log and with the reasons for declining, which could be collected as free responses during the recruitment process.

Information from the survey questionnaire allows for a more detailed comparison between decliners and participants, and for an interpretation of the decision on whether participate in the research.

To conduct this second phase of data collection, we developed an instrument – the NPART survey questionnaire – to explore key determinants of self-selection by including factors which are known to be associated with preparedness to participate in health-related research. In order to increase the response rate among the decliner group, the survey questionnaire should consist of a limited number of easy-to-answer questions, as recommended by the literature on non-response instruments [56, 57]. The NPART survey questionnaire consists of a limited number of short question items with a restricted number of response items.

The goal of the pilot study is to verify the construct validity [58, 59] of the questions used in the NPART survey questionnaire, namely the ability to cover the thematic dimensions of non-participation necessary in order to understand self-selection in digital health research.

Designing the NPART survey questionnaire

Thematic areas for the survey questionnaire are based on a review of the literature. We focused on predictors for individual decisions to participate in digital health research, resulting in five thematic areas: socioeconomic factors, subjective health and subjective overall quality of life (subjective overall QOL), social participation, time resources, and use of technology and digital skills.

The NPART survey questionnaire consists of 36 questions (see Additional file 1). The questions for the NPART survey questionnaire were retrieved from major ageing studies and existing survey questionnaires. Some question items were adapted or developed. The main sources were the survey instrument of the Survey of Health, Ageing and Retirement in Europe (SHARE) [60], the Minimum European Health Module (MEHM) [61] and the World Health Organization Quality-of-Life Scale (WHOQOL-BREF) [62].

One additional question was included only for the purpose of the pilot study. This investigates preparedness to participate, namely the possible interest in participating in digital health research. Hence, it provides direct information on the individual decision: ‘If you received an invitation to participate in research testing a new technology that allows you to access healthcare services by phone, tablet or computer, would you like to participate in this research?’, with the response options ‘Yes’ and ‘No’. Such a question served only for the purpose of the pilot study to discriminate between participants and non-participants based on their preparedness to participate. Therefore, it is not intended to investigate reasons for participation or non-participation.

The socioeconomic factors were investigated with seven questions on age, gender, education (‘What is the highest level of education you have completed?’), job position (‘What is/was your primary job?’, ‘Which of the following categories best describes your primary area of employment?’), marital status (‘Do you have a partner?’) and living alone (‘Do you live alone?’).

Health and subjective overall QOL are covered by five questions investigating self-rated health (‘How is your health in general? It is’), health limitations (‘During the last 6 months, to what extent have you been limited because of your health in activities people usually do? Would you say you have been’), memory (‘How would you rate your memory at the present time? Would you say it is’), subjective overall QOL (‘How would you rate your quality of life?’) and self-reported need for support (‘Do you require any help taking care of your health, such as taking medications or attending/booking medical appointments?’).

Social participation is measured by four questions on involvement in social activities (‘During the past 12 months, how often have you participated in social activities such as volunteer or charity work, attended a training course, visited a sports club, social club or other kind of club, participated in the activities of a religious organisation, or participated in the activities of a political or community-related organisation?’), contact with children (if relevant) (‘During the past 12 months, how often did you have contact with your children, either in person, by phone, by mail, by e-mail or by any other electronic means?’, with response options including ‘I do not have any children’), contact with friends (‘During the past 12 months, how often did you have contact with or meet your friend and/or neighbours?’), and appropriateness of the amount of social contact (‘In the past 12 months, would you like to have had more contact with or met more frequently your children, relatives and/or friends?’).

Questions on time resources was measured by job hours (two questions: ‘Are you currently employed? If yes, how many hours per week?’) and caregiving (one question: ‘How often in the past 12 months did you care for a sick or disabled person?’).

Technology-related aspects were studied through questions investigating digital skills (ten questions: ‘How well do you think you master the following activities? Sending/receiving emails / Buying goods or services over the Internet / Reading or downloading online news, newspaper or magazines / Internet banking / Accessing institutions / Playing or downloading games, images, films or music / Listening to web radio or watching web television / Telephoning or making video calls over the Internet / Social networking, for example Facebook or Twitter / Posting messages to chat sites, blogs or forums, or instant messaging’), use of digital technologies (four questions: ‘Finally, we would like to ask you how often you use: a computer / a mobile phone / a smartphone and/or a tablet / a smart television and/or a games console’), perceived usefulness of digital health (three questions: ‘Do you think using a mobile phone, smartphone, tablet or the Internet might … Support you in performing everyday activities / Be useful in monitoring your health / Be useful for contacting nurses, physicians and other healthcare professionals’). A blank space for respondents to comment was also included.

The English version of the survey questionnaire was the master version. Regarding the development of the Swedish version, some of the items retrieved from existing survey questionnaires were already available in Swedish. If not, the English items were translated into Swedish by a native speaker.

All categorical variables that were not originally binary were re-coded as binary variables (e.g., ‘high/low’, ‘poor/good’), except for job position (four categories: ‘workers’, ‘clerks’, ‘self-employed’, ‘retired/no job’) and job hours (three categories: ‘no job/retired’, ‘up to 38 hours per week’, ‘39 or more hours per week’). As the goal of the pilot study was to understand the ability of the survey instrument to discriminate between decliner and consenting groups, the categories of the variables (e.g., ‘high/low’, ‘poor/good’) were determined in such a way as to maximise the differences between decliner and consenting people.

Education was coded as up to nine years (‘low’) and ten years or more (‘high’). Self-rated health was categorised as ‘poor’ (corresponding to the response items ‘fair’, ‘bad’ and ‘very bad’) and ‘good’ (corresponding to the response items ‘very good’ and ‘good’).

Regarding health limitations, we used the categories ‘having health limitations’ (which included those who answered that they were ‘severely limited’ or ‘limited but not severely’) and ‘not having limitations’ (which covered those people who answered ‘not limited at all’). Subjective overall QOL was re-coded as ‘low’, which corresponded to the answers ‘poor’ and ‘very poor’ or ‘neither poor nor good’, and ‘high’ which referred to reporting ‘very good’ or ‘good’ subjective overall QOL. Also, self-perceived memory performance was re-coded as ‘poor’ (for the answers ‘fair’, ‘bad’ and ‘very bad’ memory) and ‘good’ (for the answers ‘good’ and ‘very good’ memory). The self-reported need for support was expressed as occurring ‘often/sometimes’ (i.e., ‘almost daily’, ‘almost every week’ or ‘almost every month’) or ‘rarely/never’ (i.e., less often).

Social participation was described as ‘low’ and ‘high’, where the former corresponded to participating in various activities less often than monthly (response item: ‘less often’) and the latter referred to taking part in social activities ‘almost daily’, ‘almost every week’ or ‘almost every month’. Contact with children was described as occurring with ‘high frequency’ (which corresponded to the response items ‘almost daily’, ‘almost every week’ and ‘almost every month’) or ‘low frequency’ (i.e., ‘less often’ than every month). The variable ‘contact with friends’ was re-coded to ‘low frequency’, which referred to having contact ‘almost every month’ or ‘less often’ than every month, and ‘high frequency’, which corresponded to staying in contact with friends ‘almost daily’ or ‘almost every week’. The appropriateness of social contact was categorised as ‘I want more social contact’, which referred to wishing for ‘much more’ or ‘a little more’ contact, and ‘I want less/I do not want more contact’. The informal caregiving provided was re-coded as being provided ‘never/rarely’ or ‘often’.

Regarding the questions with sub-items, we added together the sub-items and summarised them into two categories (i.e., ‘low’ and ‘high’). The highest score for self-assessed digital skills was 50 (i.e., if respondents rated their skills as ‘excellent’ in all the activities) and the lowest was 10 (i.e., if the respondents rated their skills as ‘very poor’ in all the activities). We re-coded self-assessed digital skills as ‘low’ if the respondent scored between 10 and 37 overall and as ‘high’ if the respondent scored between 38 and 50 overall. The total score for the question on the use of digital technologies ranged between 4 and 20. ‘Low’ use of digital technologies corresponded to a score which ranged between 4 and 13, whereas ‘high’ use ranged between 14 and 20. Finally, respondents could score between 3 and 9 for the question on the perceived usefulness of digital technologies. We re-coded the perceived usefulness of digital technologies as ‘high’ for a total score between 7 and 9, and ‘low’ for a total score between 3 and 6.

The continuous variable ‘age’ was summarised by mean and standard deviation; all categorical variables were summarised by frequency and percentage.

Piloting the NPART survey questionnaire

The NPART survey questionnaire was pre-tested with two study persons aged 68 and 61 in order to test the readability of the questions and to estimate the time needed for completion. After that, a pilot study was carried out with 70 older study persons (aged 60 or older) recruited at a day surgery unit at the Motala Hospital in Sweden. Study persons were approached during a recruitment process for a trial study within the ICT4Self-Care programme.

The average age of the participants was 68.8 years (SD = 6.2). Over half of the respondents (n = 37) were male and 33 were female. Of the 70 individuals involved, 45 reported they would agree to participate in research testing digital health technologies, while 25 of them would not.

Analysis

The distribution of variables was compared with respect to preparedness to participate. For some questions we expected differences in the individual preparedness to participate, while for other questions we did not (e.g., gender, marital status, living alone).

To determine whether there was an association between preparedness to participate and the categorical variables investigated, the Chi-square test (χ2) or Fisher’s exact test were used depending on the expected frequencies in any of the cells of the contingency tables for a given variable. Fisher’s exact test was preferred if the frequency for any of the cells for a given variable was expected to be lower than five. Student’s t-test was used for continuous variables (i.e. age).

All analyses were performed using Stata v.14 [63].

Results

The pre-test of the survey questionnaire suggested that the questions were all well readable. The survey questionnaire took approximately 10 to 15 min for the two participants to complete.

Socioeconomic factors

In accordance with the literature, people who would agree to participate (i.e., the consenting group) in digital health research were on average slightly younger than those who would not agree to participate (i.e., the decliner group), with a mean age = 69.6, SD = 6.1 and a mean age = 68.3, SD = 6.2 respectively, and had a higher level of education (27 out of 44 respondents) compared with the decliner group (10 out of 25), although the differences were not significant (Table 1). We did not expect any associations between gender, job position, marital status or living alone and preparedness to participate.

Table 1 NPART questions on socioeconomic factors by preparedness to participate or not in digital health research
Full size table

Health and subjective overall QOL

In line with the relevant literature, people in the decliner group self-rated their memory as poorer (11 out of 24) in comparison with the consenting group (7 out of 45), χ2 (1, n = 69) = 7.44, p = 0.006 (Table 2).

Table 2 NPART questions on health-related aspects and subjective overall QOL, by preparedness to participate or not in digital health research
Full size table

Also, respondents in the decliner group had poorer self-rated health (6 out of 24) and more health limitations (18 out of 24) than those who would agree to participate (6 out of 39 and 27 out of 45 respectively), even though such frequencies were not significantly different. In line with previous studies, older people in the consenting group tended to show higher subjective overall QOL (41 out of 45) than those in the decliner group (20 out of 24).

In contrast to what we expected, the item on self-reported need for support did not clearly discriminate between the two groups. This suggests that this item might need to be reconsidered.

Social participation

We found that people in the consenting group had higher social participation (25 out of 44) compared to those who would not agree to participate in digital health research (7 out of 24), χ2 (1, n = 68) = 4.76, p = 0.029 (Table 3). Consenting individuals also had more contact with their children and friends than the decliner group did, although the frequencies were not significantly different. The consenting group expressed a desire for more social contact with family and friends (22 out of 43) compared with the respondents in the decliner group (6 out of 23), χ2 (1, n = 67) = 4.33, p < 0.05.

Table 3 NPART questions on social participation by preparedness to participate or not in digital health research
Full size table

Time resources

People in the decliner group provided slightly more informal care (5 out of 22) compared to the other group (6 out of 43), although this difference was not statistically significant. However, this result must be interpreted with caution since we discovered that the wording in the item was to some extent misleading. In particular, it was not clear that the item was about informal care and did not include formal care. Hence, we cannot interpret answers which relate to respondents working in healthcare occupations. In contrast to our expectations, we found no effect of working hours on preparedness to participate (Table 4).

Table 4 NPART questions on time resources by preparedness to participate or not in digital health research
Full size table

Digital skills, use of digital technologies and perceived usefulness of digital technologies

In accordance with the literature, respondents in the consenting group showed higher digital skills (24 out of 42) compared to those in the decliner group (6 out of 21), χ2 (1, n = 63) = 4.58, p = 0.032, and higher use of digital technologies (33 out of 41) than the respondents who would decline to participate in digital health research (11 out of 20), χ2 (1, n = 61) = 4.34, p = 0.037 (Table 5). Also, although the frequencies did not differ significantly, the respondents in the consenting group indicated a higher perceived usefulness of digital technologies (28 out of 45) compared to the people in the decliner group (9 out of 24).

Table 5 NPART questions on technology-related aspects by preparedness to participate or not in digital health research
Full size table

Summary of the results

In accordance with previous studies [31, 34], people in the consenting group were on average slightly younger and more highly educated compared to the decliners.

The questions on subjective health and subjective overall QOL discriminated between people in the decliner and consenting groups, especially in terms of self-perceived memory performance. People in the consenting group reported having a better memory than the decliners, which is in line with the relevant literature suggesting that individuals with cognitive impairments are less often represented in health-related studies [32, 36].

We found an association between preparedness to participate and aspects related to social participation. People in the consenting group said that they participated more in social activities and wanted more social contact than they currently have, compared to the people in the decliner group.

As suggested by previous studies [33, 52], consenting and decliner people differed regarding technology-related aspects. People in the consenting group reported having better digital skills and using digital technologies more often, compared to the decliners.

Some of the determinants investigated by the survey questionnaire did not clearly discriminate between the consenting and decliner groups (e.g., age, education, health status, health limitations). Nevertheless, we could observe differences between the two groups which pointed in the expected directions. The absence of clearer effects is presumably due to the nature and the size of the sample.

Discussion

The overall aim of this paper was to introduce, discuss and test an instrument designed to measure non-participation in digital health research. Such an instrument would serve to generate information which is needed for implementing further sampling strategies and/or post hoc adjustments of outcomes produced from biased samples.

First, we examined the nature of non-participation and its importance in digital health research. Second, we discussed the conceptualisation of an approach to study non-participation and the two processes behind it – non-recruitment and self-selection. Based on that, we introduced an instrument for analysing self-selection and then presented the results of a pilot study.

The process of non-recruitment can be analysed using hospital or further registry data. Such hospital data is usually quite limited in terms of indicators, and we need to add survey data from those who may prefer not to participate to understand the full process of non-participation thoroughly. This means developing a survey instrument that allows self-selection to be described with a non-response study approach.

In line with the theoretical assumptions on the self-selection process, the results from the pilot study of the survey questionnaire show that the instrument is sensitive to the differences between participants and non-participants, even if we only use short and simple indicators for the short non-response instrument. Compared with decliners, people in the consenting group were on average slightly younger and more educated. They reported having better memory, higher social participation, better skills and to be more frequent users of digital technologies. This survey questionnaire can be used successfully to analyse the self-selection of older people and is suggested for further use in digital health research.

As an instrument for the analysis of self-selection, the survey questionnaire must be viewed in combination with the reasons collected for declining. Overall, such a combination of data allows for the interpretation of the individual decision to participate in a study testing new digital health technologies. In the broader context of addressing non-participation in digital health research, the survey instrument provides researchers with the information needed to implement improved sampling strategies and/or outcome corrections.

The information generated by the NPART research approach can guide focused recruitment among underrepresented groups, for example those with lower digital skills or cognitive impairments. In addition, information based on the NPART research approach is crucial for performing post hoc corrections to the study outcomes by, for example, weighting procedures that give different weights to different groups in the sample according to the proportion they represent in the total target population of interest. In sum, by allowing such strategies, data generated by the NPART research approach and its questionnaire prevents overestimating and underestimating the effects of an intervention on a target population and improves research results.

This investigation is not without limitations. The pilot study was conducted on a small non-random sample and we believe this prevented us from finding stronger associations between non-participation and key aspects investigated. Nonetheless, the directions of the results were overall in line with the relevant literature on non-participation.

As a short non-response instrument, the NPART survey questionnaire investigates only a selection of aspects for each thematic area. Therefore, some detailed information might be missed, such as details on specific health impairments. However, some of this information, e.g., hearing and sight impairments, dementia and cognitive impairments, might be accessed from hospital data or other registries.

The items that investigated time resources and the self-reported need for support should be reconsidered as they were not sufficiently sensitive to capture the differences between participants and non-participants, as expected. In connection with time availability, some relevant aspects were not covered in the questionnaire, e.g., taking care of grandchildren and children. This should also be re-examined. Moreover, as the item on informal caregiving does not clearly measure informal care, we cannot interpret the results from it.

Finally, it is likely that the NPART survey questionnaire will encounter non-response itself, especially from those people who are typically the hardest to reach. The results of an investigation on preparedness to participate would not be changed in such an eventuality but might show a smaller effect.

Conclusions

The NPART survey questionnaire is valid for analysing individual preparedness to participate in digital health research along the lines of the relevant literature.

As a whole, the NPART research approach can be used to clarify and deepen the mechanisms of non-participation in the exclusion of people from the digital health research and to illuminate the differences between older study participants and older non-participants.

The NPART research approach and its survey instrument are flexible in their application. They can be embedded into a wide range of recruitment processes in digital health studies and target diverse populations.

Recruiting representative samples of people will always present a challenge. Some people are especially hard to reach and recruit in digital health research, such as older people and those who are less educated, less familiar with technological advances and less involved socially. These people are often excluded and underrepresented in research, even though they are targeted by the digitalisation process which is changing the provision of healthcare services. By studying the two steps of non-participation – non-recruitment and self-selection – and characterising participants and non-participants, strategies supporting needs recognition among underrepresented groups can be implemented to improve research outcomes of digital health research and better inform policy and practice.

Availability of data and materials

The materials and datasets used and analysed in the current study are available from the corresponding author on request.

Abbreviations

ICT:

Information communication technology

MEHM:

Minimum european health module

NPART:

Evaluation of non-participation in digital health research

QOL:

Quality of life

SHARE:

Survey of health, ageing and retirement in Europe

WHOQOL-BREF:

World health organization quality-of-life scale

References

  1. Lindeman DA. Improving the independence of older adults through technology: directions for public policy. Public Policy Aging Rep. 2017;27(2):49–52.

    Article  Google Scholar 

  2. Czaja SJ. The potential role of technology in supporting older adults. Public Policy Aging Rep. 2017;27(2):44–8.

    Article  Google Scholar 

  3. Lloyd L. Health and care in ageing societies: a new international approach. Bristol: Policy Press; 2012.

  4. Grates MG, Heming A-C, Vukoman M, Schabsky P, Sorgalla J. New perspectives on user participation in technology design processes: an interdisciplinary approach. The Gerontologist. 2018;59(1):45-57.

    Article  Google Scholar 

  5. Merkel S, Kucharski A. Participatory Design in Gerontechnology: a systematic literature review. The Gerontologist. 2018;59(1):e16–25.

    Article  Google Scholar 

  6. Mair F, Goldstein P, Shiels C, Roberts C, Angus R, O’Connor J, Haycox A, Capewell S. Recruitment difficulties in a home telecare trial. J Telemed Telecare. 2006;12(1_suppl):26–8.

    Article  PubMed  Google Scholar 

  7. Green BB, Anderson ML, Ralston JD, Catz S, Fishman PA, Cook AJ. Patient ability and willingness to participate in a web-based intervention to improve hypertension control. J Med Internet Res. 2011;13(1). https://doi.org/10.2196/jmir.1625.

    Article  PubMed  PubMed Central  Google Scholar 

  8. König, R., Seifert, A., Doh, M. Internet use among older Europeans: an analysis based on SHARE data. Universal Access in the Information Society. 2018;17(3):621-633.

    Article  Google Scholar 

  9. World Health Organization: Global diffusion of eHealth-making universal health coverage achievable: report of the third global survey on eHealth: World Health Organization; 2017.

    Google Scholar 

  10. Barbabella F, Melchiorre MG, Quattrini S, Papa R, Lamura G. How can eHealth improve care for people with multimorbidity in Europe. Policy Brief. 2017;25.

  11. European Commission. Communication from the commission to the european parliament, the council, the european economic and social committee and the committee of the regions eu eGovernment Action Plan 2016-2020 - Accelerating the digital transformation of government. 2016. https://ec.europa.eu/newsroom/dae/document.cfm?doc_id=15268. Accessed 31 Oct 2019.

  12. Mantovani E, Turnheim B. Navigating the European landscape of ageing and ICT: policy, governance and the role of ethics. In: Ageing and Technology Perspectives from the social sciences, Transcript Verlag, Bielefeld; 2016. p. 227–56.

    Chapter  Google Scholar 

  13. Lupton D. Digital health: critical and cross-disciplinary perspectives. New York: Routledge; 2017.

    Chapter  Google Scholar 

  14. Pew Research Center: Tech Adoption Climbs Among Older Adults. 2017. https://www.pewresearch.org/internet/2017/05/17/tech-adoption-climbs-among-older-adults/. Accessed 31 Oct 2019.

  15. Eurostat: People in the EU - statistics on an ageing society. 2017. https://ec.europa.eu/eurostat/statistics-explained/index.php/People_in_the_EU_-_statistics_on_an_ageing_society. Accessed 31 Oct 2019.

  16. Robinson L, Cotten SR, Ono H, Quan-Haase A, Mesch G, Chen W, Schulz J, Hale TM, Stern MJ. Digital inequalities and why they matter. Inf Commun Soc. 2015;18(5):569–82.

    Article  Google Scholar 

  17. Siren A, Knudsen SG. Older adults and emerging digital service delivery: a mixed methods study on information and communications technology use, skills, and attitudes. J Aging Social Policy. 2017;29(1):35–50.

    Article  Google Scholar 

  18. Gilleard C, Higgs P. Internet use and the digital divide in the English longitudinal study of ageing. Eur J Ageing. 2008;5(3):233.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Wyatt SM, Oudshoorn N, Pinch T. Technology: Non-users also matter: The construction of users and non-users of the Internet; 2003. p. 67–79.

  20. Gell NM, Rosenberg DE, Demiris G, LaCroix AZ, Patel KV. Patterns of technology use among older adults with and without disabilities. Gerontologist. 2015;55:412–21.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Keränen SN, Kangas M, Immonen M, Similä H, Enwald H, Korpelainen R, Jämsä T. Use of information and communication technologies among older people with and without frailty: a population-based survey. J Med Internet Res. 2017;19(2):e29.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Gordon NP, Hornbrook MC. Differences in access to and preferences for using patient portals and other eHealth technologies based on race, ethnicity, and age: a database and survey study of seniors in a large health plan. J Med Internet Res. 2016;18:e50.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Gordon NP, Hornbrook MC. Older adults’ readiness to engage with eHealth patient education and self-care resources: a cross-sectional survey. BMC Health Serv Res. 2018;18(1):220.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Yoon H, Jang Y, Vaughan PW, Garcia M. Older adults’ internet use for health information: digital divide by race/ethnicity and socioeconomic status. J Appl Gerontol. 2018. https://doi.org/10.1177/0733464818770772.

  25. Glasgow RE. eHealth evaluation and dissemination research. Am J Prev Med. 2007;32(5):S119–26.

    Article  PubMed  Google Scholar 

  26. Broendum E, Ulrik CS, Gregersen T, Hansen EF, Green A, Ringbaek T. Barriers for recruitment of patients with chronic obstructive pulmonary disease to a controlled telemedicine trial. Health Informatics J. 2016. https://doi.org/10.1177/1460458216667166.

    Article  PubMed  Google Scholar 

  27. Jones RB, O'Connor A, Brelsford J, Parsons N, Skirton H. Costs and difficulties of recruiting patients to provide e-health support: pilot study in one primary care trust. BMC Med Inform Decision Making. 2012;12(1):25.

    Article  Google Scholar 

  28. Subramanian U, Hopp F, Lowery J, Woodbridge P, Smith D. Research in home-care telemedicine: challenges in patient recruitment. Telemed J e-Health. 2004;10(2):155–61.

    Article  PubMed  Google Scholar 

  29. Gaertner B, Seitz I, Fuchs J, Busch MA, Holzhausen M, Martus P, Scheidt-Nave C. Baseline participation in a health examination survey of the population 65 years and older: who is missed and why? BMC Geriatr. 2016;16(1). https://doi.org/10.1186/s12877-016-0185-6.

  30. Britton A, McKee M, Black N, McPherson K, Sanderson C, Bain C. Threats to applicability of randomised trials: exclusions and selective participation. J Health Services Res Policy. 1999;4(2):112–21.

    Article  CAS  Google Scholar 

  31. Cherubini A, Oristrell J, Pla X, Ruggiero C, Ferretti R, Diestre G, Clarfield AM, Crome P, Hertogh C, Lesauskaite V. The persistent exclusion of older patients from ongoing clinical trials regarding heart failure. Arch Intern Med. 2011;171(6):550–6.

    Article  PubMed  Google Scholar 

  32. Taylor JS, DeMers SM, Vig EK, Borson S. The disappearing subject: exclusion of people with cognitive impairment and dementia from geriatrics research. J Am Geriatr Soc. 2012;60(3):413–9.

    Article  PubMed  Google Scholar 

  33. Foster A, Horspool KA, Edwards L, Thomas CL, Salisbury C, Montgomery AA, O’Cathain A. Who does not participate in telehealth trials and why? A cross-sectional survey. Trials. 2015;16(1):258.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Bourgeois FT, Olson KL, Tse T, Ioannidis JP, Mandl KD. Prevalence and characteristics of interventional trials conducted exclusively in elderly persons: a cross-sectional analysis of registered clinical trials. PLoS One. 2016;11(5):e0155948.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  35. Thake M, Lowry A. A systematic review of trends in the selective exclusion of older participant from randomised clinical trials. Arch Gerontol Geriatr. 2017;72:99–102.

    Article  PubMed  Google Scholar 

  36. Domingos JM, Godinho C, Dean J, Coelho M, Pinto A, Bloem BR, Ferreira JJ. Cognitive impairment in fall-related studies in Parkinson’s disease. J Park Dis. 2015;5(3):453–69.

    Google Scholar 

  37. Flink M, Brandberg C, Ekstedt M. Why patients decline participation in an intervention to reduce re-hospitalization through patient activation: whom are we missing? Trials. 2019;20(1):82.

  38. Harrison JM, Jung M, Lennie TA, Moser DK, Smith DG, Dunbar SB, Ronis DL, Koelling TM, Giordani B, Riley PL, Pressler SJ. Refusal to participate in heart failure studies: do age and gender matter? J Clin Nurs. 2016;25(7–8):983–91.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Lloyd-Williams F, Mair F, Shiels C, Hanratty B, Goldstein P, Beaton S, Capewell S, Lye M, Mcdonald R, Roberts C, Connelly D. Why are patients in clinical trials of heart failure not like those we see in everyday practice? J Clin Epidemiol. 2003;56(12):1157–62.

    Article  PubMed  Google Scholar 

  40. Williams B, Irvine L, McGinnis AR, McMurdo ME, Crombie IK. When "no" might not quite mean "no"; the importance of informed and meaningful non-consent: results from a survey of individuals refusing participation in a health-related research project. BMC Health Serv Res. 2007;7(1):59.

  41. van Heuvelen MJG, Hochstenbach JBM, Brouwer WH, de Greef MHG, Zijlstra GAR, van Jaarsveld E, Kempen GIJM, van Sonderen E, Ormel J, Mulder T. Differences between participants and non-participants in an RCT on physical activity and psychological interventions for older persons. Aging Clin Exp Res. 2005;17(3):236–45.

    Article  PubMed  Google Scholar 

  42. Gaertner B, Seitz I, Fuchs J, Busch MA, Holzhausen M, Martus P, Scheidt-Nave C. Baseline participation in a health examination survey of the population 65 years and older: who is missed and why? BMC Geriatr. 2016;16(1):21.

  43. Provencher V, Mortenson WB, Tanguay-Garneau L, Bélanger K, Dagenais M. Challenges and strategies pertaining to recruitment and retention of frail elderly in research studies: a systematic review. Arch Gerontol Geriatr. 2014;59(1):18–24.

    Article  PubMed  Google Scholar 

  44. Elskamp AB, Hartholt KA, Patka P, Beeck EF, van der Cammen TJ. Why older people refuse to participate in falls prevention trials: a qualitative study. Exp Gerontol. 2012;47:342–5.

    Article  PubMed  Google Scholar 

  45. Chatfield MD, Brayne CE, Matthews FE. A systematic literature review of attrition between waves in longitudinal studies in the elderly shows a consistent pattern of dropout between differing studies. J Clin Epidemiol. 2005;58(1):13–9.

    Article  PubMed  Google Scholar 

  46. Yang R, Carter BL, Gums TH, Gryzlak BM, Xu Y, Levy BT. Selection bias and subject refusal in a cluster-randomized controlled trial. BMC Med Res Methodol. 2017;17(1):94.

  47. Akmatov MK, Jentsch L, Riese P, May M, Ahmed MW, Werner D, Rösel A, Prokein J, Bernemann I, Klopp N, et al. Motivations for (non) participation in population-based health studies among the elderly – comparison of participants and nonparticipants of a prospective study on influenza vaccination. BMC Med Res Methodol. 2017;17(1):18.

    Article  PubMed  PubMed Central  Google Scholar 

  48. Elzen H, Slaets JP, Snijders TA, Steverink N. Do older patients who refuse to participate in a self-management intervention in the Netherlands differ from older patients who agree to participate? Aging Clin Exp Res. 2008;20(3):266–71.

    Article  PubMed  Google Scholar 

  49. Palmas W, Teresi J, Morin P, Wolff LT, Field L, Eimicke JP, Capps L, Prigollini A, Orbe I, Weinstock RS. Recruitment and enrollment of rural and urban medically underserved elderly into a randomized trial of telemedicine case management for diabetes care. Telemed J E Health. 2006;12(5):601–7.

    Article  PubMed  Google Scholar 

  50. Coley N, Rosenberg A, van Middelaar T, Soulier A, Barbera M, Guillemont J, Steensma J, Igier V, Eskelinen M, Soininen H, et al. Older Adults' reasons for participating in an eHealth prevention trial: a cross-country, mixed-methods comparison. J Am Med Dir Assoc. 2018;20:843–849.e5.

    Article  PubMed  Google Scholar 

  51. Bongartz H, Rübsamen N, Raupach-Rosin H, Akmatov MK, Mikolajczyk RT. Why do people participate in health-related studies? Int J Public Health. 2017;62(9):1059–62.

    Article  PubMed  Google Scholar 

  52. Dodge HH, Katsumata Y, Zhu J, Mattek N, Bowman M, Gregor M, Wild K, Kaye JA. Characteristics associated with willingness to participate in a randomized controlled behavioral clinical trial using home-based personal computers and a webcam. Trials. 2014;15(1):508.

    Article  PubMed  PubMed Central  Google Scholar 

  53. Tarp K, Mejldal A, Nielsen AS. Videoconferencing-based treatment of alcohol use disorders: analyses of nonparticipation. JMIR Formativ Res. 2017;1(1):e3.

    Article  Google Scholar 

  54. Thompson D, Canada A, Bhatt R, Davis J, Plesko L, Baranowski T, Cullen K, Zakeri I. eHealth recruitment challenges. Eval Program Plann. 2006;29(4):433–40.

    Article  PubMed  Google Scholar 

  55. Kreuter F, Olson K, Wagner J, Yan T, Ezzati-Rice TM, Casas-Cordero C, Lemay M, Peytchev A, Groves RM, Raghunathan TE. Using proxy measures and other correlates of survey outcomes to adjust for non-response: examples from multiple surveys. J Royal Stat Soc. 2010;173(2):389–407.

    Article  Google Scholar 

  56. Wittwer R, Hubrich S. Nonresponse in household surveys: a survey of nonrespondents from the repeated cross-sectional study “mobility in cities – SrV” in Germany. Trans Res Procedia. 2015;11:66–84.

    Article  Google Scholar 

  57. Wagner M, Kuppler M, Rietz C, Kaspar R. Non-response in surveys of very old people. Eur J Ageing. 2018;16(2):249-258.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Senatsverwaltung für Gesundheit, Umwelt und Verbraucherschutz. Sozialstrukturatlas Berlin 2008, Gesundheitsberichterstattung Berlin Spezialbericht (Health Reporting Berlin Special Report: Social Structur Atlas Berlin 2008; German). Berlin: Senatsverwaltung für Gesundheit, Umwelt und Verbraucherschutz. 2009. https://www.berlin.de/sen/gesundheit/_assets/service/publikationen/gesundheitsberichterstattung/gbe_spezial_2009_1_ssa2008.pdf. Accessed 31 Oct 2019.

  59. Borsboom D, Mellenbergh GJ, van Heerden J. The concept of validity. Psychol Rev. 2004;111(4):1061.

    Article  PubMed  Google Scholar 

  60. Börsch-Supan A. Survey of Health, Ageing and Retirement in Europe (SHARE) Wave 5. SHARE-ERIC. Dataset 2016.

  61. Cox B, Van Oyen H, Cambois E, Jagger C, Le Roy S, Robine J-M, Romieu, I. The reliability of the minimum European health module. Int J Public Health. 2009;54(2):55–60.

    Article  PubMed  Google Scholar 

  62. World Health Organization. WHOQOL-BREF: introduction, administration, scoring and generic version of the assessment: field trial version, December 1996. Geneva: World Health Organization; 1996.

    Google Scholar 

  63. StataCorp. Stata Statistical Software: Release 14. College Station, Texas: StataCorp LP; 2015.

    Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

The NPART project is part of the Supporting Self-Care by Information and Communication Technology (ICT) for Older People with Long-Term Conditions (ICT4Self-care) research programme (2015–2018) funded by the Swedish National Science Council and the Swedish Research Council for Health, Working Life and Welfare (VR-FORTE) (ref. 2014–4100).

Author information

Authors and Affiliations

  1. Division Ageing and Social Change (ASC), Linköping University, Kungsgatan 40, 601 74, Norrköping, Sweden

    Arianna Poli, Susanne Kelfve & Andreas Motel-Klingebiel

  2. Aging Research Center (ARC), Karolinska Institutet & Stockholm University, Gävlegatan 16, 113 30, Stockholm, Sweden

    Susanne Kelfve

Authors
  1. Arianna Poli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Susanne Kelfve
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Andreas Motel-Klingebiel
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

AP contributed to the study conception, design, data collection planning, data analysis and coordination, and drafted the manuscript. SK contributed to the study design, data collection planning and data analysis. AMK conceived of the study, and supervised its design, conduct and coordination. All authors were involved in revising the manuscript critically for important intellectual content and have given their final approval of the version to be published.

Corresponding author

Correspondence to Arianna Poli.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for the NPART project was granted by the Ethical Review Board (EPN) in Linköping, Sweden (registration number 2016/47431). All study procedures are in accordance with the national regulations specified by the responsible committee and with the relevant EU standards (see e.g. EU Directive 95/46/EC). All persons involved will be provided with the presentation letter explaining the processes of the data collection and analysis, together with the questionnaire. In the presentation letter, it will be made clear that answering and returning the questionnaire corresponds to giving informed consent. Participants will be recruited on a voluntary basis according to international declarations (the Declaration of Helsinki, as revised in 2013). Further information to the participants will be provided in an appropriate manner by qualified personnel to ensure that they understand the issues involved in the research project.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1.

NPART survey questionnaire.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Poli, A., Kelfve, S. & Motel-Klingebiel, A. A research tool for measuring non-participation of older people in research on digital health. BMC Public Health 19, 1487 (2019). https://doi.org/10.1186/s12889-019-7830-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-019-7830-x

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us