Ethics approval
Ethics approval was obtained from the University of Melbourne Human Research Ethics Committee (Australia), the Royal Victorian Eye and Ear Hospital Human Research and Ethics Committee (Australia), the International Centre for Diarrhoeal Disease Research, Bangladesh (icddr, b) Ethical Review Committee (Bangladesh), and the Fijian National Research Ethics Review Committee (Fiji). The study was conducted in accordance with the tenets of the Declaration of Helsinki. All participants provided a written or verbal informed consent. In case of participants who were not literate or were unable to provide written consent their verbal consent was obtained, i.e. the consent form was read to participants and their verbal agreement was recorded by the interviewer in front of a witness.
The RAD questionnaire
The RAD questionnaire was developed based on a literature review, expert panel workshops and in-depth interviews with people with disability [10, 12]. The RAD questionnaire is interviewer administered and is comprised of two parts: a household questionnaire administered to the head of the household assessing socio-economic indicators such as source of water, electricity, sanitation, housing materials and assets including durable goods (e.g. television, radio, bicycle), and ownership of house, land and cattle and an individual questionnaire administered to each of the eligible members of the household.
The initial version of the individual questionnaire comprised five sections 1) demographics, 2) self-assessment of functioning, 3) awareness of the rights of people with disability, 4) well-being, and 5) access to the community [12]. Section 1 included 25 items related to demographics such as age, gender, ethnicity, religion, marital status, education, occupation, health conditions and assistive devices used. Section 2 comprised 18 items related to functioning in eight domains: vision (1 item), hearing (1 item), communication (1 item), mobility (2 items), self-care (1 item), gross and fine motor (1 item), cognitive (4 items), appearance (1 item) and psychological distress (6 items). The items related to psychological distress are from the Kessler-6 scale [12, 13]. Each item asked the participants to report whether they experienced difficulty in functioning in the last six months even when using assistive devices available to them (e.g. difficulty seeing even if wearing glasses). Those who answered ‘yes’ to an item were then asked to rate the frequency of difficulty as ‘some of the time’, ‘most of the time’, or ‘all of the time’. Those who self-reported having difficulty ‘most’ or ‘all of the time’ to at least one item from the first seven domains or at least two items from the psychological distress domain were considered as people potentially experiencing disability. The cut off criteria was determined based on the recommendations from the WG for short set questions [14] and also based on the consensus among the research group.
The purpose of section 3 was to obtain information about the awareness of the rights of people with disability. Section 3 comprised 17 items related to awareness of the rights of people with disability such as the right to ‘access information’, ‘live in a safe home environment’, ‘go to school or study’, ‘access health care’, ‘marry’ and ‘have children’. Each question was phrased “To what extent do you have rights to ….?”, and the response scale was rated on ‘none’, ‘some’ and ‘all’ categories.
Section 4 comprised 18 items to assess individuals’ perception of their well-being (Figure 1). The items included ‘good health’, ‘making friends’, ‘being safe in daily life’, ‘opinion counted in family’ and ‘taking care of one’s self’, ‘taking care of household’, ‘making new friends’, ‘maintaining family relationships’, and ‘respected in the community’ with the frequency of experiencing each situation reported on a four-category response scale ranging from ‘never’ to ‘all of the time’.
Section 5 asked respondents whether they had access to 14 community related domains such as access to health services, education and vocational training, work opportunities, social, legal, religious and rehabilitation. Each domain had three questions and the first question was phrased “In the last 6 months, did you have access to…… as much as you needed?” If they responded that they had not accessed services in this domain as much as they needed, they were asked to report reasons/barriers for this. Respondents were then asked which of the reasons given had limited access to the domain the most.
Psychometric evaluation of the RAD questionnaire
The psychometric evaluation of the RAD questionnaire was conducted in two phases. The first phase involved field-testing in Bangladesh to assess the psychometric properties of the questionnaire. The findings of the Bangladesh field-testing study were used to identify poorly performing items, with the goal of removing or revising such items to enhance the psychometric robustness of the RAD. The second phase was conducted in Fiji to assess measurement properties of the revised questionnaire, to assess the sensitivity and specificity of the RAD to identify people with disability, and to assess the performance of the RAD in a cross-cultural setting. The findings from this phase of the study further informed modifications to the questionnaire and the development of the final version of the RAD.
Field-testing in Bangladesh
Field-testing in Bangladesh involved a cross-sectional population-based survey using two-stage cluster random sampling. The methodology was adapted from the Rapid Assessment of Avoidable Blindness (RAAB) survey, which was developed at the International Centre for Eye Health, London, as a rapid survey methodology to assess the prevalence and causes of vision impairment and cataract surgical coverage in a specific geographic area [15]. The first stage of sampling for the RAD field-testing involved randomly selecting clusters (villages or mahallas) from the sampling frame, with probability of selection proportional to cluster size. The sampling frame comprised all villages (in rural areas) and mahallas (in urban areas) in the Bogra district of Bangladesh, using population data from the 2001 national census projected to 2010. The second stage involved selecting households within clusters through compact segment sampling. Each village and mahalla was divided into equal segments through mapping of the sites so that each segment comprised approximately 50 people. Segments to be included in the study were selected by drawing lots. All households in the segment were included in the sample sequentially, with all eligible people in a household invited to participate in the study. If fewer than 50 eligible and consenting individuals were identified in a given segment, sampling continued in the next nearest segment until 50 people were recruited in a cluster. Eligibility criteria included people aged 18 years and above who had been living in the selected household for at least 3 months of the year. At least two return visits were made to recruit absentees.
In participating households, the household questionnaire was first administered to the head of each household. All eligible members of the household, including the head of the family, were then administered sections 1 and 2 of the individual questionnaire. Individuals identified as having disability in section 2 were then administered sections 3, 4 and 5. For each participant identified as having disability, an age and gender matched control was recruited from the neighbouring household that did not house a person with disability and all sections were administered. This was to allow the extent to which people with disability experience same opportunities when compared to their age and gender matched controls.
For field testing in Bangladesh, the questionnaire was translated into Bangla and then back translated into English.
Field-testing in Fiji
Considering that there are no gold standard instruments against which to compare the RAD questionnaire, a convenience sample of known people with disability were recruited in Fiji to test the sensitivity and specificity of the proposed cut off criteria to identify people with disability. The Fiji National Disability Policy [16] defines people with disability “are persons with long term physical, mental, learning, intellectual and sensory impairments and whose participation in everyday life as well as enjoyment of human rights are limited, due to socio-economic, environmental and attitudinal barriers”. Members of registered Disabled People’s Organisations self-identifying as people living with significant impairment were recruited as cases. Participants recruited as cases were people with a range of impairments who were receiving rehabilitation services from community-based rehabilitation programs. Controls were recruited through formal and informal networks of participating organisations. The two groups were individually matched on age and gender.
Prior to the study, the modified (following Bangladesh field-testing) version of the RAD individual questionnaire was reviewed by an advisory panel in Fiji to check for its cultural relevance and appropriateness of the items and language. Apart from the questions on section 1 (e.g. ethnicity, religion) no other changes were made. All sections of the individual questionnaire were administered to all participants.
Training of interviewers
Field supervisors and interviewers were recruited based on their skills and previous experience. Some of the interviewers were people with disability. Training was provided for a week that included disability inclusion, study design, recruitment of participants, administration of the RAD questionnaire, ethics in research and collecting survey data, conducting interviews, data storage and referral mechanisms for participants. Supervised field practice sessions were also conducted as part of the training.
Statistical analysis
Household questionnaire data from the Bangladesh study were used to derive a household wealth index using principal component analysis (PCA). Individuals were then classified into one of the three socio-economic groups based on values of the wealth index: the bottom 40% as poor, the next 40% as middle and the top 20% as rich [17, 18]. Descriptive statistical analysis was performed on the socio-demographic variables in section 1. Paired samples t-tests (for continuous variables) and generalised estimating equations (for categorical variables) were used to compare socio-demographic characteristics of people with disability and their matched controls.
Locations of category thresholds [19] for items from section 2 were analysed to examine whether participants could discriminate between response categories. Following this evaluation, responses were converted to binary variables of having a disability (difficulty ‘most’ or ‘all of the time’) or not (no difficulty or ‘some of the time’). Inter-item correlations (point biserial correlation coefficients for dichotomous variables) between section 2 items were examined to identify redundant items. Correlations of 0.75 or higher were interpreted as indicating redundancy.
Sensitivity and specificity of the cut off criteria used in RAD (i.e. self-reported difficulty ‘most’ or ‘all of the time’ to at least one item from the physical/sensory/cognitive domains or at least two items from the psychological distress domain of section 2) was assessed using data from Fijian field study. Sensitivity was calculated as the proportion of participants with disability identified by the RAD questionnaire among cases. Specificity was defined as the proportion of participants not identified to have disability by the RAD questionnaire among controls. Given that the RAD is intended for use as a population-level screening tool, we considered sensitivity and specificity values 60%-69% to be acceptable, 70%-79% good, 80%-89% very good, and ≥90% excellent.
Rasch modelling, a form of item response theory (IRT) that transforms ordinal scores into interval-level estimates [19, 20] (in logit units), was used to test construct validity of the sections 3 and 4 through the assessment of threshold ordering, unidimensionality, and targeting. Only sections 3 and 4 were subjected to Rasch analysis as these are the only sections of the RAD intended to be used for calculating section totals by summing responses across section items to capture participants’ perceived access to rights and well-being, respectively.
The Andrich rating scale model was used to obtain estimates of the level of characteristic (i.e., awareness of rights and well-being) represented by each item (item measure) and perceived level of these characteristics for each participant (person measure). Locations of category thresholds [19] were also estimated and items within each section were assessed for the evidence of disordered thresholds and to examine whether participants could discriminate between response categories. The response categories at least 1.4 logits apart were considered to be clearly differentiated by participants [21].
Measurement precision of the two sections, i.e. the ability to distinguish different levels of underlying characteristic, was assessed by the person separation index (PSI) and person separation reliability (PSR) scores. A PSI of ≥2.0 and a PSR ≥0.8 are considered minimally acceptable levels of separation, indicating that the questionnaire can distinguish at least three different levels of the characteristic of interest [20].
The extent to which each section measured a single underlying concept (i.e., section unidimensionality) was evaluated using item fit statistics and PCA of residual correlations. Item fit was assessed with the information-weighted (infit) mean-square (MnSq) statistics and standardised fit residuals (Zstd), both of which measure the discrepancy between the observed responses and responses predicted from the Rasch model. MnSq values of 0.7 to 1.3 and Zstd values of −2.0 to 2.0 were considered acceptable [22]. Values below the lower limit indicate redundancy and values above the upper limit indicate unacceptable level of variability in the responses (high measurement error). When a scale functions as intended, with all items measuring a single dimension, this single dimension should explain most of the correlations in the item set. The existence of any additional dimensions is inferred through the assessment of residual variance (variance unexplained by the primary dimension) using PCA. Unidimensionality was inferred if 1) the primary dimension explained at least 50% of the variance in the responses to the section items and 2) the first residual component had an eigenvalue <2.0, indicating that variance explained by this residual component carries less information than 2 items, the smallest number of items needed to form a dimension [23].
The capacity of the items to adequately represent the full continuum of a characteristic of interest is called targeting and was assessed by visual inspection of the person-item map and the difference between means of person and item measures. For a perfectly targeting instrument, the difference in means would be 0; an absolute difference of >1.0 logits indicates significant mistargeting and occurs when the amount of characteristics in the sample is substantially higher or lower than the average level of the same characteristic targeted by the items [24].
Sections 3, 4 and 5 were assessed on known-groups validity by comparing the scores of people with disability and controls using paired-samples t-tests (sections 3 and 4) and generalised estimating equations (section 5). It was expected that people with disability would score significantly higher on each section than their matched controls.
Rasch analysis was performed using Winsteps version 3.74 (Winsteps, Chicago, IL). Remaining statistical analyses were performed using PASW Statistics 18 (PASW Statistics for Windows, SPSS Inc., Chicago, IL).