Aim and design
The aim of this paper was to report on the development, reliability evaluation and usability evaluation of the HESPER Web.
The development and reliability evaluation of the HESPER Web was conducted across three phases; I) development of the HESPER Web, II) evaluation of alternate forms reliability, and III) test-retest reliability. In all three phases, the feasibility of the HESPER Web was also evaluated.
Instrumentation
The original HESPER assesses needs as perceived by the people affected by the humanitarian emergency or disaster themselves, and consists of 26 questions covering physical, psychological and social needs. There is also possible to add context specific questions. Ratings are made by interviewers in a face-to-face interview with affected persons by defining whether or not each of the 26 questions are perceived by respondents to be a ‘serious problem’ (unmet need) or ‘no serious problem’ (no need). An additional open-ended question is also asked which gives the respondent the opportunity to identify any other serious needs not covered by the 26 core questions. At the end of the interview, respondents are then asked to rank their three most serious needs. A total sum score can be calculated by adding up the total number of ‘serious problem’ ratings. The HESPER includes as a minimum demographic questions on age, gender and current location. For this study, questions were added regarding country of origin, by providing the ten most common countries of origin for asylum seekers in Sweden together with a “stateless/ don’t want to say” alternative and a free text alternative. Also, questions on the usability and experience of using the HESPER Web were added, including technical problems, time to complete the survey, possibilities to answer the survey in privacy, potential experienced harm or other comments.
Phase I; development of HESPER web
In the first phase, a web based version of the original HESPER was developed. The original HESPER survey phrasing in English was transformed into a web based survey tool called oru-survey, which is a protected survey tool and database for research purposes. The web based survey was created and posted as an Internet link using the www.abcde.xxx format. The survey could be accessed by both mobile phone, tablet and computer. In the web version, the exact phrasing from the HESPER English language version was used, using the whole question texts including explanations. The original HESPER ratings; “1: yes, a serious problem”, “0: No, no serious problem” or “9: Don’t know/ don’t want to say/ not applicable”, were used for each of the 26 questions, in accordance with the original scale. If the additional “other needs” question was ticked by study participants, there was a free text space to fill in, in order to explain the need further. For the ranking question, the survey was programmed to show only those items marked as 1 (“yes, serious problem”) and the study participant could mark one item as “most serious problem”, one as “second most serious problem” and another as “third most serious problem”.
To evaluate the phrasing as well as the web survey design, a panel of four senior researchers with expertise in psychology, public health and health statistics as well as ten international students with varied backgrounds (country of origin: Brazil, Egypt, India, Italy, Iran, Iraq, Japan, Mozambique, and Turkey) completed the survey and provided oral or written feedback to the research team on language, usability and design. The feedback was used throughout the developmental phase to refine the web survey.
Phase II; alternate forms reliability evaluation
In the second phase, alternate forms reliability between the HESPER and HESPER Web was evaluated using a non-probability, voluntary study sample of newly arrived asylum seekers in Sweden. In total, approximately 250 adult asylum seekers were registered in the study region at the time of the data collection. Inclusion criteria were an age of 18 years old or more, ability to understand study information and survey questions in English, being an asylum seeker in Sweden and having access to a mobile phone (smart phone), tablet or computer. A power analysis indicated the need for 19 study persons in order to detect a statistically significant correlation and a power of 90%, based on the assumptions that the lowest acceptable Intra Class Correlation (ICC) was 0.7 and the target 0.9 [11]. Fifty-two study participants were recruited through information meetings in dedicated living areas for newly arrived asylum seekers and in social activity areas such as the Red Cross café in one region in Sweden. Data collection was conducted during the period of March 01 until May 31, 2018. The HESPER interviews were conducted in accordance with the HESPER manual [2] by the principal researcher (KH) and one assistant, in the study participants’ home or a private area in the social activity house. Half of the study sample were randomly selected to attend the face-to-face interview for the original HESPER first and then the HESPER Web next after approximately 1 week. The other participants answered the surveys in the opposite order. A reminder for the second data collection was sent to the study participant by text message (SMS) on day six, seven and eight after the first data collection. All data collection was done in English.
Phase III; test-retest reliability evaluation
In the third phase, test-retest reliability for the HESPER Web was evaluated using another sample of asylum seekers in Sweden, but with the same inclusion criteria as above. The same power analysis was used for the alternate forms reliability evaluation and test-retest evaluation. Forty-four study participants were recruited through information meetings in social activity areas such as the Red Cross café in one region in Sweden. No study person was involved in both the alternate reliability evaluation sample and the test-retest evaluation sample. Data collection was conducted during the period of April 01 until June 24, 2018. Study participants completed the HESPER Web survey the first time in presence of a research assistant. A reminder to answer the web survey a second time was sent out by text message on day seven, eight and nine after the first data collection. All data collection was done in English.
Data analyses phase II and III
Data from the web survey was automatically recorded into an Excel format, and thereafter imported into the statistical program SPSS (IBM Corp. Released 2016. IBM SPSS Statistics for Windows, Version 24.0. Armonk, NY: IBM Corp). All data were first checked for duplications, and dropouts were re-coded as missing data. For demographics and to analyse the evaluation questions, descriptive statistics was used. For the alternate forms reliability between the HESPER and HESPER Web, as well as for the test-retest validation of the HESPER Web, ICC, two-way mixed, absolute agreement [12] of total number of reported serious needs was calculated. To illustrate the agreement of mean number of needs between the HESPER and HESPER Web, test-retest Bland-Altman plots were constructed [13]. In the Bland-Altman plot, the difference between the two forms/tests was plotted against the mean score for each subject. The 95% limits of agreement were calculated as mean difference ± 1.95 SD. To assess agreement on an item level and precentral match between first priority need between the HESPER and HESPER Web as well as the results from the test-retest evaluation, Cohens κ was used. For calculation of association between the first prioritized need between HESPER and HESPER Web, and in the test-retest evaluation, Cramer’s V was calculated using a cross table methodology. Calculations were done by two of the researchers (KH and MH) and an external statistician.
Ethical considerations
Permission to develop the HESPER Web was obtained from WHO. Ethical approval from the Regional Ethical Committee in Sweden (2017/481) was obtained for this study. In the study information, it was clearly described where study participants could turn to address any acute needs. Written informed consent was obtained from all study participants before interviews, and a digital informed consent was obtained before entering the survey questions in the web survey. All data collections were conducted in privacy, and information obtained by the data collectors was kept confidential. All data were stored at a protected research database server owned by Örebro University.