Reliability and validity of a short form household food security scale in a Caribbean community

Background We evaluated the reliability and validity of the short form household food security scale in a different setting from the one in which it was developed. Methods The scale was interview administered to 531 subjects from 286 households in north central Trinidad in Trinidad and Tobago, West Indies. We evaluated the six items by fitting item response theory models to estimate item thresholds, estimating agreement among respondents in the same households and estimating the slope index of income-related inequality (SII) after adjusting for age, sex and ethnicity. Results Item-score correlations ranged from 0.52 to 0.79 and Cronbach's alpha was 0.87. Item responses gave within-household correlation coefficients ranging from 0.70 to 0.78. Estimated item thresholds (standard errors) from the Rasch model ranged from -2.027 (0.063) for the 'balanced meal' item to 2.251 (0.116) for the 'hungry' item. The 'balanced meal' item had the lowest threshold in each ethnic group even though there was evidence of differential functioning for this item by ethnicity. Relative thresholds of other items were generally consistent with US data. Estimation of the SII, comparing those at the bottom with those at the top of the income scale, gave relative odds for an affirmative response of 3.77 (95% confidence interval 1.40 to 10.2) for the lowest severity item, and 20.8 (2.67 to 162.5) for highest severity item. Food insecurity was associated with reduced consumption of green vegetables after additionally adjusting for income and education (0.52, 0.28 to 0.96). Conclusions The household food security scale gives reliable and valid responses in this setting. Differing relative item thresholds compared with US data do not require alteration to the cut-points for classification of 'food insecurity without hunger' or 'food insecurity with hunger'. The data provide further evidence that re-evaluation of the 'balanced meal' item is required.


Background
Food insecurity has been defined as the 'limited or uncer-tain availability of nutritionally adequate and safe foods, or limited or uncertain ability to acquire food in socially acceptable ways' [1]. Food insecurity may compromise both dietary intakes and health outcomes. The measurement of food insecurity in epidemiological studies through the use of self-reported responses to questionnaire measures has therefore attracted growing interest. In the United States, an 18 item food security questionnaire was developed [1] and applied in the 1995 Current Population survey and in subsequent surveys carried out for the US Department of Agriculture [2]. This US food security questionnaire includes items relevant to households with children as well as adults. It has properties analogous to Guttman scaling [3] such that the items can be ranked according to decreasing frequency of affirmative responses and increasing severity of food insecurity. This property permits distinctions to be made between food security, food insecurity without hunger, food insecurity with moderate hunger, and food insecurity with severe hunger [1]. In 1999 Blumberg et al [4] proposed an abbreviated short form of the household food security scale which included six items from the 18 item questionnaire. These were selected so as to be applicable to households with or without children, and to distinguish between food security, food insecurity without hunger and food insecurity with hunger only. This six item module has also been described, with a slightly different question order, by Bickel et al [5].
Application of the food insecurity scale in the US has revealed that food insecurity may be unexpectedly frequent even in a high-income country [1]. The concept and measurement of food insecurity may be even more relevant in the setting of middle-income countries where a significant proportion of households have incomes which are low enough to potentially compromise access to food. In Trinidad and Tobago, the UN Food and Agriculture Organisation estimated that 13% of the population were under-nourished in 1997-1999 [6]. We recently explored experiences of food insecurity in a population survey in Trinidad and Tobago using the short form of the household food security scale [7]. We found that 25% of subjects were classified as food insecure according to this instrument. Food insecurity was associated with low incomes, physical limitations and lower consumption of fruit and vegetables. These findings were consistent with reports from the US [8,9]. However, in contrast to some other studies [10][11][12], underweight was associated with food insecurity but overweight or obesity were not [7].
While the consistency of these associations with those of other reports pointed to the validity of the scale, our results showed an unexpectedly higher proportion of affirmative responses to the item concerning 'balanced meals' [13], and an unexpected difference in the prevalence of food insecurity across ethnic groups. The latter finding might have been explained by sampling imbal-ances, as a cluster sampling technique was used [14]. Based on these two observations, Frongillo [13] questioned the validity of the short form food security for application in Trinidad and Tobago. The present report therefore aimed to evaluate item responses obtained using the short form household food security scale in a context which is different from the one in which the measure was developed.
Trinidad and Tobago is an English-speaking country but a distinct vernacular is widely used, especially by lower income groups. Food habits have been influenced by the cultures of European, African, Indian and Chinese immigrants to the country. More recently, restrictions on trade have been dismantled, American economic influences have become increasingly important and there have been associated changes in dietary habits. Based on a comparison of statistics, the gross national income per capita in Trinidad and Tobago was about 14% that of the US being US$4,930 in 2000 [15].
In order to evaluate the reliability and validity of responses to the short form food security scale, we first explored the consistency of the scale items [3] and estimated agreement between responses obtained from individual respondents from the same households. We then used item response theory models to estimate item thresholds, and to explore whether there was evidence of differential item functioning by ethnicity. In order to evaluate criterion-related validity [16], we estimated the association of each item with household income and frequent consumption of green vegetables and salads.

Subjects and questionnaire
The sampling design for the study and data collection methods have been described previously [7]. We selected a constituency in North Central Trinidad which was considered to be representative of the socio-economic and ethnic characteristics of the country. Within the constituency, 15 clusters, each of about 20 households were selected. Adults aged ≥ 25 years were enumerated and invited to take part in the survey. A questionnaire was administered by interview, including the six questions described by Blumberg et al [4] (see Table 1). Ethnicity was self-reported according to the categories of the Trinidad and Tobago national census but were reduced for analysis to the categories 'Afro-Trinidadian', 'Indo-Trinidadian' and 'mixed' and 'other and not known'. Monthly household income was analysed using five categories reported previously [7]. A limited range of food items were included in a short food frequency questionnaire. We evaluated whether more frequently consumed foods, including green vegetables and salads, were eaten at least 5-6 days per week or not [7]. Descriptive data for the sample has been included in previous reports [7]. The study received research ethics committee approval and subjects gave informed consent to participation.

Analysis
Each item in the household food security scale was reduced to the categories of affirmative or not as described by Blumberg et al [4] (Table 1). We estimated Cronbach's alpha for the six items. We estimated the intraclass correlation coefficient for each item using analysis of variance in order to assess the correlation of item responses within households. Item response models were fitted to the data using BILOG-MG version 3.0 software from Scientific Software International [17]. Item-score correlations were estimated as the Pearson correlation between each item and the total and were taken from the BILOG output [17]. Initially a one parameter logistic (Rasch) model was fitted. Item thresholds, their standard errors and item fit Pvalues are reported. Item thresholds are estimated parameters which indicate the location of the item in relation to the latent construct. In this context, the threshold provides a measure of the relative severity of food insecurity associated with affirmative responses to an item. Under the conventions of the Rasch model, the item thresholds are scaled so that their arithmetic mean is zero. The thresholds may also be rescaled to a different mean location as discussed by Bickel et al [5].
After fitting the single group model, we explored whether there was evidence of differential item functioning in different ethnic groups. Subjects with 'not known' ethnic group were omitted from these analyses. The combined data from the three ethnic groups were fitted to a oneparameter logistic model, but with separate item thresholds estimated for each ethnic group. Only the means of the item thresholds were constrained to be equal across groups. The extent to which items functioned differentially in the three ethnic groups was then assessed by the likelihood ratio (difference in twice log likelihood) between the single-group model with item thresholds constrained to be equal for all respondents and this unconstrained model.
We used logistic regression to estimate associations between item responses, as dependent variables, and income. The slope index of inequality for income was estimated as recommended by Mackenbach and Kunst [18] and Wagstaff et al [19]. This permitted the use of a single estimate to summarise the association between income and an item, rather than requiring the presentation of odds ratios for each category of income. Logistic regres- sion was also used to estimate the association between consumption of green vegetables and salads, as dependent variable, and item responses. This variable was selected because food insecurity was associated with reduced consumption of green vegetables and salads. Regression analyses were adjusted for continuous variable age, sex and ethnic group. We also tested whether associations differed according to ethnic group. Robust standard errors were estimated to allow for clustering by household [20].

Results
There were 631 eligible adults and food security data were obtained for 531 (84%) subjects from 286 households. There were 134 (25%) of subjects who were classified as food insecure, based on a definition of two affirmative items out of six. Table 1 lists the six items of the household food security scale in rank order of frequency of affirmative responses, with the exception that item number eight is listed after item number five with which it is associated. Items are numbered following Blumberg et al [4]. Item-score correlations ranged between 0.52 to 0.79. The value of Cronbach's alpha, an index of internal consistency, was 0.87. In the Afro-Trinidadian ethnic group, Cronbach's alpha was 0.84, in Indo-Trinidadians 0.90 and in the mixed group 0.83. Table 1 also gives estimated threshold values from the Rasch model for each of the items. It was clear that the 'balanced meal' item gave the lowest severity threshold in this dataset. The severity rankings of the other items were consistent with US data. Table 2 shows the frequency of discordant responses for individuals from the same household, by household size, and the intraclass correlation coefficient. The latter is equivalent to the kappa statistic where households are considered as observations and individual subjects as raters. The overall proportion of discordant responses was low and the ICC was high for each item and for the food insecurity classification.   Table 3 shows the proportion of affirmative responses to each item by ethnic group after omitting 24 cases where ethnic group was 'not known'. The Afro-Trinidadian ethnic group gave higher proportions of affirmative responses to all items, while the Indo-Trinidadian group gave the lowest. Item-score correlations were generally consistent between ethnic groups. Table 4 gives the estimated item thresholds from the differential item functioning model. These were adjusted to allow for differences in the overall proportion with food insecurity between groups. The differential item functioning model gave a better fit to the data than the Rasch model (χ 2 92.73, df 10, P < 0.001). There was evidence that the 'balanced meal' item gave a relatively higher threshold in the Afro-Trinidadian group than in the other groups. Nevertheless, this item was still ranked lowest in terms of relative severity in each ethnic group. There was weaker evidence that the 'eat less' and 'how often' items were ranked differently in the three ethnic groups. The threshold for the 'eat less' item was significantly higher for those of mixed ethnicity compared to reference, but the difference was not significant for the Indo-Trinidadian group. Table 5 shows the associations of each item with household income. As the proportion of affirmative responses to the items decreased, and the proposed severity of food insecurity increased, so the proportion of subjects falling into the lowest income category increased and the proportion in the highest income category decreased. The slope index of income-related inequality also increased as the proposed severity of each item increased although estimates were imprecise for the low prevalence items. The slope index of inequality can be interpreted as the relative odds of an affirmative response for those at the bottom of the income scale compared with those at the top. Table 6 shows the associations of each item with frequent consumption of green vegetables and salads. Those who were food insecure consumed green vegetables and salads less frequently than those who were food secure. This pattern was consistently observed for each item, with some suggestion of a graded effect with increasing severity of the  items. The estimate for the lowest prevalence item was very imprecise. Food insecurity was associated with reduced consumption of green vegetables and salads even after additional adjustment for income (as a categorical variable) and education (as categorised previously [7]) giving an adjusted odds ratio of 0.52, 95% confidence interval 0.28 to 0.96, P = 0.038. There was no evidence for an interaction with ethnicity in either set of analyses. Table 7 shows descriptive data for the other nine food items reported previously [7] for each of the food security items separately. The previously reported associations were consistent for across food security items for consumption of fruit and green vegetables and salads. The remaining food items were not associated with food security status [7] and this finding was consistent for each food security item separately.

Discussion
The present study evaluated the food insecurity item responses and provides evidence for their reliability and validity in this setting [16]. The six items of the short form household food security scale gave consistent responses. Cronbach's alpha was similar to the values of 0.84 and 0.86 reported by Kendall et al [16], for a slightly different group of items, and 0.86 reported by Hamilton et al [21] for the US food security measure. Streiner and Norman [3] suggest that values for alpha should exceed 0.70, while values in excess of 0.90 might suggest that some items are redundant. The reliability of the measure was also supported by the high level of agreement for item responses of individuals in the same households. The item response theory models provided evidence for the validity of the instrument in this context. Estimated item thresholds were generally consistent with the relative severity of food insecurity represented by each item in US data. The main departure from this pattern related to the 'balanced meal' item which was consistently the item with lowest severity in these data. The associations between item responses and household income and consumption of green vegetables and salads were consistent, and, at least for income, graded according to the severity of food insecurity represented by the item. These results provide evidence of criterion-related validity [16] similar to the results   (19) reported for the food security measure in the US [21,22]. Furthermore the classification of food insecurity provided information additional to that from socio-economic status in terms of income and education.
In the US data, the item concerning 'food not lasting' gave the highest proportion of affirmatives [1]. In our data, the 'balanced meal' item represented the item with lowest severity but the ranking obtained for the 'balanced meal' item was also reflected in the estimated associations with income and consumption of vegetables and salads. The findings from the Trinidad and Tobago data are therefore consistent even though they differ in this respect from the US data. However, there was evidence that the 'balanced meal' item performed differently in different ethnic groups, with a higher threshold being observed in the Afro-Trinidadian group which nevertheless exhibited the highest proportion with food insecurity in this sample. This differential functioning was not sufficiently large to influence the ranking of the item in the different groups.
The 'eat less' and 'how often' items generally had similar thresholds while the 'hungry' item gave a much higher threshold. Therefore it is justifiable to conclude that the different relative item thresholds compared with US data, does not require alteration to the cut-points of two affirmatives for the classification of 'food insecurity without hunger' or five affirmatives for 'food insecurity with hunger'.
Qualitative data from Hawaii showed that the 'balanced meal' item may be subject to variable interpretation [23] and this was reflected in the performance of the food security measure [24]. Harrison et al [25] also questioned the conceptual validity of this item in the translation of the food security questionnaire into Spanish. We share the reservations expressed by Derrickson et al [23] concerning this item. Derrickson et al [23] suggested that additional explanation of this item should be provided. In a more recent survey we have introduced the following text: 'a balanced meal may contain starchy food, like rice, potatoes, bread, ground provisions or macaroni ; and a protein-rich food like meat, fish, milk, or peas or beans; and a fruit or a vegetable'. This was adapted from Derrickson et al based on the advice of experienced local nutritionists. However, the requirement for additional explanation might support the idea that the original question is unsatisfactory. Qualitative research might be carried out to identify a more culturally appropriate wording for this item but changes to an established measure should generally be made cautiously because changes made locally may compromise the generalisability of findings.
Our study had the strengths of a population-based study based on a multi-stage sampling design. The questionnaire was interview administered and there were few miss-ing values. The sample size was relatively modest but adequate for this purpose. A concern relates to the relative representativeness of the samples obtained for each ethnic group. The sample was drawn from 15 clusters of households and income, ethnicity and food insecurity were each correlated within clusters. Imbalance in the distribution of income, or food security, in relation to ethnicity could have occurred by chance through the sampling of a small number of clusters. Post hoc sub-group analyses by ethnicity therefore require replication and we caution against attaching undue weight to these findings, especially since there were few affirmative responses for some items when analysed by ethnicity. In the analyses, we were not always able to allow for the effects of clustering by household or household cluster because clustered versions of relevant tests were not available. In general the consequence would be to make standard errors and P-values too small, the results should therefore be viewed with some caution.
A considerable research effort, spanning almost a decade, was devoted to the development of the US food security measure [1,21,22]. The items were derived from the results of qualitative interviews with food insecure subjects, and the instrument is considered to be well grounded. Wolfe and Frongillo observed that the comparability of the instrument is likely to hold in 'many other countries' [26]. The present results suggest that the instrument may certainly prove of value in assessing food insecurity within countries like Trinidad and Tobago which may have a higher frequency of food insecurity than the US. Our results generally support the reliability and validity of the food security scale within this population but, in agreement with other reports, we find that the 'balanced meal' item gave less consistent responses. Like any brief instrument, the short form household food security scale is unlikely to encompass all facets of the experience of food insecurity. There is a role for future research to evaluate whether relevant aspects of food insecurity are neglected by the instrument, and whether the sensitivity and specificity of the resulting classification can be enhanced. Nevertheless, the present results support the use of this measure in epidemiological studies to evaluate the distribution, determinants and associations of food insecurity within this population.
As with other self-reported measures, there are likely to be difficulties in making comparisons across different populations because individuals' expectations and their interpretations of item response categories may vary systematically between populations [27]. This is evident in the results obtained for the 'balanced meal' item. The difference in prevalence of food insecurity between Trinidad and Tobago and the US is large and consistent. Nevertheless, comparisons between populations of the prevalence of food insecurity obtained using self-reported measures must be interpreted with caution. Associations obtained within populations are likely to be less sensitive to such problems, but even here they cannot always be ignored [28]. Our data suggested that the pattern of responses was generally similar in each ethnic group but there was evidence that two of the items might have slightly different relative severities in different groups. Murray et al [29] observe that 'a critical requirement is an improved understanding of the determinants of differences between self-reported and observed measures of performance or capacity in selected domains in health' (page 991). This observation also applies in the assessment of food insecurity.