Perceived neighborhood problems: multilevel analysis to evaluate psychometric properties in a Southern adult Brazilian population

Background

Physical attributes of the places in which people live, as well as their perceptions of them, may be important health determinants. The perception of place in which people dwell may impact on individual health and may be a more telling indicator for individual health than objective neighborhood characteristics. This paper aims to evaluate psychometric and ecometric properties of a scale on the perceptions of neighborhood problems in adults from Florianopolis, Southern Brazil.

Methods

Individual, census tract level (per capita monthly familiar income) and neighborhood problems perception (physical and social disorders) variables were investigated. Multilevel models (items nested within persons, persons nested within neighborhoods) were run to assess ecometric properties of variables assessing neighborhood problems.

Results

The response rate was 85.3%, (1,720 adults). Participants were distributed in 63 census tracts. Two scales were identified using 16 items: Physical Problems and Social Disorder. The ecometric properties of the scales satisfactory: 0.24 to 0.28 for the intra-class correlation and 0.94 to 0.96 for reliability. Higher values on the scales of problems in the physical and social domains were associated with younger age, more length of time residing in the same neighborhood and lower census tract income level.

Conclusions

The findings support the usefulness of these scales to measure physical and social disorder problems in neighborhoods.

There has been renewed interest in epidemiology on how aspects related to the place in which people live may affect health outcomes. Place-based features are hypothesized to affect health over and above other individual characteristics through multiple pathways [1–7].

Studies on the association between neighborhood conditions and health have mostly characterized neighborhoods using measures derived from census databases [2, 3]. Despite practical advantages, the use of aggregate measures implies important limitations including decennial periodicity of data gathering, changes in boundaries of units over time, and the sensitivity of some measures to the dynamic process of migration and emigration in neighborhoods. Moreover they are only indirect proxies for specific neighborhood aspects that may be relevant to health outcomes. The use of indirect indicators can hinder causal inferences regarding neighborhood effects on health in observational studies [2, 8].

A variety of other approaches are available to directly measure neighborhood attributes. These include systematic social observation [9], the use of geographic information systems to create measures about resource availability and access [10], and administration of questionnaires to residents, in order to obtain information on their perception of neighborhood conditions [8]. Each approach provides different and complementary information [8].

Measures of resident’s perceptions of neighborhood attributes can be examined in two ways. On one hand they can be studied in relation to individual-level health outcomes in individual level analyses. A second approach involves building contextual variables, through the aggregation of the responses of all residents of a given neighborhood. The underlying assumption is that the process of aggregating individual perceptions results in a more valid measure of objective attributes [8]. Analytical approaches involving the use of three-level multilevel analyses have been used to create these measures [11, 12].

Physical attributes of places in which people live, and theirs perception about them, are important health determinants beyond socioeconomic influence [13]. The perception of place in which people dwell may impact on individual health and may be a more telling indicator for individual health than objective neighborhood characteristics. Studies that combine residents perceived neighborhoods aspects and census measures have been tending to find stronger relationships between environment and outcomes than those studies that rely only on aggregate data [14]. Studies have suggested, indeed, that objective and subjective measures of neighborhood may contribute independently to health and well-being [15].

Perceptions of neighborhood problems may be influenced by individual-level characteristics of respondents (such as age, gender and individual-level social and economic characteristics) as well as by the objective features of the neighborhoods in which individuals reside [16, 17] Neighborhood socioeconomic disadvantage broadly reflects aggregate features of households. Furthermore, the clustering of households-level deprivation in neighborhoods appears to generate local contexts that induce health-related processes [18]. Diez-Roux & Mair [2] added that low income participants may live closer to areas with adverse environments than high income individuals even living at the same neighborhood [2].

Neighborhoods with higher levels of problems can be less attractive to people with higher income, and reach lower prices in the real estate market. Effects of land use with industrialization have had highly uneven effects at local levels as households experiencing a diverse range of disadvantages are increasingly clustered in poor neighborhoods [19]. Notably, in Brazil a large process of rural exodus with overcrowding in metropolitan regions encouraged poverty concentration in some areas, with worst life conditions, considered more vulnerable in periphery areas devoid of amenities, services, and even social spaces, and designed to poorer people [20].

Several studies have examined factors related to neighborhood perceptions in high-income countries [12, 17] Factors found to be related to neighborhood perceptions have included neighborhood characteristics (such as levels of poverty) as well as individual-level characteristics such as education and occupational status, age and time spent in the neighborhood a [12, 17].

Studies in different populations have reported association among perceived neighborhood aspects and distinct health outcomes, as: smoke [21], self-rated health [17, 22], cardiovascular disease [8], emotional health, and others [22].

However, few studies have examined the predictors or measurement properties of neighborhood perceptions in medium and low income countries, which may be different from those high income countries due the contextual socioeconomic and cultural differences [23]. This paper aimed to assess the ecometric and psychometric properties of a scale assessing the perception of neighborhood problems by adults living in Florianopolis, SC, a city in Southern Brazil with half a million inhabitants.

Sampling procedures

Data were derived from the baseline examination of a population-based cohort study called EpiFloripa carried out in Florianopolis, Southern Brazil, from September 2009 to January 2010 (http://www.epifloripa.ufsc.br). The objective of the EpiFloripa study was investigating health and life conditions of the adult population of the city. The study was conducted for teachers, and post-graduate students from the Federal University of Santa Catarina, from different departments. Furthermore, researches of other institutions collaborate in the design and analysis of EpiFloripa Study. The second wave of Epifloripa study began in 2011. Florianopolis is the capital of the state of Santa Catarina, with a population of 421,240 inhabitants [24], and presents a Gini Index of 0.40, lower that the country average (0.54) [24]. However, it still has striking social inequalities and, around 14% of population lives in poor housing conditions [25].

We selected 60 of the 420 urban residential census tracts of the city. All 420 urban census tracts of the city were ranked according to the average monthly income of the head of the family [26] classified into income deciles. Six tracts were randomly selected in each income decile. All selected census tracts were visited by the fieldwork team, and all occupied houses were enumerated. The enumeration identified some changes in the sizes of the census tracts. To reduce the variability in the number of households in each census tract, some of them were split and others aggregated taking into considering their income decile and geographic localization. This process resulted in sixty-three census tracts with 16,755 eligible households. Within each census tract we systematically selected 18 occupied households. In each household all adults were invited to take part in the survey.

Eligibility and exclusion criteria

All adults aged 20 to 59 years who were residents in the selected houses were eligible to participate. Exclusion criteria included amputees, bedridden individuals, individuals who could not remain in the proper position for the required measurements, and those who were unable to answer questionnaire due to physical or cognitive impairments. Anthropometric and blood pressure measurements were not obtained from pregnant women. Women who had delivered a baby within the past 6 months were excluded. We attempted to find all eligible adults in their home at least four times, with at least one visit on weekends and another in the evening; cases in which the interviewer could not locate the interviewee or there was a refusal to participate were considered losses.

Data collection

Before initiating data collection the questionnaire was pilot tested on 35 individuals and the procedures were pilot tested on 100 individuals who were not study participants Home visits included the administration of a face-to-face questionnaire, two blood pressure measurements, and anthropometric measurements such as weight and height. All interviewers (n = 35) were trained prior to field work.

Outcome

Participant reports of perceived neighborhood problems were the dependent variables. Neighborhood perceptions were evaluated based on responses to 16 items referring to: garbage, uneven pavements, unpleasant smells, air, water or ground pollution, lack of safe place to children play, traffic speed, urban transport, vandalism, burglaries, assaults, murders, drug use, safety walking after dark, bad reputation, and police problems. These items were adapted from a questionnaire developed by Ellaway et al. [27]. For each item the response options were none, some or many problems (related to the specific item) in the neighborhood, for analysis those options were codded as zero, one, or two, respectively.

Group-level covariates

We used the tertiles of the household head of the family average monthly per capita income from the 2000 Brazilian census (http://www.ibge.gov.br) for each of the 63 census tracts.

Individual-level covariates

The individual covariates included sex, age (years), educational attainment (12y and more, 9-11y, 5-8y or 0-4y), total of earnings in the last month by the household residents divided by the number of residents -per capita income in Reais (R$) (Brazilian currency; US$ 1.0 = R$1.7, during the period of data gathering), race/self-reported skin color (white, brown, and black) [24], length of time living in the neighborhood (years), and occupational status which was classified according the British Registrar General’s Social Class [28] (manual or non manual job, students, housekeeping; individual who had never worked at the moment of data gathering, were placed in a separate category).

Reliability was assessed by administering a short version of the questionnaire (n = 10) to 15% of the whole sample (n = 248) using a telephone interview. Reliability is defined as the extent to which the questionnaire produces the same results on repeated trials. The measure can be used to assess the stability or consistency of scores over time or across raters [29].

Kappa statistics and the intra-class correlation coefficient were calculated to assess reliability, and the values ranged from 0.6 (pain, medicine use and dental prosthesis) to 0.9 (length of residence time in the same neighborhood).

Statistical analysis

In order to group perceived neighborhood items in scales, we performed a principal factor analysis of all neighborhood questionnaire items after polychoric transformation [30], using orthogonal rotation. The scree test, factor loadings, scale internal consistency, and theoretical considerations were applied to define the number of factors to be extracted, as well as the items comprising each scale. The Cronbach’s Alpha was calculated to measure the internal consistency of the scales [31]. Sample size adequacy for factor analysis was evaluated using the Kaiser-Meyer-Olkin (KMO) test [32]. The values of KMO test ranged from 0.81 to 0.94 for vandalism and assaults, respectively, and the global value was 0.88.

Furthermore, assessing the measurement properties of ecological settings moves beyond an assessment of the psychometric properties to what has been termed ‘ecometrics’ [16]. Ecometrics is an extension of the two levels implicit in traditional psychometric assessments (scale item response nested within individuals) because it introduces a third level of neighborhoods. It allows the quantification not only of how consistently individuals respond to the different component items of a scale (the internal consistency measure of psychometrics) but also the extent to which residents of the same neighborhood rate their neighborhood similarly [16].

The ecometric properties of the neighborhood scales were assessed using three-level multilevel models [11]. Level 1 corresponds to item responses within individuals. Level 2 corresponds to persons nested within neighborhoods and finally level 3 corresponds to neighborhoods. Through those analyses variance components were estimated for each level: within-person, within-neighborhood, and between-neighborhood, for levels: 1, 2, and 3, respectively.

Using this estimates, we calculated the intra-class (intra-neighborhood) correlation coefficient (ICC), and the reliability of the neighborhood-level measure. The ICC quantifies the percentage of variability in the scale score that lies between neighborhoods [33]. The ICC ranges from 0 to 1, with a higher value indicating greater agreement between respondents within a neighborhood.

The neighborhood level reliability of the neighborhood score [11, 33] is a function of the ICC as well as the number of participants in each neighborhood (n _jk). It is calculated as the ratio of the “true” score variance (portion of the score which is replicable or reliable) to the observed score variance in the sample neighborhood mean, with values ranging from 0 to 1. The reliability will be high (close to 1) when: 1) the neighborhood means vary substantially across neighborhoods (holding constant the sample size per group), or 2) the sample size per neighborhood is large. Furthermore this measure also increases when the number of scale items raises.

The three-level multilevel analysis allowed the estimation of Bayesian estimates [11]. Crude scores were tested in relation to individual and census tract level variables. In order to evaluate the convergent validity, related to spread which scales were associated with other neighborhood characteristics in the expected direction, were investigated associations among perceived neighborhood scales and familiar per capita income from the Brazilian Census [24].

We fitted three different models for each neighborhood scale. The first model included demographic variables (sex, age, skin color and length of time living in the neighborhood); the second model added individual-level socioeconomic characteristics (per capita familiar income, educational attainment and occupational status), and the third model added census tract income. All variables were kept in the model. The ICC was calculated for each model.

The software STATA, version 12.0 was used to perform these analyses. Univariate and bivariate analyses were performed, taking the complex sample into account, considering unequal probability to participate of data gathering of residents from different census tract (weighted and clustered sample). All multilevel models were also weighted.

Ethical issues

The research project approved by the Ethics Committee of Research in Human Subjects of the Federal University of Santa Catarina – number 351/08. Informed consent was obtained from all participants.

The response rate was 85.3%, (1,720 adults). Participants were distributed in 63 census tract with a range of 10 to 40 persons per tract, and an average of 27.3 respondents per tract. Over half of the sample (55.5%) was female, the mean age of the sample was 38.1 yr, and almost 90% reported being white. The mean length of time living in the neighborhood was 13.4 years (Table 1).

Resident’s perceptions of neighborhood problems were grouped in two dimensions after principal factor analysis: neighborhood physical problems and neighborhood social disorder, with internal consistency of 0.67 and 0.81, respectively. The variables with higher factorial loadings on neighborhood physical problems were garbage, uneven pavements, unpleasant smells and air, water or ground pollution. The variables with higher loadings on the social disorder factor were burglaries, assaults, drug use, vandalism and murders or kidnappings (Table 2). These two factors explained 79% of the item variance. The correlation between two neighborhood scales was 0.49 (p < 0.001).

The neighborhood ICCs observed for the scales were 0.28 and 0.27, for physical problems and social disorder, respectively. Corresponding reliabilities were 0.95 and 0.96 (Table 3).

In bivariate analyses, individuals aged 50 years or over perceived fewer neighborhood physical problems than younger individuals. Those who had lived in the neighborhood more than five years reported higher neighborhood social disorder. Individual-level socioeconomic characteristics were not significantly associated with the scales. However there was a pattern by which individuals in the higher tertiles of family and neighborhood income reported lower scores than those in the bottom tertile. Conversely, higher education tended to be associated with higher scores (Table 4).

After adjustment, an inverse association between age and neighborhood problems scales remained. People living longer in the neighborhoods had higher scores for neighborhood social disorder problems, whereas those with lower educational attainment reported fewer problems. Residents in higher income neighborhoods reported lower rates for neighborhood problems. The ICC remained stable even after all adjustments (Table 5).

This paper investigated the measurement properties of scales utilized to measure neighborhood problems in an urban area in Brazil. We also examined whether neighborhood problems were associated with selected individual and census tract level characteristics. Two neighborhood problems scales were identified from the 16 items: one measuring physical characteristics and the other measuring social characteristics. The internal consistency of the scales was high (0.67 to 0.81). The ecometric properties of the scales measured by ICC and reliability were good, in the order of 0.24 to 0.28 for ICC and 0.94 to 0.96 for reliability. Higher values on the scales representing higher level of problems in the physical and social domains were associated with younger age, more length of time residing in the neighborhood and lower census tract income level.

The ecometric and psychometric properties of the scales were similar to those found in other studies. Using data from three in three United States sites (Baltimore, Maryland; Forsyth County, North Carolina; and New York, New York) Mujahid et al. [12] reported ICCs ranging 0.05 to 0.51 for activities with neighbors, and aesthetic quality, respectively. Friche et al. [24] in Belo Horizonte, Brazil tested ten scales and reported ICCs ranging from 0.02 to 0.33 for social cohesion, and walking environment, respectively. Friche et al. [23] investigated physical and social disorder scales similar to ours and reported ICCs of 0.14 and 0.13, for physical and social disorder, respectively. The ICC quantifies the percentage of variability in the scale that lies between neighborhoods. High value of ICC indicates greater agreement between respondents within a neighborhood [12]. The values of reliability observed in the Epifloripa Study were high, and similar to those found in a Southeastern Brazilian metropolis [23]. These results indicate that the mean of scores are good estimators of the true neighborhood scores for each scale [12].

Corsi et al. [34] analyzing ecometric properties of responses to questionnaire items from 2,360 individuals residing in communities of 5 countries (China, India, Brazil, Colombia and Canada) of the Environmental Profile of a Community’s Health (EPOCH) study, found reliabilities ranged from 0.81 for community social cohesion in urban communities to 0.96 for knowledge of the health effects of smoking in a rural communities.

The sixteen collected items on perceived neighborhood problems were grouped in two scales reflecting social and physical dimensions. Similar clustering of neighborhood measures has also been reported by others [27, 35]. The neighborhood physical problems scales aspects was primarily linked to environmental problems such as garbage, uneven pavements, unpleasant smells and air. On the other hand, aspects connected with social problems such burglaries, assaults, drug use, vandalism and murders loaded on a separate social disorder scale.

Neighborhood physical problems and social disorder scales were moderately correlated, suggesting that they may measure distinct although interrelated constructs [16, 17]. Neighborhood problems would be expected to be greater in residential areas with more social problems; for example, concern about issues such as litter and walking around after dark may be more severe in places in which antisocial behavior is not proscribed [14].

The convergent validity of the scale, attested for their relation with other variables, in expected direction was good. There is some evidence linking individual characteristics and perceived neighborhood problems [12, 17]. In Epifloripa Study people aged 50 years and over perceived fewer problems in their neighborhood, when compared with youngest. Similarly, in a study of Canadian adults, Pampalon et al. [36], observed that people aged 45 years and over perceived fewer neighborhood social and environmental problems than the younger persons [36].

The length of residence in the neighborhood was associated with higher scores for neighborhood social disorder. Ellaway et al. [27] found that length of residence and neighborhood stability were found to be significant in two ways: longer residence is associated, up to a point (15 years), with a stronger sense of ‘neighboring’; and longer intended residence is itself a key element of ‘attraction to neighborhood’ [27]. People living longer in the same neighborhood may have a sense of the changes that have occurred in their neighborhood over time, and perceive more violence problems in their neighborhoods [37].

As expected, people living in lower income areas reported more problems than those living in higher income areas. For a number of reasons that have to do with inequities in power, resource distributions, and access to opportunities socially disadvantaged areas are likely to face adverse physical and social environments. However, contrary to expectation we found that individual–level income was only weakly associated with neighborhood problems; people in the higher income tertile presenting slightly lower scores of problems in neighborhood. For educational attainment the association was the opposite of expectation: with lower scores for groups with lower educational attainment years. However associations with individual level SES were generally weak and often not statistically significant. Friche et al. [23] noted similar patterns in the association of neighborhood scales with socioeconomic indicators. They [23] pointed out the contrasts between poor areas adjacent to rich areas, typically observed in Brazilian urban centers, as a possible explanations for the weak associations observed. This may influence the individual responses, because regardless of socioeconomic level, people may share some similar environments and services available for a broader area, resulting in similar perceptions across economically diverse adjacent neighborhoods [23].

Like other studies [12, 23] our results also suggest that there is variation in responses within neighborhoods. Part of this may be due to the arbitrary geographic definition of neighborhoods that we used. Although clearly census tracts capture some spatial heterogeneity (as indicated but the neighborhood ICCs) there is likely to be substantial spatial heterogeneity which is not captured by census tracts [12]. Brazilian census tracts were defined to be the smallest territorial unit that can be reasonably covered in fieldwork. Each census tract has around 300 households [26].

Additional sources of variability within neighborhoods may be attributed to subjectivity inherent to perceptions and the error in measurement. The occurrence of within-neighborhood differences suggests that it may be beneficial to average over neighborhood respondents or raters in estimating neighborhood characteristics [12].

Sample size in multilevel analyses has been an ongoing area of work. The number of respondents in each cluster of Epifloripa Study ranged 10 to 40, only one of 63 census tract evaluated, had only 10 observations. The average number of individuals per group-level unit was 27.3. Mujahid et al. [12] reported that 25–30 participants per cluster often maximizes reliability [33] Mass e Hoss [38] performed a serial of simulations with different numbers of groups and individuals in each group, and observed non-significant bias for most regression and variance components under conditions similar to those observed in the Epifloripa Study. However they did report that he standard errors of the second-level variance can be underestimated when the number of groups is substantially lower than 100 [4].

There is an ongoing debate on the nature of variables involved in neighborhood analyses. Cummins et al. [4] argue that the distinction we often make between composition and context is somewhat artificial. This can acquire special relevance when it comes to residents perceptions about their own neighborhood, because an individual-level variable is used to capture neighborhood-level realities. However, disaggregated individual and group sources of variability can be useful. The fact that variance in reported can be decomposed into between area and within area variability indicates that these perceptions are at least in part capturing truly contextual features [38].

The findings of this study showed acceptable ecometric properties of the proposal scales, and documented associations of perceptions with individual and contextual socioeconomic characteristics. Those scales have been applied in analysis of EpiFloripa study against other outcomes, i.e. self-rated health, showing important association even after adjustment for other socioeconomic, demographic, health related behaviors and health status variables. Place specific characteristics related to broader geographic and social contexts such as cities or countries may influence the relationship between perceived neighborhood problems and objective socioeconomic measures at both the individual and census tract level. Future works can apply these scales to examine how places influence health.

95% CI:

95% confidence intervals

ICC:

Intraneighborhood correlation

KMO:

Kaiser-meyer-olkin

OR:

Odds ratio.

Amélia Augusta Friche, of the Observatory for Urban Health of Belo Horizonte of Federal University of Minas Gertais, Belo Horizonte, Brazil, for her contribution during the performing of three level multilevel analyses.

We thank Dr. Nilza Nunes da Silva, Department of Epidemiology, School of Public Health of University of São Paulo, São Paulo, Brazil, for her advice on sample procedures. We would like to thank the Brazilian Institute of Geography and Statistics (IBGE) and the Florianópolis Health Authority staff for their useful help with the practical aspects of the study.

Affiliations

1. Post-Graduate Program in Public Health, Federal University of Santa Catarina, Florianópolis, Brazil

   • Doroteia Aparecida Höfelmann

2. Center of Social Epidemiology and Population Health, School of Public Health, University of Michigan, Ann Arbor, USA

   • Ana V Diez-Roux

3. Department of Epidemiology, School of Public Health, University of São Paulo, São Paulo, Brazil

   • José Leopoldo Ferreira Antunes

4. Australian Research Centre for Population Oral Health, School of Dentistry, the University of Adelaide, Adelaide, Australia

   • Marco Aurélio Peres

Corresponding author

Correspondence to Marco Aurélio Peres.

Competing interests

This paper is based on the EpiFloripa Adults 2009 – Florianópolis Adults Health Survey. The Project was sponsored by the Brazilian National Council for Scientific and Technological Development (CNPq), grant number 485327/2007-4. This research was developed in the Post-graduate Program in Public Health, Federal University of Santa Catarina, Brazil. Doroteia A. Höfelmann had scholarship of Coordination of Improvement of Higher Education Personnel (CAPES) grant number BEX 4978110. José Leopoldo F. Antunes and Marco Aurélio Peres received grants for research productivity (CNPq). Ana Diez-Roux received a grant from Fogarty Institute/National Institute of Health (NIH) 5R03 TW008105.

Authors’ contributions

DAH participated in the study design, including data quality control, supervised the fieldwork activities, performed multilevel analysis of the data, and study’s analytic strategy. JLFA, collaborated in the review of literature, helped with the analytic strategy, instructed multilevel analysis of the data and reviewed the final version. AVD-R, participated in the manuscript conception, helped with the analytic strategy, reviewed critically the manuscript and helped with statistical analysis, MAP coordinated the Epifloripa Study, participated in the study design, study’s analytic strategy, and in the conception and writing of the manuscript. All authors read and approved the final manuscript.

Reprints and Permissions