Skip to main content
Download PDF

Instruments to measure e-cigarette related constructs: a systematic review

BMC Public Health volume 22, Article number: 1135 (2022) Cite this article

Abstract

Background

Electronic cigarettes (e-cigarettes) are relatively new tobacco products that are attracting public attention due to their unique features, especially their many flavor options and their potential as an alternative to cigarettes. However, uncertainties remain regarding the determinants and consequences of e-cigarette use because current research on e-cigarettes is made more difficult due to the lack of psychometrically sound instruments that measure e-cigarette related constructs. This systematic review therefore seeks to identify the instruments in the field that are designed to assess various aspects of e-cigarette use or its related constructs and analyze the evidence presented regarding the psychometric properties of the identified instruments.

Methods

This systematic review utilized six search engines: PubMed, Medline, CINAHL, PsycINFO, Web of Science, and EMBASE, to identify articles published in the peer-reviewed journals from inception to February 2022 that contained development or validation processes for these instruments.

Results

Eighteen articles describing the development or validation of 22 unique instruments were identified. Beliefs, perceptions, motives, e-cigarette use, and dependence, were the most commonly assessed e-cigarette related constructs. The included studies reported either construct or criterion validity, with 14 studies reporting both. Most studies did not report the content validity; for reliability, most reported internal consistencies using Cronbach’s alpha, with 15 instruments reporting Cronbach’s alpha > 0.70 for the scale or its subscales.

Conclusions

Twenty-two instruments with a reported development or validation process to measure e-cigarette related constructs are currently available for practitioners and researchers.

This review provides a guide for practitioners and researchers seeking to identify the most appropriate existing instruments on e-cigarette use based on the constructs examined, target population, psychometric properties, and instrument length. The gaps identified in the existing e-cigarette related instruments indicate that future studies should seek to extend the validity of the instruments for diverse populations, including adolescents. Instruments that explore additional aspects of e-cigarette use and e-cigarette related constructs to help build a strong theoretical background and expand our current understanding of e-cigarette use and its related constructs, should also be developed.

Peer Review reports

Background

Electronic cigarette (e-cigarette) use is emerging as a major item on the public health agenda, attracting both greater attention from researchers, and intense scrutiny from the popular media. A significant increase in the prevalence of e-cigarette use among US adults has been reported since 2010 [1,2,3]; in 2019, 4.5% of adults in the US self-reported using e-cigarettes, of whom 36.9% identified as dual users [4] Among youth, steep rises in nicotine vaping and e-cigarette product use have resulted in an overall increase in the use of tobacco products. In 2020, 19.5% of high school students and 4.7% of middle school students used electronic nicotine delivery systems (ENDS) [4, 5].

Developed to closely approximate the sensory experience of smoking combustible cigarettes, e-cigarettes produce an aerosol by heating a liquid containing a solvent (generally vegetable glycerin, propylene glycol, or a mixture of the two), one or more flavorings, and nicotine, although liquids containing no nicotine are available on the market for some devices [6]. E-cigarettes have gained considerable popularity among both youth and adults in recent years in spite of the dearth of research into the devices’ safety, effects, and efficacy [7]. Hence, while the research regarding the potential health effects of e-cigarette use is still in its infancy, researchers are beginning to try to understand people’s perceptions, reasons, and behaviors in order to better understand their use of e-cigarettes.

Despite reports that e-cigarettes emit substantially lower levels of carcinogens and thus represent a safer alternative to combustible cigarettes [8], young people who use e-cigarettes have shown increased risk of trying combustible cigarettes [9]. Moreover, with hundreds of e-cigarette brands already on the market, vaping products are evolving rapidly in terms of their mechanisms, engineering, design, and usability, all of which are aimed at boosting their appeal for curious youngsters and thus posing an additional concern as sales of these products continue to rise. This raised serious concerns for young people’ health because young e-cigarette users are reported to have more physical and mental health issues [10]. E-cigarettes contain nicotine, and exposure to toxicants, such as nicotine, has deleterious effects on the developing brain [11, 12]. Furthermore, there are substances, such as formaldehyde, and acetaldehyde, which cause cancers [13, 14].

As research in this area increases, it is vital that studies that focus on e-cigarette use are able to utilize reliable and valid e-cigarette use measures when assessing their results. Major gaps remain in our knowledge of the effects and potential hazards posed by e-cigarettes that require extensive research, particularly when it comes to exploring major factors associated with e-cigarette use such as the motivators influencing the decision to use e-cigarettes and the consequences of e-cigarette use. However, there are some unique challenges for those developing new instruments to measure these constructs. E-cigarettes are relatively new and rapidly evolving products and thus, there is significant variability in the products currently on the market, including refillable options as well as pens, pods, and other configurations; the different patterns of e-cigarette use include experimentation, regular use, and dual use. However, presently, there is limited information regarding the validity of the various instruments developed to examine the multi-faceted issues involved and a clear need to evaluate the measurement properties of each of these instruments.

To date, there have been no systematic evaluations of the available evidence supporting the measurement properties of these validated instruments for e-cigarette use. Hence, the purpose of this systematic review is to review and synthesize validated survey instruments in the literature that are specifically designed to explore e-cigarette related constructs. In this context, survey instrument refers to the data collection tool that measures a construct in survey methods [15]. In addition, e-cigarette-related construct is defined as a construct chosen by researchers to explore and analyze the mechanisms of e-cigarette use or the phenomena associated with its use in survey studies. Construct is defined as “an image, idea, or theory, especially a complex one formed from a number of simpler elements ([16] , p.1),” and it usually refers to the latent construct that is inferred by observing indicating behaviors. For example, constructs such as motivation, dependence, perceived harms and benefits, and dependency of e-cigarette use are often explored in e-cigarette survey research, thus, these are typical examples that we would expect to include as e-cigarette-related constructs. Specifically, this study aims to provide an overview of existing instruments developed for measuring e-cigarette-related constructs including the development or validation process and psychometric properties, thus bridging a serious gap in e-cigarette survey research. Our ultimate goal is to assist both researchers in the field and clinicians to make informed choices when selecting an appropriate instrument for the measurement of e-cigarette use.

Methods

This systematic review was registered with the PROSPERO international prospective register of systematic reviews and was conducted following the guidelines laid out in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) [17].

Search strategy

A systematic search was conducted using six electronic databases: PubMed, Medline, Cumulative Index of Nursing and Allied Health Literature (CINAHL), PsycINFO, Web of Science, and Excerpta Medica dataBASE (EMBASE) from inception to February 2022. In addition, the works cited in the reviews and the references in the retrieved articles were screened. To broaden our search results, we entered our search terms using two categories, namely e-cigarette and instrument. The following search keywords or Medical Subject Headings (MeSH) were thus used: vaping device or vape or electronic cigarettes or e-cigarettes or e-liquid or electronic nicotine delivery systems [mesh] AND psychometrics [mesh] or questionnaires or surveys or surveys and questionnaires [mesh]. It is important to note that we did not specify search terms that would limit the constructs related to e-cigarette use. For example, we did not utilize terms such as motivation, belief, symptom, perceived harms or benefits, consequences, and behavior even though we were aware of instruments that measured some of these constructs as they are often used in e-cigarette survey research. By adopting this approach, we were able to explore the extent of the constructs that are assessed by validated instruments.

A filter was applied to retrieve articles published in English, but no language restriction was applied for the instruments. In addition, a snowballing technique was used to suggest additional searches: if the article referred to earlier articles that described the process of development or validation of the instruments, we also retrieved those articles and checked their eligibility for inclusion in this review.

Inclusion and exclusion criteria

Predetermined inclusion criteria were applied to select relevant studies, which included (1) Articles reporting the development and/or validation process for survey instruments designed to measure electronic cigarette related constructs (e.g., motivation, dependence, perceived harms and benefits, consequences, and behavior); (2) Full text articles published in peer-reviewed research journals; and (3) Articles published in English, where the instruments were translated into English for the purpose of the analysis. Likewise, the study specified the following exclusion criteria: (1) studies that are not empirical; (2) single-item instruments; (3) the validation or development process of survey instruments were not reported; and (4) instruments designed for use in laboratory settings.

Selection process

Applying the aforementioned inclusion and exclusion criteria, two authors screened relevant titles and abstracts independently. Studies that met the criteria were accessed and independently reviewed multiple times by all the authors and discrepancies were reconciled through consensus discussions.

Data extraction

To provide an overview of the instruments and the psychometric properties of each, the following coding schemes were used: (1) Basic information on the instruments, including the name of the instrument, the name of the first author, the constructs that the instrument is designed to assess, the country where the study took place, the theoretical background of the instrument, the mode of administration, the completion time, and the response options (Table 1); and (2) The psychometric properties of the instruments, including the constructs, sub-constructs, various types of reliability reported (e.g., internal consistency, test-retest reliability) and validity (construct, content, and criterion validity) tested (Table 2).

Table 1 Overview of Instruments
Full size table
Table 2 Basic Information about the Instruments
Full size table

Quality appraisal and risk of Bias

We assessed quality appraisal and risk of bias in the included studies using the COSMIN Risk of Bias checklist (Table 4) [39]. The COSMIN Risk of Bias checklist addresses ten specific domains: (1) PROM design; (2) content validity; (3) structural validity; (4) criterion validity; (5) internal consistency (6) cross-cultural validity/ measurement error; (7) reliability, which is tested with test and retest; (8) measurement error; (9) criterion validity; (10) hypotheses testing for construct validity; and (11) responsiveness. Three review authors (M.K., Y.Z., and C.C.) independently applied the tool to the included studies (n = 23) and recorded judgements of risk of bias for each domain (very good, adequate, doubtful, or inadequate). The judgements of “very good” or “adequate” indicated the quality of studies. Following guidance given for COSMIN, we derived an overall summary in the “Quality of appraisal and risk of bias” table for each specific domain, whereby the overall COSMIN for each study was determined by the quality level and risk of bias in ten domains.

Results

Search results

In total, the search yielded 1454 articles. After two researchers had independently reviewed the titles and abstracts of all articles, 87 were selected to undergo a full text examination, after which the same two researchers independently reviewed their full texts. Of the 87 articles, 65 were excluded as they did not report specific information about the development or validation process utilized. After the full text review based on the eligibility criteria, 23 studies were found to be suitable for inclusion in the current study (see Fig. 1. PRISMA flow chart). One study tested the validity of three instruments that measure the same construct, namely e-cigarette dependence [40]; one study reported the validity of both the long and short versions of the instrument, and revised the youth e-cigarette outcome expectancies respectively [18], and two studies conducted a validity test on the same instrument [18, 37, 40]. Thus, a total of 22 instruments from 23 different studies were analyzed for the current study.

Fig. 1
figure 1

PRIMA Diagram

Full size image

Overview of instruments

Tables 1 and 2 provide an overview of the characteristics of the instruments presented in the included articles. This section presents an overview of the settings of these studies, the age ranges of their participants, the theoretical frameworks utilized, the modes of administration and durations of the tests, the number of test items, and the response options. This information will guide the users to choose the appropriate instruments depending on their purpose. For example, users will know which instrument exists to measure a construct of their interests, and the target population and the settings under which instruments were validated. Then, the modes of administration, duration of the tests, number of items, and response options can provide additional practical information in choosing which instrument they want to choose.

Country and language

Of the 22 instruments, 77.3% (17/22) studies for validation or reliability test in the US; the others were conducted in Canada, Australia, and Hungary [24, 32, 38, 41]. All but one instrument was developed in English. The exception was an instrument originally developed in Hungarian and later translated into English [41]. Although the items of this instrument are available in English, the instrument has not been validated using a cross-cultural translational process.

Participants

Regarding the ages of the participants, 72.7% (16/22) of the instruments were designed for use with participants aged 18 years or above; the remaining 27.3% (6/22) were for younger participants who were under 18 years of age [19, 23, 27, 32, 33]. Among the instruments validated for participants aged 18 or above, six instruments were specifically targeted at young adults (18 to 25 years old or college students) [18]. One instrument was validated based on its use with hospitalized patients [26].

Administration, number of items, and responses

There were some variability and ambiguity with regard to the modes of administration of the instruments. Although the majority of the tests were administered “online” (63.6%, 14/22), 27.3% (6/22) of the studies did not report the mode of administration [18, 19, 27, 32, 40]. The vast majority of the studies (77.3%, 17/22) did not report the completion time for their instruments; the remaining 13.6% (3/22) specified either the completion time of the instrument (n = 1) or the completion time of the study (n = 4) [35]. However, the number of items in each instrument can provide a rough estimate of the completion time required. There was a considerable variability with respect to the number of items in the instruments, which ranged from 2 to 55 with a mean of 15.81. Finally, the majority of the instruments (68.2%,15/22) utilized Likert-type response options, varying from 1 to 2 to 1–10, a further 18.2% (4/22) did not report the response options [21, 23, 40], 9.19% (2/22) had mixed response options [37, 38], and 4.5% (1/22) had True/False response options [29].

Theoretical background

The majority of the studies (68.2%,15/22) did not describe the theoretical background of their instrumentation clearly (Table 2). The other 22.7% (5/22) did present the theoretical framework underpinning their instruments by including a discussion of the relevant motivation theories, theories of planned behaviors, social learning theory, and/or expectancy theory [18, 21, 23, 24, 26, 28].

Constructs

The 22 instruments identified a total of 15 different constructs reflecting the multiple constructs explored in e-cigarette survey research (Table 1). General beliefs and perceptions were identified as the most commonly explored construct, with individual studies specifically including outcome expectancies [18], sensory vaping expectancies [20], and the risks and benefits of e-cigarettes [22, 23].

Fourteen instruments sought to assess beliefs, perceptions, and attitudes about e-cigarettes, specifically comparing the beliefs or perceptions to the beliefs or perceptions about cigarette smoking using constructs such as comparative beliefs of e-cigarette use [24], e-cigarette expectancies compared to cigarette smoking [29], and perceived harms compared with cigarettes [27]. One instrument was designed to assess the perceived harms and social norms of both e-cigarette and smokeless tobacco in a single instrument [28], while another was specifically developed to assess the expectancies of simultaneous e-cigarette and alcohol use [29].

Motivation for e-cigarette experimentation and susceptibility to future use were identified as constructs that were explored by four instruments [41]. These instruments assess motivation or likelihood of using e-cigarettes specifically among non-cigarettes users [32, 33, 41]. Among these instruments, one assessed susceptibility to four different tobacco products, namely e-cigarettes, cigarettes, cigars, and hookahs [33]. Looking at e-cigarette use exclusively, one instrument assessed habitual e-cigarette use [34], but no studies that specifically reported the development or validation of instruments assessing e-cigarette use were identified.

The next most commonly assessed constructs were e-cigarette craving and e-cigarette dependence. One instrument assessed e-cigarette craving based on three sub-constructs, namely desire, intention, and positive outcome [35]. For smoking dependence, four instruments were identified [37, 38, 40]. Of these, three instruments had a one single construct, but one instrument (e-WISDM) has 37 items consisting of 11 sub-constructs: affiliative attachment, affective enhancement, automaticity, loss of control, cognitive enhancement, craving, cue exposure, social/environmental goals, taste, tolerance, and weight control [40].

In terms of the number of sub-constructs within each instrument, most had several sub-constructs (range = 1 to 11; Mean = 3.29). Interestingly, six instruments had no sub-constructs and only a single domain, either e-cigarette dependence [38], susceptibility to future e-cigarette use [32, 37, 40], or habitual e-cigarette use [34].

Psychometric properties of instruments

In the current study, we examined the psychometric properties of the various instruments included in the 22 relevant instruments identified (Tables 1 & 3).

Table 3 Psychometric Properties of the Included Instruments
Full size table

Reliability

Reliability refers to the degree to which the participants’ responses are repeatable, which is often measured by test–retest. In addition, reliability is often referred to as internal consistency, meaning the degree to which the set of items in the scale vary relative to their sum score, which is often estimated by Cronbach’s alpha [42]. Internal consistency based on Cronbach’s alpha was the only reliability test used in the identified studies. None reported an item analysis or the test and retest reliability. Among the identified studies, most instruments (86.4%, 19/22) reported internal consistencies, and two studies did not report (68.2%, 15/22) were supported with the Cronbach’s alpha ≥ .70 [19, 32], although two borderline values of .67 [26] and .68 [41] were found. Two studies reported Cronbach’s alpha <.67 [38, 40]. Most studies did report domain-specific Cronbach’s alpha scores.

Validity

Content validity refers to the adequacy of items or content relevant to an instrument for the construct that is measured [43]. A common method to support content validity is through computing experts’ ratings of item relevance [44]. Content validity was reported in only one instrument. This study also reported the process used for the participant interviews as part of the process of the instrument development [18, 37, 40].

Construct validity determines the extent of the construct dimensions and their underlying relationships. This is often confirmed by confirmatory factor analysis. In addition, other tests, such as convergent validity, discriminant validity, correlation analysis, group different tests, can be used. Convergent validity, which examines the same concept, is measured by different instruments but yields similar results. Discriminant validity examines if different concepts are measured by different instruments as intended. Correlation analysis examines the relationship between the newly developed instrument and new instruments, and group difference tests the differences between distinct different groups [42].

Among the included studies, construct validity was reported in 86.4% (19/22) of the all instruments included (Table 3). Most studies tested the dimensionality to support construct validity using either confirmatory factor analysis (CFA) or exploratory factor analysis (EFA). Only seven instruments were tested by both EFA and CFA or principal factor analysis (PFA) and CFA as analytic methods [18, 20, 24, 29, 34, 35]. In addition, construct validity was supported by testing the correlation with existing e-cigarette use-related measures, such as dependence measures [22, 26, 35, 37, 40]. Two studies reported testing the convergence and divergence validity [27, 35]. A number of studies reported measurement invariance in testing the instrument [21, 34], as well as construct validity in order to test differences in e-cigarette use status [22, 23].

Criterion validity is to ensure the instrument measures the latent dimension as intended, which is often tested with predictive validity and concurrent validity. Those tests determine whether the score of the instrument predicts or has a strong relationship with the outcome measures or criterion measures. In this case, studies that tested whether the construct of the instrument predicted future e-cigarette use behaviors (predictive validity) or have a significant association between e-cigarette use and the construct of the instrument (concurrent validity) were considered that they checked criterion validity. Criterion validity was reported by 77.3% (17/22) of the included articles (Table 3). Most studies reported either the concurrent validity or predictive validity. For concurrent validity, the associations with existing measures such as e-cigarette use or e-cigarette experimentation were tested and reported acceptable criterion validity [18, 20, 23, 24, 26,27,28, 41]. For predictive validity, four instruments were tested in two studies to explore whether the constructs of the instrument measures would predict positive future e-cigarette use [32, 40].

Quality appraisal

The overall quality of the studies based on the Cosmin Risk of Bias checklist varied (Table 4). Most studies had problems with the PROM design criteria. Studies should provide clearer description of the constructs to be measured with the theoretical framework. In addition, most studies did not test for content validity through qualitative methods. Structural validity was supported by CFA and EFA. However, a few studies only conducted EFA, but needed to conduct CFA as well. For internal consistency, most studies reported Cronbach’s alpha based on subscales, but a few studies only reported Cronbach’s alpha for the whole scale, even when they were not measuring unidimensional construct. In addition, most studies did not report reliability with appropriate methods, such as the intraclass correlation coefficient or Kappa score. In addition, most studies did not check measurement error, and responsiveness, and studies need to check criterion validity and measurement invariance.

Table 4 Quality of appraisal and risk of bias
Full size table

Discussion

This paper is the first systematic review of existing instruments on e-cigarette related constructs. A total of 23 studies were identified that focus on the development or validation of 22 instruments. This study provides an overview of these instruments as well as development process, theoretical framework, target population, and psychometric properties. This review can serve as a useful guide for healthcare professionals and researchers seeking to conduct assessments or conduct research into the phenomenon of e-cigarette use.

In this review, we identified several e-cigarette related constructs in existing instruments. Beliefs or perceptions about e-cigarettes were considered the most commonly studied determinants of current e-cigarette use based on the validated instruments. To explore beliefs or perceptions, constructs including outcome expectancies, sensory expectancies, and perceived risks and benefits were explored. These constructs were supported by motivation theories, social learning theory, and the theory of planned behavior. In addition, beliefs or perceptions about the relative merits of e-cigarettes and smoking conventional cigarettes were another commonly explored construct. In terms of the motivations for e-cigarette experimentation and susceptibility to future use were constructs explored, although habitual e-cigarette use was the only construct used to assess current e-cigarette use. The consequences or symptoms related to e-cigarette use were also explored with the constructs of e-cigarette craving and dependence. Only a few studies included in this review provided a theoretical background of the instrumentation and did not clearly present the conceptual framework or definition. This may be related to potential issues of clarity of the constructs that each instrument measures. The constructs that each instrument intends to measure may not be specific enough without a theoretical guide [45]. In addition, most studies only described the validation process but did not provide detailed steps of the development process of the instruments, which also limits to clarify constructs that each instrument intended to measure. Interestingly, there were two instruments available for the outcome expectancies although the target populations were different. Three different instruments existed for e-cigarette dependence; however, only one instrument was supported by content, construct, and criterion validity. This may indicate that there is not yet a consensus, and further studies are needed to test the validity of these instruments through comparison studies to draw results that are more accurate.

Regarding reliability, most studies reported acceptable internal consistencies. However, reliability was supported by only one type of reliability and tested the internal consistencies with a single method. This can be a potential threat of internal consistency [46]. It is suggested that three broad types of reliability need to be assured, including (1) reliability from administering parallel forms of instruments (alternate-form coefficients), (2) reliability from administering the same instrument on separate times (test-retest), and (3) reliability based on total scores or subsets of items within a single test (internal consistency coefficient) [47]. It is important to test multiple types of reliability by multiple methods, such as “test-retest” or “item-analysis”. In this way, any systematic error or variations of instruments can be prevented and the generalizability of the use of instruments can be improved.

For validity, only a limited number of studies conducted both EFA and CFA during their analysis, which again limits the construct validity, and few tested either content validity or criterion validity. Part of the reason why content validity was not tested in most studies may be related to the historical aspects of the development process commonly used for e-cigarette-related measures, most of which are based on existing instruments originally developed to assess cigarette smoking related constructs. As these measures have already been extensively validated, the various authors have simply adapted these for e-cigarette specific constructs, not considering this to be a necessary step in the development process. However, it is actually important to capture the unique aspects of e-cigarettes, which are in many ways very different from cigarettes [45]. Moreover, in terms of the criterion validity, although most studies did provide a test of the criterion validity, only a limited number also tested the predictive validity. This seriously limits the validity of the majority of the existing e-cigarette related instruments [42].

Limitations

As always with studies of this nature, there is a risk that relevant articles may have been missed even though we have used a range of different techniques to systematically search for articles; there is also the potential for errors to occur in the review and coding process. To minimize these errors and ensure the reliability of the coding process, two researchers independently coded the articles, and three researchers double checked the accuracy of the coding multiple times. Where discrepancies were identified, three authors reviewed the articles together and came to a consensus. Further review from other researchers would have been considered to deal with any unresolved issues had any such occurred, but this was not found to be necessary.

Moreover, there is also a possibility that not all studies were able to report the full details of their instrumentation or validation process due to limited space in peer-reviewed journals. It is thus possible that the authors were not able to gather sufficient information for each measure in this review from the published reports.

Recommendations for practice

Twenty-two unique instruments assessing the constructs related to e-cigarette use in population studies were identified in this study. Our findings suggest that practitioners should first consider choosing instruments based on the constructs that they are most interested in, depending on the purpose of the assessment. For example, if practitioners are interested in the reasons for e-cigarette use, it would be most appropriate to select an instrument that assesses various types of beliefs or perceptions, while to assess non-users’ motivation or susceptibility to future use, they can choose from three different measures: motivations for e-cigarette experimentation [41], a susceptibility scale [32], and susceptibility to four product classes (e-cigarettes, cigars, hookah and cigarettes) [33]. If they are interested in current habitual use, one instrument has been specifically developed to study this, the Self-Report Habit Index (SRHI) [21], and if the practitioners need to assess the symptoms of current e-cigarette users, a number of measures are available to assess craving or dependence, namely the Questionnaire of Vaping Craving (QVC) [35], the Penn State Electronic Cigarette Dependence Index (PS-ECDI) [37, 40], the E-cigarette Fagerström Test of Cigarette Dependence (e-FTCD) [11], the E-cigarette Wisconsin Inventory of Smoking Dependence Motives (e-WISDM) [11], and the Fagerström Test for Nicotine Dependence applied to Vaping (FTND-V) [38].

After narrowing down the broad categories of constructs depending on the purpose of the assessment, practitioners should consider the age of their target population and make sure that the instrument has been validated for this user group. Among those instruments, those that report a value of Cronbach’s alpha higher than .70 and support both construct and criterion validity should be preferred, although the number of items should be considered to determine the feasibility of their use in clinical settings.

Recommendations for future research

Future work on survey instruments measuring e-cigarette-related constructs that take into account content validity and criterion validity will be necessary if we are to establish a stronger evidence base for e-cigarette research. Currently, most studies report only construct validity, with few also reporting the content validity or predictive validity. It is important that multiple types of reliability in addition to Cronbach’s alpha alone need to be explored and supported when a new instrument is developed. In addition, for the instruments reported low reliability coefficients, item modification is needed to ensure a desirable internal consistency [45, 47].

There is a critical need to develop a reliable and valid instrument with which to assess e-cigarette-related constructs of diverse populations. Currently, only a limited number of instruments assessing e-cigarette related constructs have been validated for adolescent populations that have been specifically designed to assess the perceived risks and benefits of e-cigarettes, their perceived harms compared with cigarettes, and susceptibility to future use. However, among the available instruments for this age group, only one study was reported as having an acceptable internal consistency. Given the dramatic increase in the prevalence of e-cigarette use, there is clearly a need to develop and validate instruments targeted specifically at adolescents and young adults, particularly given that these are the people most likely to be using e-cigarettes. The availability of such an instrument will enhance the rigor of research on e-cigarettes and help us understand the rapid growth in the popularity of e-cigarettes among this population. Furthermore, instruments need to be validated in diverse clinical settings, and there is also a need for validated universal instruments that can be administered across age groups to help us understand the impact of differences in the various associated factors, the characteristics of the different types of e-cigarettes, and the symptoms across both clinical and non-clinical groups.

Providing details of the instrument administration is also important for researchers and practitioners. Currently, only a few articles provide basic information on the characteristics of their instruments, such as the mode and response type. It is important for researchers to report detailed information, such as whether the instrument is provided online or on paper, the various response options, and the number of items to make the instruments more suitable for use by practitioners and other researchers alike.

Conclusions

This systematic review provides a critical appraisal and repository of the instruments measuring e-cigarette-related constructs in the current literature. It serves as a user-friendly guide to help researchers select the most appropriate instrument to suit their needs based on the constructs, target population, psychometric properties, and number of items, all of which can help develop a more accurate understanding of e-cigarette related phenomena for practitioners. For future studies, researchers need to expand the validation of the existing instruments to include more diverse populations, and develop new instruments that are specific to the unique aspects of e-cigarettes. The development of instruments capable of assessing different aspects of e-cigarette use with a strong theoretical background and validation process will be essential to support efforts to develop effective e-cigarette use prevention and cessation programs.

Availability of data and materials

The data that support the fin available used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

E-cigarettes:

Electronic cigarettes

CINAHL:

Cumulative Index of Nursing and Allied Health Literature

EMBASE:

Excerpta Medica dataBASE

ENDS:

electronic nicotine delivery systems

PRISAMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

MeSH:

Medical Subject Headings

CFA:

Confirmatory Factor Analysis

EFA:

Exploratory factor analysis

PFA:

principal factor analysis

SRHI:

Self-Report Habit Index

QVC:

Questionnaire of Vaping Craving

e-FTCD:

E-cigarette Fagerström Test of Cigarette Dependence

e-WISDM:

E-cigarette Wisconsin Inventory of Smoking Dependence Motives

FTND-V:

Fagerström Test for Nicotine Dependence applied to Vaping

References

  1. McMillen RC, Gottlieb MA, Shaefer RM, Winickoff JP, Klein JD. Trends in electronic cigarette use among U.S. adults: use is increasing in both smokers and nonsmokers. Nicotine Tob Res. 2015;17:1195–202.

    Article  Google Scholar 

  2. Schoenborn CA, Clarke TC. QuickStats: percentage of adults who ever used an e-cigarette and percentage who currently use e-cigarettes, by age group—National Health Interview Survey, United States, 2016. MMWR Morb Mortal Wkly Rep. 2017;66(33):892.

    Article  Google Scholar 

  3. Schoenborn CA, Gindi RM. Electronic cigarette use among adults: United States, 2014. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics; 2015.

  4. Centers for Disease Control and Prevention (CDC). About electronic cigarettes (E-Cigarettes). https://www.cdc.gov/tobacco/basic_information/e-cigarettes/about-e-cigarettes.html. Accessed 12 March 2021.

  5. Cullen KA, Ambrose BK, Gentzke AS, Apelberg BJ, Jamal A, King BA. Notes from the field: increase in use of electronic cigarettes and any tobacco product among middle and high school students — United States, 2011–2018. MMWR Morbid Mortal Wkly Rep. 2018;67(45):1276–7.

    Article  Google Scholar 

  6. Dinakar C, O’Connor GT. The health effects of electronic cigarettes. N Engl J Med. 2016;375(14):1372–81.

    Article  Google Scholar 

  7. Etter JF, Bullen C. Electronic cigarettes: users profile, utilization, satisfaction and perceived efficacy. Addiction. 2011;106(11):2017–28.

    Article  Google Scholar 

  8. Goniewicz ML, Knysak J, Gawron M, Kosmider L, Sobczak A, Kurek J, et al. Levels of selected carcinogens and toxicants in vapour from electronic cigarettes. Tob Control. 2014;23(2):133–9. https://doi.org/10.1136/tobaccocontrol-2012-050859.

    Article  PubMed  Google Scholar 

  9. Stratton K, Kwan LY, Eaton DL. Public health consequences of e-cigarettes: consensus study report: National Academies Press; 2018. http://www.cascobaycan.org/cbcwp/wp-content/uploads/2018/02/Public-Health-Consequences-of-E-Cigarettes.pdf. Accessed 1 Feb 2021

    Book  Google Scholar 

  10. Livingston JA, Chen CH, Kwon M, Park E. Physical and mental health outcomes associated with adolescent E-cigarette use [published online ahead of print, 2022 Feb 1]. J Pediatr Nurs. 2022;64:1–17. https://doi.org/10.1016/j.pedn.2022.01.006.

    Article  PubMed  Google Scholar 

  11. Cross J, Linker KE, Leslie FM. Sex-dependent effects of nicotine on the developing brain. J Neurosci Res. 2017;95(1–2):422–36. https://doi.org/10.1002/jnr.23878.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Goniewicz ML, Knysak J, Gawron M, Kosmider L, Sobczak A, Kurek J, et al. Levels of selected carcinogens and toxicants in vapour from electronic cigarettes. Tob Control. 2014;23:133–9. https://doi.org/10.1136/tobaccocontrol-2012-050859.

    Article  PubMed  Google Scholar 

  13. Kosmider L, Sobczak A, Fik M, Knysak J, Zaciera M, Kurek J, et al. Carbonyl compounds in electronic cigarette vapors: effects of nicotine solvent and battery output voltage. Nicotine Tob Res. 2014;16:1319–26. https://doi.org/10.1093/ntr/ntu078.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  14. Lavacchi D, Roviello G, Rodriquenz MG. Electronic nicotine delivery systems (ENDS): not still ready to put on END. J Thorac Dis. 2020;12(7):3857–65. https://doi.org/10.21037/jtd.2019.12.124.

    Article  PubMed  PubMed Central  Google Scholar 

  15. Elangovan N, Sundaravel E. Method of preparing a document for survey instrument validation by experts. MethodsX. 2021;8:101326. https://doi.org/10.1016/j.mex.2021.101326.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  16. McCoach DB, Gable RK, Madura JP. Instrument development in the affective domain: school and corporate applications. 3rd ed. USA: Springer; 2013.

    Book  Google Scholar 

  17. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4(1):1–9.

    Article  Google Scholar 

  18. Pokhrel P, Lam TH, Pagano I, Kawamoto CT, Herzog TA. Young adult e-cigarette use outcome expectancies: validity of a revised scale and a short scale. Addict Behav. 2018;78:193–9. https://doi.org/10.1016/j.addbeh.2017.11.019.

    Article  PubMed  Google Scholar 

  19. Cristello JV, Sutherland MT, Trucco EM. A preliminary validation of the adolescent e-cigarette consequences questionnaire [published online ahead of print, 2020 Jun 12]. Drug Alcohol Depend. 2020;213:108118. https://doi.org/10.1016/j.drugalcdep.2020.108118.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Morean ME, Krishnan-Sarin S, Sussman S, et al. Development and psychometric validation of a novel measure of sensory expectancies associated with e-cigarette use. Addict Behav. 2019;91:208–15. https://doi.org/10.1016/j.addbeh.2018.08.031.

    Article  PubMed  Google Scholar 

  21. Morean ME, L’Insalata A. The short form vaping consequences questionnaire: psychometric properties of a measure of vaping expectancies for use with adult e-cigarette users. Nicotine Tob Res. 2017;19(2):215–21. https://doi.org/10.1093/ntr/ntw205.

    Article  PubMed  Google Scholar 

  22. Copeland AL, Peltier MR, Waldo K. Perceived risk and benefits of e-cigarette use among college students. Addict Behav. 2017;71:31–7. https://doi.org/10.1016/j.addbeh.2017.02.005.

    Article  PubMed  Google Scholar 

  23. Chaffee BW, Gansky SA, Halpern-Felsher B, Couch ET, Essex G, Walsh MM. Conditional risk assessment of adolescents’ electronic cigarette perceptions. Am J Health Behav. 2015;39(3):421–32. https://doi.org/10.5993/AJHB.39.3.14.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Hershberger AR, Karyadi KA, VanderVeen JD, Cyders MA. Beliefs about the direct comparison of e-cigarettes and cigarettes. Subst Use Misuse. 2017;52(8):982–91. https://doi.org/10.1080/10826084.2016.1268628.

    Article  PubMed  PubMed Central  Google Scholar 

  25. Kale D, Pickering A, Cooper A. Examining the psychometric properties of the CEAC (Comparing e-Cigarette and Cigarette) Questionnaire and its usefulness as a predictor of e-cigarette use. Subst Use Misuse. 2020;55(1):147–55.

    Article  Google Scholar 

  26. Hendricks PS, Cases MG, Thorne CB, et al. Hospitalized smokers’ expectancies for electronic cigarettes versus tobacco cigarettes. Addict Behav. 2015;41:106–11. https://doi.org/10.1016/j.addbeh.2014.09.031.

    Article  PubMed  Google Scholar 

  27. Persoskie A, O’Brien EK, Nguyen AB, Tworek C. Measuring youth beliefs about the harms of e-cigarettes and smokeless tobacco compared to cigarettes. Addict Behav. 2017;70:7–13. https://doi.org/10.1016/j.addbeh.2017.01.033.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Waters EA, Mueller-Luckey G, Levault K, Jenkins WD. Perceived harms and social norms in the use of electronic cigarettes and smokeless tobacco. J Health Commun. 2017;22(6):497–505. https://doi.org/10.1080/10810730.2017.1311972.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Hershberger AR, Karyadi KA, VanderVeen JD, Cyders MA. Combined expectancies of alcohol and e-cigarette use relate to higher alcohol use. Addict Behav. 2016;52:13–21. https://doi.org/10.1016/j.addbeh.2015.08.005.

    Article  PubMed  Google Scholar 

  30. Diez SL, Cristello JV, Dillon FR, De La Rosa M, Trucco EM. Validation of the electronic cigarette attitudes survey (ECAS) for youth. Addict Behav. 2019;91:216–21.

    Article  Google Scholar 

  31. Pénzes M, Foley KL, Balázs P, Urbán R. Intention to experiment with e-cigarettes in a cross-sectional survey of undergraduate university students in Hungary. Subst Use Misuse. 2016;51(9):1083–92.

    Article  Google Scholar 

  32. Cole AG, Kennedy RD, Chaurasia A, Leatherdale ST. Exploring the predictive validity of the susceptibility to smoking construct for tobacco cigarettes, alternative tobacco products, and e-cigarettes. Nicotine Tob Res. 2019;21(3):323–30. https://doi.org/10.1093/ntr/ntx265.

    Article  PubMed  Google Scholar 

  33. Carey FR, Wilkinson AV, Harrell MB, Cohn EA, Perry CL. Measurement and predictive value of susceptibility to cigarettes, e-cigarettes, cigars, and hookah among Texas adolescents. Addict Behav Rep. 2018;8:95–101. https://doi.org/10.1016/j.abrep.2018.08.005.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Morean ME, DeMartini KS, Foster D, et al. The self-report habit index: assessing habitual marijuana, alcohol, e-cigarette, and cigarette use. Drug Alcohol Depend. 2018;186:207–14. https://doi.org/10.1016/j.drugalcdep.2018.01.014.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Dowd AN, Motschman CA, Tiffany ST. Development and validation of the questionnaire of vaping craving. Nicotine Tob Res. 2019;21(1):63–70. https://doi.org/10.1093/ntr/nty046.

    Article  PubMed  Google Scholar 

  36. Piper ME, Baker TB, Benowitz NL, Smith SS, Jorenby DE. E-cigarette dependence measures in dual users: reliability and relations with dependence criteria and e-cigarette cessation. Nicotine Tob Res. 2020;22(5):756–63.

    Article  CAS  Google Scholar 

  37. Foulds J, Veldheer S, Yingst J, et al. Development of a questionnaire for assessing dependence on electronic cigarettes among a large sample of ex-smoking e-cigarette users. Nicotine Tob Res. 2015;17(2):186–92. https://doi.org/10.1093/ntr/ntu204.

    Article  PubMed  Google Scholar 

  38. Browne M, Todd DG. Then and now: consumption and dependence in e-cigarette users who formerly smoked cigarettes. Addict Behav. 2018;76:113–21. https://doi.org/10.1016/j.addbeh.2017.07.034.

    Article  PubMed  Google Scholar 

  39. Mokkink LB, de Vet HC, Prinsen CA, Patrick DL, Alonso J, Bouter LM, et al. COSMIN risk of bias checklist for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1171–9. https://doi.org/10.1007/s11136-017-1765-4.

    Article  CAS  PubMed  Google Scholar 

  40. Piper ME, Baker TB, Benowitz NL, Smith SS, Jorenby DE. E-cigarette dependence measures in dual users: reliability and relations with dependence criteria and e-cigarette cessation. Nicotine Tob Res. 2020;22(5):756–63. https://doi.org/10.1093/ntr/ntz040.

    Article  CAS  PubMed  Google Scholar 

  41. Pénzes M, Foley KL, Balázs P, Urbán R. Intention to experiment with e-cigarettes in a cross-sectional survey of undergraduate university students in Hungary. Subst Use Misuse. 2016;51(9):1083–92.

    Article  Google Scholar 

  42. Boateng GO, Neilands TB, Frongillo EA, Melgar-Quinonez HR, Young SL. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front Public Health. 2018;6:149. https://doi.org/10.3389/fpubh.2018.00149.

  43. Oducado RM. Survey instrument validation rating scale. SSRN Electron J. 2020. https://doi.org/10.2139/ssrn.3789575.

  44. Polit DF, Beck CT. The content validity index: are you sure you know what's being reported? Critique and recommendations. Res Nurs Health. 2006;29(5):489–97. https://doi.org/10.1002/nur.20147.

    Article  PubMed  Google Scholar 

  45. DeVellis RF, Thorpe CT. Scale development : theory and applications. 3rd ed. SAGE; 2012.

  46. Nunnally JC, Bernstein IH. Psychometric theory. 3rd ed. New York: McGraw-Hill; 1994.

    Google Scholar 

  47. American Educational Research Association, American Psychological Association, & National Council on Measurement in Education (Eds.). Standards for educational and psychological testing: American Educational Research Association; 2014.

    Google Scholar 

Download references

Acknowledgements

None.

Funding

This was funded by HL138052 K12 Mentored Career Development Award to Eunhee Park by the National Heart, Lung, and Blood Institute (NHLBI).

Author information

Authors and Affiliations

  1. University at Buffalo, School of Nursing, 3435 Main St. University at Buffalo, Wende Hall, Buffalo, NY, 14214-8013, USA

    Eunhee Park, Misol Kwon, Thomas Chacko, Yanjun Zhou, Chiahui Chen, Chin-Shang Li & Yu-Ping Chang

  2. Roswell Park Comprehensive Cancer Center, Elm and Carlton Streets, Buffalo, NY, 14263, USA

    Maciej L. Goniewicz

Authors
  1. Eunhee Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Misol Kwon
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Chacko
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yanjun Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Chiahui Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maciej L. Goniewicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Chin-Shang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Yu-Ping Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

EP developed the systematic review protocol, conducted the review and wrote the first draft of the manuscript. MK, TC, YZ, and CC contributed to search articles, coding, analysis, interpretation, and writing. MK, CL, and YC contributed to the analysis and editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Eunhee Park.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Park, E., Kwon, M., Chacko, T. et al. Instruments to measure e-cigarette related constructs: a systematic review. BMC Public Health 22, 1135 (2022). https://doi.org/10.1186/s12889-022-13510-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-022-13510-4

Keywords

BMC Public Health

ISSN: 1471-2458

Contact us