Skip to main content

Development of a social distancing monitoring system in Republic of Korea: results of a modified Delphi process

Abstract

Background

Social distancing policies work in different ways and at different levels. In addition, various forms of monitoring systems have been implemented in different countries. However, there is an almost complete lack of specific monitoring system in Republic of Korea to effectively monitor social distancing measures compliance and outcome. This study aims to develop a monitoring system for social distancing measures compliance and outcome in Korea to evaluate and improve the implemented policy.

Methods

A draft monitoring system was developed after reviewing Korea’s social distancing measures (central and local government briefings) and checking available data about social distancing behavior. The modified Delphi process was used to evaluate the draft of the social distancing monitoring system. In total, 27 experts participated in the evaluation. The round 1 evaluation includes (1) commenting on the composition of the monitoring fields (open response), (2) monitoring indicators for each monitoring field (10-point Likert scale), and (3) commenting on the source of data used to develop the monitoring system (open response). In the round 2 evaluation, 55 indicators, excepting open responses, were re-evaluated.

Results

The response rate for the Delphi survey was 100% in both the first and second rounds. Of the 55 indicators, 1 indicator, which did not satisfy the quantitative criteria, was excluded. According to the experts’ open response comments, 15 indicators were excluded, as these indicators overlapped with other indicators or had little relevance to social distancing. Instead, 23 new indicators were added. Finally, 62 indicators were included with 12 available data sources. The monitoring system domain was divided into ‘social distancing measures state, social distancing measures compliance, social distancing outcome’.

Conclusions

This study is significant in that it is the first in Korea to develop a comprehensive monitoring system for checking if social distancing measures are being followed well, and is applicable to estimates utilizing data that are immediately available for each indicator. Furthermore, the developed monitoring system could be a reference for other countries that require the development of such systems to monitor social distancing.

Peer Review reports

Background

On March 11, 2020, the World Health Organization (WHO) declared the COVID-19 pandemic. In total, more than 118,300 people had been infected, and 4291 had died in 114 countries [1]. To overcome the situation, several pharmaceutical companies have been working to develop vaccines and treatments. Vaccination is taking place worldwide. However, the effects of vaccination for preventing the spread of viruses will take some time and Covid-19 virus variants continue to be reported [2]; thus, public health measures such as wearing masks and maintaining social distance would likely remain important [3, 4].

Social distancing is a public health measure that reduces the spread of viruses by reducing physical contact between people [5]. Social distancing measures are defined and implemented in various ways at different levels depending on the country [6]. The WHO has delineated guidelines for social distancing measures according to situations and locations (e.g., workplaces, schools, group gatherings, public places, and transportation) [7]. The European Centre for Display Prevention and Control (ECDC) defined and delineated social distancing according to the individual and group level [8].

As social distancing policies vary, various monitoring systems are available [9,10,11]. Social distancing worldwide is mainly monitored by non-profit organizations and understood as the presence or degree of implementation of the policy in different countries. In addition, various private organizations such as Google provide data to identify social distancing levels [12,13,14]. While some institutions presented “blocking and closing” and “healthcare systems” on a 100-point scale based on policy intensity [10], others evaluated whether implementing such affective measures would impact the socioeconomic environment, healthcare system capabilities, and government intervention [9]. A few studies have proven the efficacy of social distancing policy in preventing the spread of COVID-19 beyond monitoring [5, 6].

In Republic of Korea, several institutions have attempted to analyze and present only part of the social distancing behavior [15, 16]. The National Disaster and Safety Status Control Center has announced the results of analyzing mobile phone traffic, credit card use data, and public transportation utilization in the Seoul metropolitan area. Private research companies also reported indicators based on their regular COVID-19 risk perception survey results such as individuals’ social distancing practice rate, and restrictions on outdoor activities or eating out. However, the above results have a limitation that only a part of social distancing behavior was analyzed and it was only a one-time analysis. Therefore, social distancing measures compliance and outcome have not yet been sufficiently evaluated and no comprehensive monitoring system is in place.

This study aims to develop an appropriate monitoring system of social distancing measures compliance/outcome and indicators based on available data sources, identify the status of social distancing measures and behavior in Korea, and utilize them for policy evaluation and improvement in the future.

Methods

Study design

The modified Delphi process [17], consisting of two surveys and a three-step process, was used to reach consensus on the overall draft of the developed monitoring system for social distancing domains and indicators. The Delphi method is used to achieve consensus on a particular topic by having experts vote through multiple meetings and provide feedback [18, 19]. The difference between the modified and conventional Delphi process is that it can reduce the number of surveys from rounds 3 to 2 using structured questionnaires from the beginning, improving the convenience and efficiency of the survey. In addition, it was carried out online to increase the participation of experts from various fields and overcome the shortcoming of converging expert opinions according to their reputations and authorities. After evaluating the primary draft (checking scores and open responses) in the first round, we obtained consensus on open responses through adding and deleting indicators. In the second round, we reassessed the drafts (score checking only) to establish a final monitoring system. Figure 1 shows a flowchart of the study method.

Fig. 1
figure 1

Methodological workflow

Participant recruitment

Since social distancing is a policy [20] to prevent the spread of COVID-19, 27 experts were recruited based on two major conditions (Table 1). They were (1) 15 clinical healthcare professionals related to COVID-19(11 department of infectious diseases, 1 pulmonology, 1 psychiatry, 1 pediatrics, 1 emergency medicine), and (2) 12 healthcare system development and analysis specialists (7 preventive medicine, 2 social medicine, 2 occupation & environment Medicine, 1 health communication). The composition of experts included not only those knowledgeable in the COVID-19 virus, but also other infectious disease specialists, nursing professionals who work with routine infection control and patient management in hospitals, and public health and preventive medicine specialists with experience and knowledge in public health measures and policies. The majority of experts are policy-makers on public health measures, including social distancing in Korea. They were 1 Health Care Committee (National government level), 7 Distancing in Daily Life Committee (National government level), 13 COVID-19 Emergency Response Committee (Local government levels) and 6 Policy Committee of infectious disease society of Korea (Academic). Experts were invited to participate in the modified Delphi process via email. Those who agreed to participate were sent an email with an online link to conduct the first survey.

Table 1 Delphi Panellists (N = 27)

Initial dimensions and criteria of monitoring system

Only a few studies have investigated social distancing policies and behavior related to COVID-19 [21,22,23]. In addition, while various forms of social distancing monitoring systems are being established in other countries, there are several limitations to the use of a monitoring system in Korea. First, since different countries have implemented social distancing measures in various ways, it is not appropriate to include their measures indicators (e.g., complete lockdowns, restrictions on movement between cities) that have not been considered in Korea. Second, even though similar measures may have been implemented in Korea, these are still difficult to apply because the recommended level or detailed methods may differ. Because of these problems, we believed that a systematic review of the methodology and monitoring systems in other countries would not be suitable for this study. Therefore, this study reviewed social distancing measures at the national and local government levels and available data sources about social distancing measures compliance and outcome to develop a monitoring system draft. First, 207 regular COVID-19 infection briefings (from January 31 to October 7, 2020) by the National Disaster and Safety Status Control Center were reviewed to understand the overall social distancing measures in Korea. In addition, 1101 regular press briefings and press releases issued (from March 4 to October 7, 2020) by the Gyeonggi-do provincial government were reviewed to determine whether a special form of social distancing measures was implemented [24, 25]. Domestically available data sources about social distancing compliance and outcome included both public and private data; thus, 12 data sources including secondary data from surveys were selected. Furthermore, 55 indicators were selected according to available data sources in Korea, and the monitoring system domain was divided into five categories based on two levels (Table 2).

Data analysis

Round 1

The first round of the modified Delphi process was conducted for 6 days from July 9–14, 2020. In total, 27 experts from various fields received e-mails with information on the overall research, purpose of the modified Delphi process, detailed monitoring domains and indicators, and survey links for the appropriateness of the data source assessment. The survey consisted of (1) feedback on monitoring domains (open response), (2) evaluation of indicators by monitoring domain (10-point Likert scale: from “1=no need at all” to “10 = absolutely needed” and Open Responses), and (3) opinions regarding the data sources used (Open Response). Based on SMART criteria [26], each indicator was evaluated on a 10-point Likert scale so that quantitative results could be estimated, and the open responses were utilized to provide various opinions for the overall evaluation parts. The ‘SMART’ criteria (Specific, Measurable, Attributable, Reliable, and Timely) is often used in developing indicators for monitoring and evaluation. Those who had not yet completed the survey were reminded to do so 3 days before the deadline. The response rate for the first Delphi survey was 100%.

Between rounds

Data from round 1 were collected and analyzed. For the evaluation of the indicators, the mean, median, and standard deviations were calculated. SAS 9.3 was used for the statistical analysis. The second round of the survey was conducted for indicators with an average score of 7 or higher. All open responses were submitted to the entire research team prior to the initiation of the second round of the study, including recommendations for changes to other indicators or comments regarding the addition or exclusion of indicators, as well as to the purposes and scope of the monitoring domains or opinions about data sources. In response, our research team members discussed whether to reflect on the study based on accessibility, feasibility, timeliness, etc. Thus, except for the open response questions, the second round of evaluation for indicators in each monitoring domain was conducted using a 10-point Likert scale. In the second round, the results from the first round (average and median) were shared with the experts by providing the survey link via e-mail.

Round 2

The second round of the modified Delphi process lasted 6 days from July 17–22, 2020. The survey was written using Google Forms and distributed to the same experts (n = 27). An assessment of the same 55 indicators by monitoring domain as in the first round was performed in the second round. The mean and median values were provided for each indicator and evaluated on a 10-point Likert scale using the same SMART criteria as in the first round. Those who did not complete the survey were reminded to do so 1 day before the deadline. The response rate for the second Delphi survey was 100%.

Priority

The most important principles for developing a monitoring system were (1) policy linkage and (2) measurability (application in reality), which were reflected in the open response comments. In addition, the indicators were chosen based on an average score of 7 or higher and included any other indicators from the discussion among research team members, which may have been greatly affected by the social distancing policy.

Results

Round 1 – based on monitoring indicator evaluations

In total, 55 potential indicators were included in the first round of voting (Table 2), and all indicators excepting “public transportation restrictions” (6.93 points), “residential mobility rate” (6.89 points), and “high expenses for daily necessities experience rate” (6.85 points) were evaluated as higher than 7 points. Thus, all indicators were evaluated in the second round. However, apart from the calculated scores, the exclusion, segmentation, and addition of indicators were suggested in the experts’ open responses. Additional discussions to reflect on their opinions were conducted. Open responses will be addressed in the discussion on these later in the paper.

Table 2 Results of the assessments of indicators in the Delphi process

Round 2 – based on monitoring indicator evaluations

Similar to the first round of evaluation, an assessment of the same 55 indicators by monitoring domain was conducted in the second round (Table 2). For each indicator, the mean and median scores were obtained, which made a difference from the previous round. As with the results of the first round, two indicators were considered less important: “residential mobility rate” (6.85 points) and “high expenses for daily necessities experience rate” (6.77 points), which scored less than 7 points on average. However, in the open responses, only “residential mobility rate” was excluded from the final monitoring system since the importance of “high expenses for daily necessities experience rate” was emphasized. In addition, “restriction of public transportation” scored 7.04 points in the second round, highlighting its importance. Although the “restriction on domestic mobility” indicator was considered important in the first round (7.11 points), it was evaluated as 6.92 points in the second round. Both indicators were included in the direct policy level domain, which was excluded from the finalized monitoring system because the monitoring domain was modified based on the discussion of open responses. A detailed explanation is provided in the section discussing the open responses.

Discussion about open-ended responses – monitoring domain

Most experts stated that the monitoring system domain was not clearly classified. The issue was raised that if monitoring system indicators are distinguished as being direct or indirect, some may overlap. Thus, it was suggested that the monitoring domains be delineated as ‘social distancing measures state, social distancing measures compliance, social distancing outcome’.

The indicators related to a state of national social distancing measures were included in the social distancing measures state. Unlike existing public health policies, Covid19-related policies have a short cycle of change. As the details of social distancing measures in South Korea are constantly changing, it is necessary to monitor changes of the measures.

The indicators affected by social distancing measures in action were included in the social distancing measures compliance. A key domain of the monitoring system is the ‘social distancing measures compliance’ domain. This domain consists of indicators that show how well people are being complied with social distancing.

Furthermore, the indicators related to the consequences of the social distancing measures and compliance (or behavior) in the social distancing outcome. This domain provides the information about the result of social distancing measures and compliance, and becomes evidence to control the level (details) of measures at the same time.

Therefore, the research team removed the direct/indirect level of domain 1. The research team revised the policy level of domain 2 to the “Social distancing measures state” domain, and the community/individual/indirect level of domain 2 to the “Social distancing measures compliance” domain. In addition, as the consequences of social distancing measures and behavior, the indicators for results were added. As such, the indicators were categorized as ‘social distancing measures state, social distancing measures compliance, social distancing outcome’ domains (Fig. 2).

Fig. 2
figure 2

Modified monitoring domains based on the open-ended responses in the Delphi process

Discussion about open-ended responses – indicators

In addition to the index-specific evaluation scores, the necessary indicators were added and conventional indicators subdivided based on the experts’ feedback. Some indicators were combined or excluded according to the revised monitoring domains. According to the experts’ open response comments, 15 indicators were excluded, as these indicators overlapped with other indicators or had little relevance to social distancing. Instead, 23 new indicators were added.

Direct indicators: policy level → social distancing measures state

Nine monitoring indicators for direct indicators in the policy level domain were suggested by the team referring to the social distancing policies implemented worldwide. The experts provided their opinions about the nine monitoring indicators. Since the chosen indicators were based on the contexts of other countries, they have a limitation in that it is difficult to adequately monitor social distancing measures in Korea. Thus, indicators were changed to a state of social distancing measures in Korea. Therefore, measures such as the prohibition of events (e.g., general community meetings, sports events) and closure of institutions (e.g., multi-facilities, schools, large companies), and continuously emphasized personal quarantine guidelines were included as indicators in this domain.

Direct indicators: community level → social distancing measures compliance

Among the direct indicators presented by the researchers, four indicators were selected, displaying the degree of social distancing behavior at the community (group) level. The experts’ opinions were reflected as follows: Most experts suggested adding more indicators. “Whether to participate in online church worship services” and “the rate of conducting online lectures” (schools, private academic classes, etc.) were added based on representativeness and validation. There were recommendations to add “monitoring rate for employees or facility visitors,” “recording of guest book or QR code utilization rate,” and “temperature measurement rate of visitors,” which replaced the “facility measures performance rate” presented in the early version. Some suggested adding an “non-contact meeting conducting rate” indicator, but this was not reflected in the monitoring system since it was not possible to obtain data sources for the measurement.

Direct indicators: individual level → social distancing measures compliance

Among the direct indicators presented by the researchers, 38 displayed the degree of social distancing behavior at the individual level. The experts’ opinions were reflected as follows: It was suggested that a representative indicator be selected. Overall, the initial versions of the monitoring indicators and additional indicators based on the experts’ comments were collected to determine representative indicators (e.g., Since “Hand washing practice rate” encompasses “Use rate of hand sanitizers or soap”, “Hand washing practice rate” was selected as a representative indicator. For the same reason, “Facial mask wearing rate” was chosen instead of “Indoor facial mask wearing rate”.). Specifically, indicators such as “Change rate of movement at public transportation stops”, “residential mobility rate” and “purpose-specific transportation utilization rate” were excluded as monitoring indicators since they were not clearly defined because of data source limitations. In particular, the “residential mobility rate” was scored below 7 points in the evaluation.

For additional indicators, considering the definition and purpose of the process/output domain and indicator validity, indicators such as “rate of commuting with own car”, “Change rate of average travel distance”, “frequency of daily ventilation at home”, “Change rate of personal hygiene products” and “Perception of social distancing” were added. There were suggestions to add the measurement indicator “use of CCTV images and infrared sensors,” but this was difficult to reflect in the monitoring system because there are currently no available data sources.

Indirect indicators: indirect level → social distancing measures compliance

The indirect level of monitoring indicators suggested by the researchers displayed eight indicators that could not be considered directly affected by social distancing measures, but could be indirectly impacted. However, because of the vague domain definition of indirect effects, there were more opinions on domain modifications than opinions on indicators within the domain. In conclusion, all indicators within the indirect level were included in the domain of ‘Social distancing measures compliance’.

Social distancing outcome

Reflecting the opinion that the results of social distancing measures and compliance (or behavior changes) should be included in the monitoring system, this domain was added to the monitoring system. The 5 indicators related to controlling the stages of social distancing measures in Korea were added.

Discussion about open-ended responses – data source

There were many negative opinions regarding the data sources. Most opinions revealed that currently available data sources have problems of timeliness, feasibility, and representativeness. However, these data sources were not excluded, as they were considered meaningful and could be immediately applicable to monitoring social distance policy. Thus, the monitoring system developed in this study was based on (1) the available data source at this stage and (2) data source accessible at the metropolitan level.

Finally, 12 data sources were included: 3 data sources with characteristics of primary data (Saramin survey, COVID-19 perception survey by Korea Research, Reports (current situation) by the National Disaster and Safety Status Control Center), and 9 data sources with secondary data (Google Mobility, Floating populations and phone calls provided by SKT in Tmap data, Floating populations data provided by KT, Credit card Sales data provided by Shinhan Card, Credit card Sales data provided by BC Card, Transportation data provided by Gyonggido Traffic control center, Korean Travel Survey, Korea Tourism Statistics, Air transport statistics). Table 3 provides details on each data source.

Table 3 Data source explanations

Final monitoring system

The final monitoring system was delineated as the ‘social distancing measures state, social distancing measures compliance, social distancing outcome’, and included 62 indicators (Fig. 3). Detailed definitions are provided in Additional file 1.

Fig. 3
figure 3

Final monitoring system

The social distancing measures state is a domain to check the status of changes in domestic social distancing measures. According to the social distancing stage in Korea, the measures taken for events (general gatherings, meetings, social events, and sports events) and in institutions (multi-facilities/schools/private companies), and continuously emphasized personal preventive practices were included as indicators. In other words, the indicators in this domain are the same as the measures of social distancing (not implementation or compliance).

The social distancing measures compliance is created during the process influenced by social distancing measures. This means we can see how well social distancing measures are being followed in this domain. This is a key domain of our monitoring system. It was divided into eight scopes including daily movement, domestic and international movement, personal hygiene and quarantine, school and workplace, personal daily life, consumption, risk perceptions, and facility measures, and included relevant indicators for each domain. In the daily movement scope, the rates for the change in floating population, restrictions on public transportation, change in types of transportation use, commuting, retail store and leisure facility movement, grocery store and pharmacy movement, movement in parks, and commuting with own car were included. In the national domestic and international movement scope, the change rates of the use of domestic flights, use of international flights, outbound travel by locals, arrival of foreigners, and average travel distance were included. In the personal hygiene and quarantine scope, the rates of hand washing practice, wearing of face masks, frequency of ventilation in the home, and practicing social distancing of 2 m between individuals were included. The school and workplace scope included the rates of implementation of telecommuting, experience of working even when sick, working hours adjustment, conducting online meetings, and online lectures. The personal and daily life scope included the rates of controlling one’s going out, cancellation of meetings, cancellation of hospital visits, restrictions on using multi-use facilities, domestic travel experience, changes in tourist visits to hotspots, and participation in online church worship services. The consumption scope included the change rates of private academy sales, number of sports/culture/leisure facilities sales, number of travel/transportation sales, number of restaurant/entertainment sales, non-contact purchases (home shopping, Internet shopping, etc.), food delivery, experience of purchasing necessities in bulk, and total sales of personal hygiene products (face masks, hand sanitizer). The (risk) perception scope included the degree of change in daily life and degree of perception of social distancing measures. Measures for facilities scope included the rates of implementation of these measures and closure of public areas.

The social distancing outcome is the consequence of social distancing measures and compliance (or behavior change) related to controlling the stages of these measures. Indicators such as number of daily confirmed cases, rate of unknown infection routes, status or number of mass outbreaks, rate of management in quarantine areas and number of new self-quarantines were included.

Discussion

This study developed a social distancing monitoring system based on social distancing measures and available data about social distancing behavior in the country. The modified Delphi process was used to elicit the participation of experts from various fields such as infectious diseases, nursing, and health sciences. In total, 62 indicators were included in ‘social distancing measures state, social distancing measures compliance, social distancing outcome’ domains.

It is possible to create a social distancing monitoring system, by referring to the process and method of developing the monitoring system in this study. First, using national policy briefings and materials, list up social distancing measures. The list can be organized around measures that are taken on an ongoing basis. Second, identify available data to confirm compliance with measures or check whether measures are taken. The list is organized again based on available data, then a draft of monitoring system can be prepared. After that, the final monitoring system can be derived by refining the draft of monitoring system through Delphi process. And when referring to this study, the following must be considered.

In this study, a monitoring system was developed in consideration of Korea’s social distancing policy and environment. Since different countries have implemented social distancing policy in various ways, it is not appropriate to include their policy indicators (e.g., complete lockdowns, restrictions on movement between cities) that have not been considered in Korea. Even though similar policies may have been implemented in Korea, these are still difficult to apply because the recommended level or detailed methods may differ. For example, some countries implemented closure of educational institutions and others did not. Among countries that implemented closure of educational institutions, some countries have closed all schools, some countries have closed only kindergartens∙primary∙secondary schools, and some countries have implemented part-time school attendance. In addition to school closures, there are countless social distancing measures, and the indicators to be checked also vary depending on whether or not the measures are taken and the intensity of the measures. Thus, the social distancing monitoring system should be developed by reflecting the environment and policies implemented in each country.

The domain-specific suggestions of the monitoring system presented in this study are as follows: Currently, the social distancing measures state simply monitors changes of the social distancing measure status. In the future, a policy evaluation study is needed to evaluate the feasibility and appropriateness of domestic social distancing stages. Furthermore, it will be necessary to develop tools to compare the levels of social distancing policies in Korea and other countries. For the social distancing measures compliance, a large number of indicators were provided by the surveys. However, various national monitoring methods are still needed to measure practice rates in a “realistic” or “objective” way, such as using CCTV to monitor hand washing and the wearing of masks, attaching infrared sensors on faucets, conducting observation surveys, and using video conference equipment in public conference rooms. In addition, it is necessary to develop and support measurement methods for indicators that cannot be added because of the lack of proper data sources. The social distancing outcome should clarify the definition and formula of the result indicators of COVID-19, which is intermittently produced by the National Disaster and Safety Status Control Center, so that the actual effectiveness of social distancing policy can be accurately determined.

Various types of environmental and institutional support must be presented for the monitoring systems developed in this study to work realistically. Above all, a national social distancing monitoring system should be established that combines all policies from each government department. It is necessary to design systematic surveys to assess the effectiveness of social distancing behavior in a more objective way, rather than over the long term. And a clear definition of social distancing measures is needed. The current social distancing policies in Korea include national/institutional-level measures (e.g., business restrictions on high-risk facilities, and school regulations, etc.) and compliance with individual-level quarantine guidelines. While policies have been constantly changing, no clear definition has been provided. So, confusion in the community continues in line with changing guidelines, as the definition of social distancing policy is not clear [27, 28]. Therefore, it is essential to draw social consensus on the definition of social distancing policies, and share them in a more consistent way.

Although the vaccine for COVID-19 has been developed and used, it has been announced that it will still take a long time to achieve the desired expectations [3, 4]. Social distancing measures to reduce the transmission of COVID-19 are being implemented worldwide, but research is lacking on how the policy affects the prevalence and mortality of COVID-19 [29]. It is very important to assess and monitor the effectiveness of social distancing policy not only for COVID-19 but also for an effective response to possible new infectious diseases in the future [29].

Strengths and weaknesses

The limitations of this study are as follows: First, a guidance on using the framework as a tracking tool is not provided in this study. Second, this study developed monitoring indicators based on currently available data sources, excluding infinite measurement methods and indicators that can measure social distancing behavior. Third, clinical healthcare professionals related to COVID-19 and healthcare system development and analysis specialists were only invited to participate in the modified Delphi process. In Korea, clinical healthcare professionals are the main discussion group (or the policy decision-making group) for all public health measures, including social distancing measures. It is a characteristic feature of the public health system in Korea.

Nevertheless, this study is meaningful in that it developed a comprehensive monitoring system for compliance of social distancing for the first time in Korea and provided available data sources by indicator. In addition, based on the social and economic impact created by social distancing [6], we developed indicators that distinguish domains by field and quantitatively understand the degree of social distancing practice in each. Since the COVID-19 pandemic is prolonged, not only in Korea but also in other countries, we can refer to our developed monitoring system for decision-making regarding the timing and intensity of various infectious disease response policies.

Conclusion

For the first time in Korea, this study developed a comprehensive and realistic monitoring system for checking if social distancing measures are being followed well by providing data sources that can be immediately identified by indicators. This study will be available not only in Korea but also in other countries that need to develop monitoring systems for social distancing. However, the results of this study should be applied in accordance with each country’s measures and contexts (environments), and the status continuously reported, analyzed, and updated, even after the monitoring system is established. Subsequently, efforts are required to continuously monitor the status of the system and utilize it to improve social distancing measures.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

WHO:

World health organization

ECDC:

European centre for display prevention and control

References

  1. World Health Organization (WHO). WHO Director-General's opening remarks at the media briefing on COVID-19-11 March 2020. In: Geneva, Switzerland; 2020. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19%2D%2D-11-march-2020. [cited 18 Jun 2021].

  2. World Health Organization (WHO). The effects of virus variants on COVID-19 vaccines. 2021. Available from: https://www.who.int/news-room/feature-stories/detail/the-effects-of-virus-variants-on-covid-19-vaccines?gclid=CjwKCAjwiLGGBhAqEiwAgq3q_q7rWj6iCvZ_vrGOGBAvMM1YRPYRRoUY-4eFjYO4ahL29Ix8ivz1yhoCH1oQAvD_BwE. [cited 18 Jun 2021].

  3. Patel MD, Rosenstrom E, Ivy JS, et al. Association of Simulated COVID-19 vaccination and nonpharmaceutical interventions with infections, hospitalizations, and mortality. JAMA Netw Open. 2021;4(6):e2110782.

    Article  Google Scholar 

  4. World Health Organization (WHO). Behavioural considerations for acceptance and uptake of COVID-19 vaccines: WHO technical advisory group on behavioural insights and sciences for health. 2020. Available from: https://www.who.int/publications/i/item/9789240016927. [cited 18 Jun 2021].

  5. Ahmed F, Zviedrite N, Uzicanin A. Effectiveness of workplace social distancing measures in reducing influenza transmission: a systematic review. BMC Public Health. 2018;18(1):518.

    Article  Google Scholar 

  6. Teslya A, Pham TM, Godijk NG, Kretzschmar ME, Bootsma MCJ, Rozhnova G. Impact of self-imposed prevention measures and short-term government-imposed social distancing on mitigating and delaying a COVID-19 epidemic: a modelling study. PLoS Med. 2020;17(7):e1003166.

    Article  CAS  Google Scholar 

  7. World Health Organization (WHO). Overview of public health and social measures in the context of COVID-19: interim guidance. 2020. Available from: https://www.who.int/publications/i/item/overview-of-public-health-and-social-measures-in-the-context-of-covid-19. [cited 18 Jun 2021].

    Google Scholar 

  8. European Centre for Disease Prevention and Control (ECDC). Considerations relating to social distancing measures in response to COVID-19 – second update. 2020. Available from: https://www.ecdc.europa.eu/en/publications-data/considerations-relating-social-distancing-measures-response-covid-19-second. [cited 18 Jun 2021].

    Google Scholar 

  9. Assessment Capacities Project (ACAPS). COVID-19 Government measures. 2020. Available from: https://www.acaps.org/projects/covid19/data. [cited 18 Jun 2021].

    Google Scholar 

  10. Blavatnik School of Government (BSG). Coronavirus Government Response Tracker. 2020. Available from: https://www.bsg.ox.ac.uk/research/research-projects/coronavirus-government-response-tracker. [cited 18 Jun 2021].

    Google Scholar 

  11. PoliMap. A global repository of policy responses to tackle COVID-19. 2020. Available from: https://polimap.org/. [cited 18 Jun 2021].

  12. Google. COVID-19 Community Mobility Reports - Google. 2020. Available from: https://www.google.com/covid19/mobility/. [cited 18 Jun 2021].

  13. Deloitte. Predicting the post-pandemic rebound. 2020. Available from: https://www2.deloitte.com/ca/en/pages/about-deloitte/articles/covid-dashboard.html. [cited 18 Jun 2021].

    Google Scholar 

  14. Maryland Transportation Institute. COVID-19 Impact Analysis Platform. 2020. Available from: https://data.covid.umd.edu/. [cited 18 Jun 2021].

    Google Scholar 

  15. Central Disaster Management Headquarters. Regular COVID-19 infection briefings-5 Oct 2020. In: Seoul, South Korea; 2020. Available from: http://www.mohw.go.kr/upload/viewer/skin/doc.html?fn=1601863697249_20201005110817.hwp&rs=/upload/viewer/result/202102/. [cited 18 Jun 2021].

  16. Hankook Research. Dashboard on regular COVID-19 risk perception survey results. 2020. Available from: https://public.tableau.com/profile/lee.donghan#!/vizhome/HRC_covid19_survey_dashboard/Intro. [cited 18 Jun 2021].

    Google Scholar 

  17. Murry JW Jr, Hammons JO. Delphi: a versatile methodology for conducting qualitative research. Rev High Educ. 1995;18(4):423–36.

    Article  Google Scholar 

  18. Fink A, Kosecoff J, Chassin M, Brook RH. Consensus methods: characteristics and guidelines for use. Am J Public Health. 1984;74(9):979–83.

    Article  CAS  Google Scholar 

  19. de Meyrick J. The Delphi method and health research. Health Educ. 2003;103(1):7–16.

    Article  Google Scholar 

  20. Kissler SM, Tedijanto C, Lipsitch M, Grad Y. Social distancing strategies for curbing the COVID-19 epidemic. medRxiv. 2020; Available from: https://www.medrxiv.org/content/medrxiv/early/2020/03/24/2020.03.22.20041079.full.pdf.

  21. Chen SL, Yen AM, Lai CC, Hsu CY, Chan CC, Chen TH. An index for lifting social distancing during the COVID-19 pandemic: algorithm recommendation for lifting social distancing. J Med Internet Res. 2020;22(9):e22469.

    Article  Google Scholar 

  22. Gibson L, Rush D. Novel coronavirus in Cape Town informal settlements: feasibility of using informal dwelling outlines to identify high risk areas for COVID-19 transmission from a social distancing perspective. JMIR Public Health Surveill. 2020;6(2):e18844.

    Article  Google Scholar 

  23. Nilsen P, Seing I, Ericsson C, et al. Implementing social distancing policy measures in the battle against the coronavirus: protocol of a comparative study of Denmark and Sweden. Implement Sci Commun. 2020;1:77.

    Article  Google Scholar 

  24. Central Disaster Management Headquarters. Regular COVID-19 infection briefings. In: Seoul, South Korea; 2020. Available from: http://www.cdc.go.kr/board/board.es?mid=a20501000000&bid=0015. [cited 18 Jun 2021].

  25. Gyeonggi-do Provincial Government. Regular COVID-19 infection briefings. In: Suwon, South Korea; 2020. Available from: https://www.gg.go.kr/bbs/board.do?bsIdx=724&menuId=2910. [cited 18 Jun 2021].

  26. Doran GT. There’s a SMART way to write management’s goals and objectives. Manag Rev. 1981;70(11):35–6.

    Google Scholar 

  27. Park JW. Audiences confused over different COVID-19 measures between theaters. The Korea Times [Internet]. 06 Dec 2020 [cited 28 Jan 2022]: Entertainment & Arts. Available from: http://www.koreatimes.co.kr/www/art/2020/12/398_300438.html

  28. Lim JH. '8 people? 4 people?' Due to frequent changes of social distancing guideline... Citizens "I don't know anymore.". Daily Newspaper. 02 Jul 2021 [cited 28 Jan 2022]: Society. Available from: https://news.imaeil.com/page/view/2021072216383196529

  29. McGrail DJ, Dai J, McAndrews KM, Kalluri R. Enacting national social distancing policies corresponds with dramatic reduction in COVID19 infection rates. PLoS One. 2020;15(7):e0236619.

    Article  CAS  Google Scholar 

Download references

Acknowledgments

Not Applicable.

Funding

The work leading to these results was part of the study “Development and analysis of a Social Distancing Monitoring System”. Local government (Gyeonggi-do Provincial Government) budget was used to conduct the research. The funders had no role in the design of the study and collection, management, analysis, and interpretation of the data and in writing the manuscript. The authors and their contributions to the manuscript are independent of the funder.

Author information

Authors and Affiliations

Authors

Contributions

SI, JO, SO, HY, designed the study. SI, JO, SO contributed to the data acquisition and analysis. All authors contributed to the interpretation of the analysis results. SI, JO drafted the publication, and managed subsequent revisions. All authors reviewed and approved the final manuscript. HY led the entire study process.

Corresponding author

Correspondence to Heeyoung Lee.

Ethics declarations

Ethics approval and consent to participate

Ethical approval for the project was provided by Seoul National University Bundang Hospital Institutional Review Board (IRB No: n-2106688604). The need for consent was waived by the IRB at Seoul National University Bundang Hospital Institutional Review Board because the study was deemed not human subjects research as defined by HRPP SOP ver 3.9. The study was considered public health practice. Scoring and discussions during the Delphi process was anonymous. All methods were carried out in accordance with relevant guidelines and regulations.

Consent for publication

Not Applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seo, S., Han, JO., Shin, S. et al. Development of a social distancing monitoring system in Republic of Korea: results of a modified Delphi process. BMC Public Health 22, 860 (2022). https://doi.org/10.1186/s12889-022-13277-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12889-022-13277-8

Keywords