Graphical abstract

Introduction

Risk assessments provide information essential for managing the adverse effects of public health risks [1] and must be completed promptly to enable actions such as prevention and response to disease outbreak. Some countries conducted risk assessments even during the coronavirus disease 2019 (COVID-19) pandemic [2,3]. In the Republic of Korea, rapid risk assessments (RRAs) were carried out to inform strategic responses [4]. In the early stages of the outbreak, when cases were primarily concentrated in China, public health authorities in various countries conducted RRAs to evaluate the potential threat of the outbreak spreading from China to other countries. These assessments were based on limited information about the virus’s characteristics and transmission routes, aiming to control the spread and prevent new outbreaks [5].

During the COVID-19 public health emergency, which lasted from January 30, 2020, to May 5, 2023 [6], the pattern of the outbreak evolved due to the emergence of new variants. This change also affected risk assessment practices. For instance, the Robert Koch Institute developed risk levels and recommendations based on a qualitative evaluation of the latest global data, the epidemiological situation of COVID-19 in Germany, and the availability of preventive and treatment options [7]. Additionally, the United Kingdom Health Security Agency (UK HSA) employed impact indicators, such as the size and severity of outbreaks, to determine the alert levels for UK chief medical officers [8].

Owing to changes in outbreak patterns and ongoing outbreaks, the Republic of Korea gradually eased social distancing measures in November 2021, introducing a policy termed “Gradual Recovery to a New Normal.” Additionally, weekly COVID-19 risk assessments were proposed to accurately monitor changes in COVID-19 outbreaks following the implementation of this new policy. These weekly COVID-19 risk assessments, which were conducted starting in November 2021, utilized different methods and indicators compared to the initial RRAs.

Some countries also modified their risk assessments to better reflect their specific needs. In the United States, indicators such as the number of newly confirmed cases per week, the number of newly hospitalized patients per week, and the percentage of hospital beds occupied by COVID-19 patients were used to inform community levels [9]. The UK HSA set the alert level by combining expert judgment and risk assessment with more quantifiable indicators and thresholds [8]. Utilizing available information to understand and communicate the situation is a crucial method in responding to COVID-19.

In this context, this study aimed to summarize the findings from the weekly COVID-19 risk assessments conducted by the Korea Disease Control and Prevention Agency (KDCA). It also sought to explore the relationship between the outcomes of these assessments and various factors, including incidence rates and contextual elements. Contextual factors encompass indicators that are considered in risk evaluations, taking into account all influences on risk, such as infrastructure, cultural practices, and beliefs [1].

Materials and Methods

The data analyzed in this study were sourced from KDCA press releases with information on the weekly risk assessments conducted from November 2021 to May 2022. Additionally, raw data and COVID-19 risk assessment indicator data were obtained from the National Infectious Disease Surveillance System website. Key indicators such as the “ratio of the incidence rate to maximum medical capacity” and the “availability of intensive care unit (ICU)” were crucial for evaluating changes in the COVID-19 risk assessments. Expert opinions were subsequently utilized to determine the derived risk levels. The weekly COVID-19 risk assessment was categorized into 5 levels: very low, low, moderate, high, and very high.

The weekly COVID-19 risk assessment incorporated 36 indicators across 4 categories: COVID-19 response capacity, incidence, vaccination, and diagnostic testing, as detailed in Table S1. The response capacity category included indicators such as ICU availability, hospitalization of severe cases, the ratio of incidence rate to medical capacity, and the availability of various treatment facilities for confirmed cases. Additional indicators in this category were the percentage of patients treated at home, the daily average number of patients treated at home per week, and the percentage managed under quarantine. The incidence category comprised indicators including death cases, fatality rate, newly severe cases, severity rate, number of hospital admissions, daily average number of confirmed cases, newly confirmed cases, percentage of unvaccinated individuals (including those with only the first vaccination), the time-varying reproduction number (Rt) [10], and the weekly percentage of positive test results (Table 1). Indicators in the vaccination category encompassed the percentage of individuals fully vaccinated, the percentage receiving a booster shot, the percentage of those aged 60 years and older who are fully vaccinated, the percentage of those aged 60 years and older receiving a booster, the rate of fourth vaccinations in those aged 60 years and older, and the vaccination’s effectiveness against infection, severity, and death. For the diagnostic testing category, indicators included the omicron variant detection rate and the BA.2 detection rate (Table S1).

Among the above indicators, the ratio of the incidence rate to medical capacity was used to assess medical capacity based on the number of hospital beds allocated for patients with severe diseases. This capacity was estimated by considering the number of patients that the available hospital beds (ICU and step-down unit) could accommodate, factoring in the proportion of cases that progress to severe disease and the average duration of hospital stays. The ratio of the incidence rate to medical capacity was determined by dividing the weekly average of daily confirmed cases by the estimated medical capacity (Table S1).

Community mobility, the risk perception rate, and the Oxford stringency index were analyzed alongside COVID-19 risk assessment indicators to explore the social and policy changes prompted by the COVID-19 pandemic [11].

The community mobility rate, as reported in the Google Community Mobility Report, reflects the percentage change in activity at retail and leisure facilities compared to baseline days. These baseline days were established by taking the median of data collected daily over a 5-week period from January 3 to February 6, 2020. Retail and leisure facilities include venues such as restaurants, cafes, shopping centers, amusement parks, museums, libraries, and movie theaters. The risk perception rate was derived from responses to the survey question regarding the “perception of the seriousness of COVID-19 spreading in South Korea,” conducted by Hankook Research. This web-based survey was distributed biweekly via mobile phone or email to a sample of 1,000 male and female participants aged 18 years or older. The sample was proportionally allocated by region, sex, and age. Participants could choose from 5 responses: not serious, not so serious, mederate, serious, and very serious. The percentage of responses categorized as “very serious” was used to calculate the risk perception rate in this study. Additionally, to address gaps in the data due to the biweekly frequency of the survey, weeks without surveys were estimated using linear interpolation to fill in the missing data.

The Oxford stringency index was used as an indicator to evaluate the characteristics of policies. This index measures the strictness of each country's policy responses, including public health and social intervention measures, to combat the spread of COVID-19. It is calculated by taking a simple average of the converted values of 9 constituent indicators on a ranking scale, covering containment, closure, and health system policies, each scored out of 100 points. For this study, the index values reported on Saturdays during the COVID-19 risk assessment period were used.

To examine the distributions of the parameters analyzed in the weekly COVID-19 risk assessments in the Republic of Korea, the results for each indicator were presented as means and ranges. Linear graphs were utilized to compare the weekly COVID-19 risk levels with corresponding changes in COVID-19 incidence, public perception, and policy adjustments during the same timeframe. Additionally, correlation analysis was performed among various metrics: newly confirmed cases per week, the number of confirmed cases aged over 60 years per week, confirmed cases under 18 years per week, weekly hospitalizations, and weekly severe disease cases. Further correlation analyses included the daily average number of hospitalized patients with severe disease, weekly death counts, community mobility rates, risk perception rates, and the Oxford stringency index, categorized by weekly risk levels. The risk level was adjusted for each week to 1 week prior, 1 week subsequent, and 2 weeks subsequent to determine the effect of elapsed time. Correlations were assessed using Spearman correlation analysis.

Ethics Approval

This study was approved by the Institutional Review Board of the KDCA (No: 2023-02-05PE-A) and performed in accordance with the principles of the Declaration of Helsinki.

Results

The risk assessments analyzed in this study spanned 30 weeks, from November 2021 to May 2022. They encompassed data on national risk levels and risk levels in non-capital areas, which varied from very low to very high. In capital areas, risks ranged from low to very high (Table 2). The review of risk assessment indicators primarily focused on the incidence rate and the countermeasures implemented during that period.

The weekly COVID-19 risk level escalated from “very low” in the first week of November 2021 to “very high” for 6 consecutive weeks, spanning from the fourth week of November to the fifth week of December. The risk level then consistently remained high before it began to decline gradually from the first to the fifth week of March 2022 (Figure 1). During the time when the delta and omicron variants were dominant, several indicators exhibited similar trends. These included the number of newly confirmed cases, the number of new cases in individuals aged over 60, the number of new cases in individuals under 18, the number of hospitalized patients, the number of patients hospitalized with severe disease, the number of newly confirmed severe cases, the number of deaths, and the risk perception rate. These indicators typically peaked between 2 and 5 weeks after the risk level reached “very high” (Figures 1–3). Additionally, when the risk assessment was “very high,” the community mobility rate decreased in March 2022, dropping to 0% in December 2021 when the delta variant was prevalent (Figure 4). The Oxford stringency index remained unchanged until the second week of April 2022, after which it decreased and stabilized (Figure 5).

An analysis of the weekly COVID-19 risk level showed correlations with several variables: the Rt, hospital admissions, newly confirmed severe cases, severe cases in hospitals, death cases, community mobility rate, and risk perception rate. The community mobility rate showed a negative correlation, while the Rt, hospital admissions, newly confirmed severe cases, severe cases in hospitals, death cases, and risk perception rate all exhibited positive correlations. Notably, the risk perception rate (r=0.841) and hospital admissions (r=0.690) were significantly positively correlated (p<0.05) (Table 3).

The weekly COVID-19 risk level from “1-week lead” showed a significant positive correlation with the Rt, hospital admissions, newly severe cases, and risk perception. In addition, the risk perception rate showed a strong positive correlation (r=0.678, p<0.05) (Table 3).

The weekly COVID-19 risk level, based on the 1-week lagged value, demonstrated significant correlations with several indicators: hospital admissions, newly confirmed severe cases, severe cases in hospitals, death cases, community mobility rate, and risk perception rate (p<0.05). The community mobility rate was negatively correlated (r=–0.537), while the other significant indicators exhibited positive correlations. Specifically, the risk perception rate (r=0.840), severe cases in hospitals (r=0.740), and newly confirmed severe cases (r=0.693) all showed strong positive correlations (p<0.05) (Table 3).

The weekly COVID-19 risk level from the 2-week lagged value showed significant associations with the same indicators as those observed with the 1-week lagged value (p<0.05). Notably, during this period, the strongest correlation occurred among severe cases in hospitals (r=0.769, p<0.05) (Table 3).

Discussion

Between November 2021 and May 2022, the Delta and Omicron-type BA.1 and BA.2 variants became prevalent in the Republic of Korea. The incidence of severe disease escalated with the prevalence of the Delta variant. Additionally, the prevalence of the Omicron variant led to a hundredfold increase in outbreak size and exhibited a distinct pattern [12,13]. During this period, the KDCA conducted weekly COVID-19 risk assessments, adjusting the indicators to account for the emergence of mutant viruses and shifts in national disease control policies. Consequently, the number of indicators expanded to 47.

Since November 2021, the KDCA has been conducting weekly COVID-19 risk assessments. The results, along with the 4-week values of the indicators used, are shared through briefings and regular press releases. These indicators objectively reflect the current situation, informing stakeholders of necessary measures and alerting the public to any changes in conditions. However, relying solely on the fluctuations of these indicators to determine the weekly COVID-19 risk level does not suffice for making accurate evaluations. Therefore, the “ratio of the incidence rate to medical capacity” and the “availability of hospital beds for severe patients” were incorporated as anchor indicators, following the assessment method that utilizes algorithms and a matrix similar to those used in general RRAs. The inclusion of various indicators and anchor indicators has enhanced the timeliness and appropriateness of evaluations. The United States Centers for Disease Control and Prevention categorizes community levels (low, medium, high) based on a scale that includes new cases, new admissions, and inpatient bed usage [9]. Similarly, the UK HSA sets the alert level by combining expert judgment and risk assessment with more quantifiable indicators and thresholds [8]. Information such as community levels and alert levels is communicated by establishing criteria for key indicators that are tailored to the specific conditions of each country. Effective and clear communication is crucial, as people are more likely to comply with government public safety advisories when they are well-informed about the risks they face and understand the preventative measures available [14,15].

The risk perception rate appeared to be strongly correlated with the COVID-19 risk level, suggesting that the perceived risk and the actual risk level were similar at the time of assessment. Public perception of risk plays a crucial role in shaping individual decisions related to COVID-19, where behavioral and policy support act as positive influences on quarantine measures [16,17]. Chatterjee et al. [15] highlighted that awareness and understanding of risk at the community level are essential for improving prevention efforts. Therefore, contextual information is vital in enhancing risk assessment. Additionally, policy adjustments are considered in light of the potential economic and social impacts that could result from a prolonged pandemic. In the Republic of Korea, the effectiveness and sustainability of infection prevention and control measures are experiencing diminishing returns due to cumulative effects, such as distress among micro-enterprises suffering significant sales losses, weakened support systems for vulnerable populations, and disruptions in student education [18]. Social factors, including education and income, along with infectious disease outbreaks, have also been observed to influence policy changes in the Republic of Korea. Consequently, in the face of an extended epidemic, further research is necessary to prioritize and understand how infectious disease outbreaks and social factors impact policy decision-making.

The significant correlation between the Rt and the weekly COVID-19 risk level, with a 1-week lead, indicated that this indicator responded rapidly to changes in the situation. The risk level from a 1-week lagged value showed the strongest correlation with the occurrence of newly confirmed severe cases. The highest correlation was observed between the risk level from the 2-week lagged value and the number of severe cases in hospitals, as well as between the risk level from the 2-week lagged value and the number of deaths. This suggests that the assessed risk level functioned as an early warning system.

This study compared the findings of COVID-19 risk assessments with subsequent national perceptions and policy changes. However, the Oxford stringency index, used as a representative measure of policy changes, did not accurately reflect the policy shifts in a single country. This index measures the extent of a country’s response policies, such as public health and social intervention measures, in reaction to the spread of COVID-19. A more sensitive indicator that accurately represents the Republic of Korea’s COVID-19 outbreak control policies would enable a more precise correlation. Currently, the Oxford stringency index is the primary indicator used to quantify and compare policies, and it was also employed in this study. Another limitation is that, although the same names for indicators are used, the methods of aggregating some indicators have changed. These adjustments were made in response to evolving circumstances, including the expansion of testing methods such as the rapid antigen test. As the criteria within the Republic of Korea were updated, this study incorporated these changes in aggregation methods (Table S1).

The KDCA conducted weekly risk assessments as part of its response to COVID-19. The results of these assessments were utilized in policymaking and influenced the adoption of evidence-based policies. The risk levels were communicated weekly through briefings and used for risk communication. In the context of COVID-19 becoming endemic, risk assessments in the Republic of Korea were conducted using methods different from the RRAs. These assessments were effectively implemented for early warning by incorporating useful indicators and reflecting expert opinions. However, it is anticipated that risk assessments tailored to specific targets could be even more effective for risk communication. The timing for raising alertness may vary depending on the target audience, such as the medical community or the general public. Global preparedness for, and response capacities to, emerging and re-emerging infectious diseases with epidemic potential can be enhanced with each outbreak [19].

In the next pandemic, it may be possible to conduct advanced risk assessments that are tailored to specific situations. Contextual indicators, which take into account time lags and scales of implementation [1], could enhance the evaluation of future risk assessment outcomes, particularly when there are challenges in reflecting specific situational factors in crisis management.

HIGHLIGHTS

• Risk assessments, which were developed using various indicators, and risk levels were released weekly to the public in the Republic of Korea, aiding in the implementation of preventive measures.

• Weekly coronavirus disease 2019 risk assessments were conducted to serve as an early warning system and to provide timely justification for government intervention policies.

• If risk assessments are tailored for target audience and incorporate contextual indicators, they could provide more useful and timely support for public health measures.

Supplementary Material

Article information

Ethics Approval

This study was approved by the Institutional Review Board of the KDCA (No: 2023-02-05PE-A) and performed in accordance with the principles of the Declaration of Helsinki.

Conflicts of Interest

The authors have no conflicts of interest to declare.

Funding

None.

Availability of Data

All data generated or analyzed during this study are included in this published article. For other data, these may be requested through the corresponding author.

Authors’ Contributions

Conceptualization: ST, SH; Data curation: SH, JO; Formal analysis: SH; Methodology: ST, SH, JL, YK, BIK; Resources: JO, MJL, HK; Visualization: SH, YK; Writing–original draft: SH; Writing–review & editing: all authors. All authors read and approved the final manuscript.

Acknowledgements

We would like to express our gratitude to the numerous officers and managers, including the Commissioner of the Korea Disease Control and Prevention Agency (KDCA), for their invaluable support during the COVID-19 response. We also extend our thanks to all those who contributed to this work and participated in the process of risk assessment.

Figure 1.

Weekly COVID-19 risk levels in the Republic of Korea and the number of confirmed COVID-19 cases (November 2021 to May 2022).

W1, week 1; W2, week 2; W, week 3; W4, week 4; W5, week 5; Rt, time-varying reproduction number.

Figure 2.

Weekly COVID-19 risk levels in the Republic of Korea and the number of severe COVID-19 cases (November 2021–May 2022).

W1, week 1; W2, week 2; W, week 3; W4, week 4; W5, week 5.

Figure 3.

Weekly COVID-19 risk levels in the Republic of Korea and the risk perception rate (November 2021 to May 2022).

W1, week 1; W2, week 2; W, week 3; W4, week 4; W5, week 5.

Figure 4.

Weekly COVID-19 risk levels in the Republic of Korea and the community mobility rate (November 2021 to May 2022).

W1, week 1; W2, week 2; W, week 3; W4, week 4; W5, week 5.

Figure 5.

Weekly COVID-19 risk levels in the Republic of Korea and the Oxford stringency index (November 2021 to May 2022).

W1, week 1; W2, week 2; W, week 3; W4, week 4; W5, week 5.

References

• 1. World Health Organization (WHO) (CH). Rapid risk assessment of acute public health events. WHO; 2012.
• 2. European Centre for Disease Prevention and Control (ECDC) (SWE). Operational tool on rapid risk assessment methodology ECDC 2019 [Internet]. ECDC; 2019 [cited 2023 Feb 4]. Available from: https://www.ecdc.europa.eu/sites/default/files/documents/operational-tool-rapid-risk-assessment-methodolgy-ecdc-2019.pdf.
• 3. Cheng HY, Li SY, Yang CH. Initial rapid and proactive response for the COVID-19 outbreak: Taiwan’s experience. J Formos Med Assoc 2020;119:771−3.ArticlePubMedPMC
• 4. Kim I, Lee J, Lee J, et al. KCDC Risk Assessments on the initial phase of the COVID-19 outbreak in Korea. Osong Public Health Res Perspect 2020;11:67−73.ArticlePubMedPMCPDF
• 5. Wu J, Yang X, Ji H, et al. Risk assessment of COVID-19 pandemic worldwide, December 2020. Dis Surveill 2021;36:11−5.
• 6. Burki T. WHO ends the COVID-19 public health emergency. Lancet Respir Med 2023;11:588. ArticlePubMed
• 7. Robert Koch Institute (RKI) (DEU). Risk assessment on COVID-19 [Internet]. RKI; 2023 [cited 2023 Feb 4]. Available from: https://www.rki.de/DE/Content/InfAZ/N/Neuartiges_Coronavirus/Risikobewertung.html?nn=2386228. German.
• 8. UK Health Security Agency (UKHSA) (GBR). Guidance: UK COVID-19 alert level methodology: an overview, updated 29 March 2023 [Internet]. UKHSA; 2023 [cited 2023 Feb 4]. Available from: https://www.gov.uk/government/publications/uk-covid-19-alert-level-methodology-an-overview/uk-covid-19-alert-level-methodology-an-overview.
• 9. Centers for Disease Control and Prevention (CDC) (US). United States COVID-19 community levels by county (version date: May 11, 2023) [Internet]. CDC; 2023 [cited 2024 Feb 4]. Available from: https://data.cdc.gov/Public-Health-Surveillance/United-States-COVID-19-Community-Levels-by-County/3nnm-4jni/about_dat.
• 10. Yoo M, Kim Y, Baek S, et al. The concept of reproduction number and changes according to government response policies. Public Health Wkly Rep 2021;14:288−9. Korean.ArticlePubMedPDF
• 11. Hale T, Angrist N, Goldszmidt R, et al. A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker). Nat Hum Behav 2021;5:529−38.
• 12. Mathieu E, Ritchie H, Rodes-Guirao L, et al. Coronavirus pandemic (COVID-19) [Internet]. OurWorldinData.org; 2020 [cited 2023 Feb 4]. Available from: https://ourworldindata.org/coronavirus.Article
• 13. Ryu B, Shin E, Kim NY, et al. Severity of COVID-19 associated with SARS-CoV-2 variants circulating in the Republic of Korea. Public Health Wkly Rep 2022;15:2873−83. Korean.ArticlePubMedPMCPDF
• 14. Barry JM. Pandemics: avoiding the mistakes of 1918. Nature 2009;459:324−5.ArticlePubMedPMC
• 15. Chatterjee R, Bajwa S, Dwivedi D, et al. COVID-19 Risk Assessment tool: dual application of risk communication and risk governance. Prog Disaster Sci 2020;7:100109. ArticlePubMedPMC
• 16. Cori L, Bianchi F, Cadum E, et al. Risk perception and COVID-19. Int J Environ Res Public Health 2020;17:3114. Article
• 17. Warren GW, Lofstedt R. Risk communication and COVID-19 in Europe: lessons for future public health crises. J Risk Res 2022;25:1161−75.
• 18. Korea Disease Control and Prevention Agency (KDCA) (KR). Central Disaster Management Headquarters Press release: roadmap for gradual recovery to a new normal announced (2021.11.29) [Internet]. KDCA; 2021 [cited 2024 Feb 4]. Available from: https://kdca.go.kr/board/board.es?mid=a20501010000&bid=0015&list_no=717703&cg_code=&act=view&nPage=1&newsField=. Korean.ArticlePubMedPMC
• 19. Ippolito G, Hui DS, Ntoumi F, et al. Toning down the 2019-nCoV media hype-and restoring hope. Lancet Respir Med 2020;8:230−1.

Figure & Data

References

Citations

Citations to this article as recorded by