Advertisement
Research Article| Volume 118, P73-82, May 2022

Stringency of containment and closures on the growth of SARS-CoV-2 in Canada prior to accelerated vaccine roll-out

Open AccessPublished:February 21, 2022DOI:https://doi.org/10.1016/j.ijid.2022.02.030

      Highlights

      • The empiric study of non-pharmaceutical interventions (NPIs) against COVID-19 is important for pandemic preparedness.
      • We explored the correlation between NPI stringency and COVID-19 case growth in Canada.
      • We observed little relationship between NPI stringency and growth of SARS-CoV-2, particularly in fall 2020.
      • Understanding which NPIs “work” and "for whom" will inform future pandemic responses overall.

      Abstract

      Background

      Many studies have examined the effectiveness of non-pharmaceutical interventions (NPIs) on SARS-CoV-2 transmission worldwide. However, less attention has been devoted to understanding the limits of NPIs across the course of the pandemic and along a continuum of their stringency. In this study, we explore the relationship between the growth of SARS-CoV-2 cases and an NPI stringency index across Canada before the accelerated vaccine roll-out.

      Methods

      We conducted an ecological time-series study of daily SARS-CoV-2 case growth in Canada from February 2020 to February 2021. Our outcome was a back-projected version of the daily growth ratio in a stringency period (i.e., a 10-point range of the stringency index) relative to the last day of the previous period. We examined the trends in case growth using a linear mixed-effects model accounting for stringency period, province, and mobility in public domains.

      Results

      Case growth declined rapidly by 20–60% and plateaued within the first month of the first wave, irrespective of the starting values of the stringency index. When stringency periods increased, changes in case growth were not immediate and were faster in the first wave than in the second. In the first wave, the largest decreasing trends from our mixed effects model occurred in both early and late stringency periods, depending on the province, at a geometric mean index value of 30⋅1 out of 100. When compared with the first wave, the stringency periods in the second wave possessed little association with case growth.

      Conclusions

      The minimal association in the first wave, and the lack thereof in the second, is compatible with the hypothesis that NPIs do not, per se, lead to a decline in case growth. Instead, the correlations we observed might be better explained by a combination of underlying behaviors of the populations in each province and the natural dynamics of SARS-CoV-2. Although there exist alternative explanations for the equivocal relationship between NPIs and case growth, the onus of providing evidence shifts to demonstrating how NPIs can consistently have flat association, despite incrementally high stringency.

      Key Words

      Introduction

      Throughout 2020, non-pharmaceutical interventions (NPIs) were the primary tools employed by governments and public health agencies to slow the spread of SARS-CoV-2 (
      • Ferguson NM
      • Laydon D
      • Nedjati-Gilani G
      • Imai N
      • Ainslie K
      • Baguelin M
      • et al.
      Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand.
      ,
      • Koo JR
      • Cook AR
      • Park M
      • Sun Y
      • Sun H
      • Tim JT
      • et al.
      Interventions to mitigate early spread of SARS-CoV-2 in Singapore: a modelling study.
      ). In Canada, as in many other countries, common NPIs included border closures, bans on non-essential travel, and mandatory physical distancing measures (
      • McCoy LG
      • Smith J
      • Anchuri K
      • Berry I
      • Pineda J
      • Harish V
      • et al.
      Characterizing early Canadian federal, provincial, territorial and municipal nonpharmaceutical interventions in response to COVID-19: a descriptive analysis.
      ). However, in contrast to many countries—particularly those in Europe—the authority and responsibility to implement these policies fall on provincial and territorial governments, meaning there is no formally coordinated response between them (

      Cameron-Blake E, Breton C, Sim P, Tatlow H, Hale T, Wood A, et al. Variation in the Canadian provincial and territorial responses to COVID- 19. Blavatnik School of Government Working Paper 2021. www.bsg.ox.ac.uk/covidtracker.

      ). Instead, provincial and territorial responses, like any other large-scale intervention, were contingent upon local political and social contexts, and thus contained substantial variability (
      • McCoy LG
      • Smith J
      • Anchuri K
      • Berry I
      • Pineda J
      • Harish V
      • et al.
      Characterizing early Canadian federal, provincial, territorial and municipal nonpharmaceutical interventions in response to COVID-19: a descriptive analysis.
      ). These potentially divergent approaches create unique opportunities when seeking to evaluate Canada's pandemic responses in a systematic way.
      Composite measures—which combine a set of indicators into an index—help abstract away from the subtleties at the sub-national level (
      • Hale T
      • Angrist N
      • Goldszmidt R
      • Kira B
      • Petherick A
      • Phillips T
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
      ). The main strengths of composite indicators are that they 1) allow for systematic comparisons across different jurisdictions and 2) permit quantitative comparisons between the “intensity” of government responses and spread of infection (
      • Hale T
      • Angrist N
      • Goldszmidt R
      • Kira B
      • Petherick A
      • Phillips T
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
      ). These strengths inherently contain the concept of dose, which can be broadly understood as the “amount” of intervention provided to a target population (
      • Rowbothum S
      • Conte K
      • Hawe P.
      Variation in the operationalisation of dose in implementation of health promotion interventions: insights and recommendations from a scoping review.
      ). However, most existing research focuses on isolating the effect of common individual NPIs (
      • Haug N
      • Geyrhofer L
      • Londei A
      • Dervic E
      • Desvars-Larrive A
      • Loreto V
      • et al.
      Ranking the effectiveness of worldwide COVID-19 government interventions.
      ,
      • Flaxman S
      • Mishra S
      • Gandy A
      • Unwin HJT
      • Mellan TA
      • Coupland H
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      ,
      • Liu Y
      • Morgenstern C
      • Kelly J
      • Lowe R
      • et al.
      The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
      ,
      • Li Y
      • Campbell H
      • Kulkarni D
      • Harpur A
      • Nundy M
      • Wang X
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      ,
      • Hsiang S
      • Allen D
      • Annan-Phan S
      • Bell K
      • Bolliger I
      • Chong T
      • et al.
      The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
      ,
      • Bendavid E
      • Oh C
      • Bhattacharya J
      • Ioannidis JPA
      Assessing mandatory stay-at-home and business closure effects on the spread of COVID-19.
      ,
      • Küchenhoff H
      • Günther F
      • Höhle M
      • Bender A.
      Analysis of the early COVID-19 epidemic curve in Germany by regression models with change points.
      ,
      • Berry CR
      • Fowler A
      • Glazer T
      • Handel-Meyer S
      • MacMillen A.
      Evaluating the effects of shelter-in-place policies during the COVID-19 pandemic.
      ).
      The few studies that have used composite measures show a dose-response association between incrementally stringent interventions and the value of either the effective reproductive number (
      • Turbé H
      • Bjelogrlic M
      • Robert A
      • Gaudet-Blavignac C
      • Goldman JP
      • Lovis C.
      Adaptive time-dependent priors and Bayesian inference to evaluate SARS-CoV-2 public health measures validated on 31 countries.
      ) or the cumulative incidence (
      • Mezencev R
      • Klement C
      Stringency of the containment measures in response to COVID-19 inversely correlates with the overall disease occurrence over the epidemic wave.
      ) of SARS-CoV-2. However, an important challenge with these analyses is that they have focused on the early periods of the pandemic—specifically, Europe's first wave—and they have not recognized that outbreak dynamics are time-varying, where slowed transmission will occur without the influence of any NPI (
      • Bendavid E
      • Oh C
      • Bhattacharya J
      • Ioannidis JPA
      Assessing mandatory stay-at-home and business closure effects on the spread of COVID-19.
      ).
      Even with approved COVID-19 vaccines, NPIs are thought to remain a crucial component of SARS-CoV-2 control strategies (
      • Moore S
      • Hill EM
      • Tildesley MJ
      • Dyson L
      • Keeling MJ.
      Vaccination and non-pharmaceutical interventions for COVID-19: a mathematical modelling study.
      ,
      • Gozzi N
      • Bajardi P
      • Perra N.
      The importance of non-pharmaceutical interventions during the COVID-19 vaccine rollout.
      ). Thus, quantifying dose and dose-response-like relationships through composite measures will be useful for assessing if NPIs correlate with a decline in the growth of SARS-CoV-2. It will also enhance our understanding of the different types and timing of NPI “packages,” not only across regions within a country but across the temporal span of the pandemic. In this paper, we explore the relationship between a stringency index of SARS-CoV-2 mitigation strategies and the growth of confirmed cases across 5 provinces in Canada, throughout2 epidemic “waves”: spring 2020 (wave 1) and fall-winter 2020/2021 (wave 2).

      Methods

      The conceptual model behind our approach is that before meaningful levels of population immunity, individual behavior drives transmission and that the “packages” of government-mandated NPIs encourage individual behavior change (
      • Bendavid E
      • Oh C
      • Bhattacharya J
      • Ioannidis JPA
      Assessing mandatory stay-at-home and business closure effects on the spread of COVID-19.
      ). We assumed that case growth would incrementally exhibit larger decreases over time as the stringency index increased: If “less stringent” measures provide small nudges to individual behavior, “more stringent” measures will produce larger behavioral effects, and thus larger reductions in the growth of new cases. Specifically, we anticipated these effects to either manifest early—if initial responses were “highly stringent”—or over time—as pandemic responses intensified.

      Study Design & Surveillance Data

      We conducted a longitudinal study of daily SARS-CoV-2 incidence during the first and second waves in the Canadian provinces of Alberta, British Columbia, Manitoba, Ontario, and Quebec, which comprised 95% of all cases in Canada by the start of March 2021. The data used in these analyses cover the interval February 28, 2020 to February 15, 2021.
      We obtained the count of daily laboratory-confirmed cases from provincial surveillance databases to calculate the growth ratio of SARS-CoV-2: in Alberta, from Alberta Health's Interactive Aggregate Data on COVID-19 (

      Alberta Health. COVID-19 Alberta Statistics: https://www.alberta.ca/stats/covid-19-alberta-statistics.htm (Last accessed: 24 August 2021).

      ); in British Columbia, from the British Columbia Centre for Disease Control's COVID-19 dashboard (

      British Columbia Centre for Disease Control. BC COVID-19 Data: http://www.bccdc.ca/health-info/diseases-conditions/covid-19/data (Last accessed: 24 August 2021).

      ); in Manitoba, from the Province of Manitoba's Interactive Dashboard on COVID-19 (

      Manitoba Health. Cases and Risk of COVID-19 in Manitoba: https://www.gov.mb.ca/covid19/updates/cases.html (Last accessed: 24 August 2021).

      ); in Ontario, from the ICES (formerly the Institute of Clinical and Evaluative Sciences); and in Québec, from the Institut national de santé publique du Québec (
      Institut national de santé publique du Québec
      Données COVID-19 au Québec.
      ). In contrast to the other four provinces, Québec's case counts also included cases without laboratory confirmation that were epidemiologically linked to known SARS-CoV-2 positives.
      Start and end dates of the first wave coincide with each province accumulating more than 10 cases and the last day before the relaxation of restrictions began, respectively; for the second wave, start and end dates coincide with rising case counts in early to late summer through to the trough in case counts after the fall-winter peak. We excluded all province-days with fewer than 10 cumulative cases from this analysis because early case counts could have been driven by imported cases rather than local transmission.

      Data on Non-Pharmaceutical Interventions

      We used data on all NPIs from the Oxford COVID-19 Government Response Tracker (OxCGRT,
      • Hale T
      • Angrist N
      • Goldszmidt R
      • Kira B
      • Petherick A
      • Phillips T
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
      ). The project has tracked government policies and interventions across a standardized series of indicators for over 180 countries and includes subnational jurisdictions within Canada. The tool contains 8 categories on containment and closures, 4 on economic policies, and 8 on health system policies. Most indicators are reported on monotonic ordinal scales, with others coded on continuous scales, allowing for quantitative analysis of the degree of government response (
      • Hale T
      • Angrist N
      • Goldszmidt R
      • Kira B
      • Petherick A
      • Phillips T
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
      ).
      The containment and closures indicators used in this analysis reflect the more restrictive NPIs and include school and workplace closures, cancelling public events, restricting gathering sizes, closing public transportation, restrictions on internal movement, and travel controls. A score for each indicator is created by taking an ordinal value for that indicator and adding an extra 0⋅5 points if the policy applies across the entire jurisdiction as opposed to a specific locality. Each indicator is then re-scaled by its maximum ordinal value and summed to create a stringency index with scores between 0 and 100.

      Mobility Data

      We used data from Google's Community Mobility Reports (

      COVID-19 Community Mobility Reports. Google. Available: www.google.com/covid19/mobility/(Last accessed: 24 August 2021).

      ) to create a mobility covariate for the main analysis (described below). Broadly, these data represent the percent change, from pre-pandemic movement, of where Canadians were going based on a proportion of cell phone “pings” across 6 different categories: residential, workplace, transit stations, parks, groceries and pharmacies, and retail and recreation. Apart from “residential,” we averaged the fractional change from baseline (between February 15 and March 11, 2019) to represent movement outside people's homes.
      Because the stringency index and mobility data are likely collinear (
      • Hale T
      • Angrist N
      • Goldszmidt R
      • Kira B
      • Petherick A
      • Phillips T
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
      ), we disentangled their unique contributions by regressing mobility on the stringency index and substituted the value of mobility with the residuals of this simple regression model (
      • Graham MH.
      Confronting multicollinearity in ecological multiple regression.
      ). We linked the surveillance, NPI, and mobility datasets by province and date to generate our working dataset.

      Outcome Variable

      Similarly to
      • Brown KA
      • Soucy J-PR
      • Buchan SA
      • Sturrock SL
      • Berry I
      • Stall NM
      • et al.
      The mobility gap: estimating mobility thresholds required to control SARS-CoV-2 in Canada.
      , we hypothesized that “more stringent” NPIs would impact SARS-CoV-2 dynamics in terms of changes in rates, rather than absolute levels of infection. Therefore, our outcome was the daily growth rate, g(t), calculated as the ratio of the incident SARS-CoV-2 cases on a given day divided by those from the previous day: g(t)=I(t)I(t1). Because surveillance data are a product of inherent delays related to both the biology of infection (e.g., incubation period) and reporting (e.g., cases are not reported by the date they were infected), we produced a back-projected version of the daily growth rate for infections using the R package EpiNow2 (
      • Abbott S
      • Hellewell J
      • Thompson RN
      • Sherratt K
      • Gibbs HP
      • Bosse NI
      • et al.
      Estimating the time-varying reproduction number of SARS-CoV-2 using national and subnational case counts [version 2; peer review: 1 approved with reservations].
      ; Figure 1, panel A).
      Figure 1
      Figure 1Schematic diagrams of: (A) Back-projected cases by their estimated date of infection (blue ribbon) versus their date of report (gray bars). Because of delays between being infected and confirmed as a case, the back-projected cases occur earlier in time. The light and dark blue ribbons are the 90% and 50% credible intervals, respectively; and (B) our calculation of the relative growth ratio. Day t is defined as the t-th day of a stringency period (i.e., since O.S.I changed deciles). Day n represents the last day of the period. Note that different stringency periods could have different numbers of days; O.S.I = Oxford Stringency Index; g(t) = growth rate on day t. Panel (A) is modified from
      • Abbott S
      • Hellewell J
      • Thompson RN
      • Sherratt K
      • Gibbs HP
      • Bosse NI
      • et al.
      Estimating the time-varying reproduction number of SARS-CoV-2 using national and subnational case counts [version 2; peer review: 1 approved with reservations].
      . Panel (B) is a modified version of in
      • Li Y
      • Campbell H
      • Kulkarni D
      • Harpur A
      • Nundy M
      • Wang X
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      .
      Briefly, EpiNow2 uses the distributions of the incubation period, generation time, and the time from symptom onset to report date as part of an inverse convolution to estimate the number of infections per day from the time series of daily reported cases. The distributions for the incubation period and generation time were based on well-established values for SARS-CoV-2 (
      • Lauer SA
      • Grantz KH
      • Bi Q
      • Jones FK
      • Zheng Q
      • Meredith HR
      • et al.
      The incubation period of coronavirus disease 2019 (COVID-19) from publicly reported confirmed cases: Estimation and application.
      ,
      • Ganyani T
      • Kremer C
      • Chen D
      • Torneri A
      • Faes C
      • Wallinga J
      • et al.
      Estimating the generation interval for coronavirus disease (COVID-19) based on symptom onset data.
      ), as well as a log-normal distribution for the reporting delay time with a mean of 0⋅84 days and a standard deviation of 1⋅44 days (
      • Abbott S
      • Hellewell J
      • Thompson RN
      • Sherratt K
      • Gibbs HP
      • Bosse NI
      • et al.
      Estimating the time-varying reproduction number of SARS-CoV-2 using national and subnational case counts [version 2; peer review: 1 approved with reservations].
      ).

      Analysis

      We have based our approach on the analysis of
      • Li Y
      • Campbell H
      • Kulkarni D
      • Harpur A
      • Nundy M
      • Wang X
      • et al.
      The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
      and defined a stringency period as a time through which the stringency index remained within a 10-point range. We used these periods to sub-divide the timeline of each provincial epidemic into segments based on the status of the containment and closure NPIs. For each period, we defined a daily growth rate, g(t), as the growth rate on the t-th day of that period (i.e., since the stringency index changed deciles) and g(0) as the growth rate of the last day of the previous period (i.e., before new NPIs were introduced, Figure 1, panel B).
      Because we also expected any association of NPIs with case growth to be relative to previous periods, we calculated a relative growth ratio between g on day t and g on day 0 as a measure of association between NPI stringency and the growth of SARS-CoV-2 cases. A relative growth ratio larger than 1 indicates an increase in transmission since the change in the stringency index, and a growth ratio smaller than 1 indicates a decrease. Based on the change of NPIs between 2 adjacent periods and the corresponding growth ratio, we were able to observe the behavior associated with incrementally stringent NPIs over time.
      In a descriptive analysis, we first quantified the extent to which higher starting values of the stringency index correlate with either high or low values of relative growth ratio using Kendall's τB. We also described any lagged declines in the relative growth ratio as each province entered different stringency periods. Here, the timeline for all provinces and stringency periods (combined) was stratified by the first and second waves. For each wave, we plotted a single loess curve for the longitudinal relative growth ratios.
      In the main analysis, we modeled the relative growth ratio using a linear mixed effects modeling approach, with the following equation:
      yij(t)=(β0+ν0i+γ0j)+(β1+ν1i+γ1j)×tij+β2Mij,


      where yij(t) is the relative growth ratio in the ith province, over the jth stringency period, on day t.
      As fixed effects, we entered a time trend, tij, and “residualized” mobility, Mij, (without interaction) into the model. As random effects, we had intercepts for province, ν0i, and stringency period, γ0j, as well as by-province, ν1i, and by-stringency slopes across time, γ1j. Any correlation between relative case growth and the values of stringency index was quantified through the slope term (β1+ν1i+γ1j) of the trending variable, tij. Using province and stringency period as random effects accounts for 2 sources of non-independence: 1) repeated measures from the same province are more likely to be similar; and 2), the current value of the stringency index depends on its previous values.
      Similar to
      • Bendavid E
      • Oh C
      • Bhattacharya J
      • Ioannidis JPA
      Assessing mandatory stay-at-home and business closure effects on the spread of COVID-19.
      , this simple model structure balances the strengths of empirical analyses, whereas accounting for pre-existing trends that naturally accompany outbreak dynamics (
      • Graham MH.
      Confronting multicollinearity in ecological multiple regression.
      ,
      • Kermack WO
      • McKendrick AG
      • Walker GT.
      A contribution to the mathematical theory of epidemics.
      ,
      • Philipson T.
      Economic epidemiology and infectious diseases.
      ). We fit 2 separate models—one for each wave—using the nlme package (
      • Pinheiro JC
      • Bates DM.
      Mixed-effects models in S and SPLUS.
      ) from R statistical software, version 4.1.0 (
      R Core Team
      R: A language and environment for statistical computing.
      ).

      Results

      The epidemic curves in Figure 2 show a first wave of cases concentrated between March and May 2020, with lower numbers of cases detected during the summer months, and a much larger second wave between August and September 2020. When compared with the observed cases in each wave, back-projected infections peaked earlier in time by 6 to 8 days (in the first wave) and 1 to 14 days (in the second). Converting the back-projected infections into growth rates demonstrated a shared cumulative decline across all provinces ranging between 20 and 60% (Figure 3). Except for Alberta and Manitoba, these declines stabilized within 1 month of March 11, 2020 and continued into the second wave.
      Figure 2
      Figure 2Epidemic curves of observed cases (gray bars) and the mean back-projected cases (blue line) in 5 Canadian provinces. Because of delays between being infected and being confirmed as a case, the back-projected cases occur earlier in time.
      Figure 3
      Figure 3Growth ratios in cases for the study provinces. The panels below show the daily growth ratio in cases and demonstrate a shared decline in case growth across all provinces. The y-axis re-scales the growth ratios against that on the first day after study inclusion. The dashed line represents the average cumulative reduction in growth ratio across all provinces in the second wave (20%); Str. Period = Stringency Period.
      Tables 1 and 2 contain the results of our descriptive analysis. After each province accumulated 10 reported cases, stringency index values ranged from 11⋅1 to 56⋅5 out of 100, with Manitoba introducing the largest array of containment and closure policies (Table 1). When centred on the time to reach 10 cases, there was little correlation between the higher relative starting value of the stringency index and the relative differences in daily case growth between Manitoba and the remaining 4 provinces (Kendall's τ = -0⋅18, p = 1⋅00, Table 2).
      Table 1Survey of individual NPIs that correspond to the stringency index at study inclusion for each province. Individual NPI categories (table columns) and the labels linked to the ordinal coding of the intervention (table cells) correspond to those defined in the containment and closure policies of the Oxford COVID-19 Government Response Tracker.
      O.S.I. at study inclusionSchoolClosuresWorkplaceClosuresPublic EventsBan GatheringsStay at HomeInternal MovementTravel ControlsPublic Information
      Manitoba56⋅5Recommend closingRecommend closingRequire canceling11-100 peopleRecommend caution in publicNot recommendedBan arrivalsUrging caution
      Alberta22⋅2Recommend cancelingQuarantine arrivalsCoordinated campaigns
      B.C.22⋅2Recommend cancelingQuarantine arrivalsCoordinated campaigns
      Ontario13⋅9Recommend cancelingCoordinated campaigns
      Québec11⋅1Recommend cancelingUrging caution
      O.S.I. = Oxford Stringency Index; B.C. = British Columbia.
      Table 2Assessing the correlation between initial values of the stringency index and the daily growth ratio of SARS-CoV-2 cases, adjusted for the time to reach 10 cases.
      O.S.I. at study inclusion
      O.S.I. = Oxford Stringency Index; study inclusion: > 10 cases in each province; B.C. = British Columbia.
      % Diff. O.S.I. (vs. MB)
      % Difference in O.S.I. = (O.S.IOther−O.S.IManitoba)/O.S.IManitoba×100; negative values indicate stringency index values at study inclusion were lower than Manitoba; MB = Manitoba.
      % Diff. g(t)Median (Q1, Q3)
      % Difference in daily growth ratio, g(t)=(g(t)Other−g(t)Manitoba)/g(t)Manitoba×100; where t = 7,…, 53 are the days in Manitoba's data that correspond to study inclusion and the beginning of their relaxation of restrictions; positive values indicate median daily growth ratios were higher than Manitoba's.
      Kendall's τ (p-value)
      Manitoba56⋅5
      Alberta22⋅2-60⋅75⋅7 (-8⋅3, 25⋅1)-0⋅18 (1⋅00)
      B.C.22⋅2-60⋅7-0⋅3 (-6⋅2, 17⋅1)
      Ontario13⋅9-75⋅429⋅3 (25⋅5, 31⋅0)
      Québec11⋅1-80⋅43⋅5 (-0⋅7, 54⋅7)
      low asterisk O.S.I. = Oxford Stringency Index; study inclusion: > 10 cases in each province; B.C. = British Columbia.
      # % Difference in O.S.I. = (O.S.IOtherO.S.IManitoba)/O.S.IManitoba×100; negative values indicate stringency index values at study inclusion were lower than Manitoba; MB = Manitoba.
      || % Difference in daily growth ratio, g(t)=(g(t)Otherg(t)Manitoba)/g(t)Manitoba×100; where t = 7,…, 53 are the days in Manitoba's data that correspond to study inclusion and the beginning of their relaxation of restrictions; positive values indicate median daily growth ratios were higher than Manitoba's.
      Figure 4
      Figure 4Temporal changes in the relative growth ratio following 10-point changes in the stringency index, in each wave, for all provinces combined. For each stringency period, the reference point is the day before entering a that period. It is important to note that different stringency periods can have different numbers of days. Because of limited data availability, particularly in the first wave, we did not plot timelines longer than 30 days. A locally weighted smoothing function (black line) plots the trend (i.e., the average) across each wave. Shaded regions are 95% confidence intervals. Str. Period = Stringency Period.
      Any association with introducing new (or intensifying) NPIs across each wave was not immediate (Figure 4). In the first wave, a local regression curve demonstrates that (on average) the largest associated decline in the relative growth ratio occurs within the first 9 days of changing stringency periods (left panel). After 10 days, relative growth ratios rebounded and plateaued. For the second wave, the largest associated decline occurs more slowly, approximately 16 days after changing stringency periods before also rebounding (right panel).
      Table 3 and Figure 5 present the results of the mixed effects regression analysis. For both waves, a trend model with random slopes and intercepts by province and by stringency period contained more information than a model with only random intercepts (LR = 27⋅03, df = 4, p < 0⋅0001; LR = 16⋅23, df = 4, p < 0⋅0001). Each model explained approximately 46% (for the first wave) and 69% (for the second) of the variation in relative case growth by including mixed effects in our model (Table 3).
      Table 3The Results of the Linear Mixed Effects Model.
      First WaveSecond Wave
      Fixed EffectsEstimate95% CIEstimate95% CI
      Intercept0⋅9650⋅867, 1⋅0641⋅0260⋅981, 1⋅071
      Time0⋅000-0⋅002, 0⋅0020⋅000-0⋅0002, 0⋅0002
      “Residual” Mobility-0⋅072-0⋅185, 0⋅0420⋅0310⋅005, 0⋅057
      Random Effects
      Province
      σIntercept0⋅0560⋅005, 0⋅6130⋅0130⋅000, 6⋅424
      σTime0⋅0010⋅000, 0⋅0100⋅00010⋅000, 0⋅001
      Stringency Index
      σIntercept0⋅1510⋅095, 0⋅2390⋅0510⋅019, 0⋅132
      σTime0⋅0030⋅001, 0⋅0050⋅00020⋅0001, 0⋅0005
      RM20⋅0070⋅004
      RC20⋅4630⋅691
      RM2 = “Marginal” and RC2 = “Conditional” coefficients of determination; each, respectively, represent the amount of variation explained by the fixed effects and with fixed and random effects combined (
      • Nakagawa S
      • Shielzeth H.
      A General and simple method for obtaining R-squared from general linear mixed models.
      ).
      Figure 5
      Figure 5Steepness (slope) of relative growth ratio across different stringency periods (i.e., deciles of the Oxford Stringency Index) for each province (colored dots) as estimated by the mixed effects regression analysis. Values > 0 (dashed line) indicate increased case growth, and values < 0 indicate decreased growth. Here, the greatest decline in growth rate in each province happened with the least stringent measures, at the earliest stages of the pandemic. A locally weighted smoothing function (black dotted line) is plotted to visualize trends in the estimated slopes across all provinces. Shaded regions are 95% confidence intervals.
      Linear trends within stringency periods displayed slopes that were more variable in the first wave than in the second across all provinces (Figure 5). For British Columbia, Ontario, and Quebec, the steepest negative slopes were observed for smaller, early values of the stringency index (ranging between 11⋅1 and 29⋅2 out of 100). These trends also decelerated (i.e., became less negative—in Ontario), turned upward (in Quebec), or remained nearly unchanged (in British Columbia) as the stringency index increased. In Alberta and Manitoba, we observed the steepest negative slopes for incrementally higher values of the stringency index ranging from 72⋅2 out of 100 (in Manitoba) and 75⋅9 out of 100 (in Alberta). Across all 5 provinces in the first wave, a local regression line demonstrates that case growth was initially negative, turned positive, and returned to earlier levels across incrementally higher values of the stringency index (Figure 5, left panel). Overall, the steepest declines in case growth correspond to a (geometric) mean stringency index of 30⋅1 out of 100. When compared with the first wave, the stringency periods in the second wave possessed very little association with case growth (Figure 5, right panel).

      Discussion

      The results presented here demonstrate a limited dose-response association between values of the stringency index and growth of SARS-CoV-2 cases in the first and second waves across 5 Canadian provinces. The minimal association in the first wave and lack of association in the second are compatible with the hypothesis that NPIs, per se, do not lead to a decline in case growth. These results were observed regardless of the initial (or maximum) value of the stringency index and are, overall, contrary to what we anticipated. At first glance, these findings might suggest that Canada's pandemic responses were sub-optimal at slowing the spread of SARS-CoV-2 as the stringency index increased. However, the explanation for our results requires nuance that extends beyond a simple dichotomy of whether government mandated NPIs did, or did not, have their intended effects. Instead, other potential explanations include how NPIs work across risk factors for onward transmission, and people's adherence to interventions over time.
      In the first wave, individual NPIs were generally focused on decreasing the risk of community-level spread of SARS-CoV-2. As a result, these early NPIs may have initially benefited those who, with a small reduction in contact, could effectively mitigate nearly all their transmission risks (
      • Mishra S
      • Kwong JC
      • Chan AK
      • Baral SD.
      Understanding heterogeneity to inform the public health response to COVID-19 in Canada.
      ,
      • Gomes MGM
      • Corder RM
      • King JG
      • Langwig KE
      • Souto-Maior C
      • Carneiro J
      • et al.
      Individual variation in susceptibility or exposure to SARS-CoV-2 lowers the herd immunity threshold.
      ). When coupled with higher rates of testing and isolation among high-income neighborhoods (
      • Sundaram ME
      • Calzavara A
      • Mishra S
      • Kustra R
      • Chan AK
      • Hamilton MA
      • et al.
      The individual and social determinants of COVID-19 in Ontario, Canada: a population-wide study.
      ), the early benefits may have rapidly saturated among those who could work remotely and remain well housed (
      Public Health England Transmission Group
      Factors contributing to risk of SARS-CoV2 transmission in various settings.
      ). Thus, early NPIs may have eliminated a small risk among many, but could not sufficiently mitigate a large risk among the few (
      • Sun K
      • Wang W
      • Gao L
      • Wang Y
      • Luo K
      • Ren L
      • et al.
      Transmission heterogeneities, kinetics, and controllability of SARS-CoV-2.
      ,
      • Cevik M
      • Baral SD.
      Networks of SARS-CoV-2 transmission.
      ), such as essential workers and those employed in public-facing jobs (
      • Sundaram ME
      • Calzavara A
      • Mishra S
      • Kustra R
      • Chan AK
      • Hamilton MA
      • et al.
      The individual and social determinants of COVID-19 in Ontario, Canada: a population-wide study.
      ,
      • Paul LA
      • Daneman N
      • Brown KA
      • Johnson J
      • van Ingen T
      • Joh E
      • et al.
      Characteristics associated with household transmission of SARS-CoV-2 in Ontario, Canada: a cohort study.
      ,
      • Rao A
      • Ma H
      • Moloney G
      • Kwong JC
      • Juni P
      • Sander B
      • et al.
      A disproportionate epidemic: COVID-19 cases and deaths among essential workers in Toronto.
      ). These same justifications also apply to the results of the second wave, despite starting at higher stringency index values than in the first. Although some provinces introduced new NPIs that targeted “intermediate” per-contact risks of transmission, such as extended family contacts (
      • Sun K
      • Wang W
      • Gao L
      • Wang Y
      • Luo K
      • Ren L
      • et al.
      Transmission heterogeneities, kinetics, and controllability of SARS-CoV-2.
      ), the majority of NPIs simply recommitted to a focus on community contacts. Therefore, the impact of many NPIs in the second wave—despite being more stringent—speaks more to a mismatch between where the largest risk of infection was in the population than how restrictive they were.
      Our results are also consistent with other important voluntary behaviors not accounted for by government policies (
      • Berry CR
      • Fowler A
      • Glazer T
      • Handel-Meyer S
      • MacMillen A.
      Evaluating the effects of shelter-in-place policies during the COVID-19 pandemic.
      ,
      • Goolsbee A
      • Syverson C.
      Fear, lockdown, and diversion: comparing drivers of pandemic economic decline 2020.
      ). In our analysis, we accounted for 2 sources of people's movements: those behaviors linked to the stringency index, and those that were voluntary. The latter was negatively—though not distinguishable from 0 (in the first wave)—and positively associated (in the second) with case growth (Table 2). This observed association has a plausible underlying behavior model indicating that many people were already avoiding busier public spaces—voluntarily—before being mandated to do so (
      • Bendavid E
      • Oh C
      • Bhattacharya J
      • Ioannidis JPA
      Assessing mandatory stay-at-home and business closure effects on the spread of COVID-19.
      ,
      • Berry CR
      • Fowler A
      • Glazer T
      • Handel-Meyer S
      • MacMillen A.
      Evaluating the effects of shelter-in-place policies during the COVID-19 pandemic.
      ,
      • Goolsbee A
      • Syverson C.
      Fear, lockdown, and diversion: comparing drivers of pandemic economic decline 2020.
      ). For example, increased mobility may have reallocated people from visiting “non-essential” businesses (bars and restaurants) to “essential” ones (groceries and other food sellers) where risk of infection was higher (
      • Goolsbee A
      • Syverson C.
      Fear, lockdown, and diversion: comparing drivers of pandemic economic decline 2020.
      ). Increasingly strict NPIs might also have limited association with case growth if people are less concerned about being infected; reduced concern could lead to a higher likelihood of restrictions getting ignored (
      • Goolsbee A
      • Syverson C.
      Fear, lockdown, and diversion: comparing drivers of pandemic economic decline 2020.
      ). Restrictions on activity toward the type of business also induce large reallocations of movement away from “disallowed” businesses, toward “allowed” ones. Thus, in contrast to recent work (
      • Brown KA
      • Soucy J-PR
      • Buchan SA
      • Sturrock SL
      • Berry I
      • Stall NM
      • et al.
      The mobility gap: estimating mobility thresholds required to control SARS-CoV-2 in Canada.
      ), our findings suggest that the limited association between case growth and greater stringency cannot be fully explained by changes in mobility. However, the consistency across provinces—particularly in fall 2020—is notable, given their diverse geo-political landscapes, which suggests that adherence may have less of an impact than appropriately targeting the risk of onward transmission.
      There are several key limitations to consider when interpreting these results. The first is that our estimates of case growth did not account for changes in testing regimes, over time. If a province increased their testing capacity or widened their eligibility criteria, we are likely to observe an artificial increase in case growth. A potential solution is to use other outcomes, such as hospitalization or mortality, which are considered less susceptible to testing-related biases. However, because NPIs are intended to decrease contact rates between individuals in a population, their primary impact, if effective, is on transmission (
      • Flaxman S
      • Mishra S
      • Gandy A
      • Unwin HJT
      • Mellan TA
      • Coupland H
      • et al.
      Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
      ;
      • Ferguson NM
      • Laydon D
      • Nedjati-Gilani G
      • Imai N
      • Ainslie K
      • Baguelin M
      • et al.
      Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand.
      ). Any impact on hospitalization and mortality will be delayed in some cases by several weeks. Previous research has also demonstrated that if stricter measures are not correlated with cases, they are unlikely to share that correlation with COVID-19 hospitalizations and mortality (
      • Berry CR
      • Fowler A
      • Glazer T
      • Handel-Meyer S
      • MacMillen A.
      Evaluating the effects of shelter-in-place policies during the COVID-19 pandemic.
      ). If our results have been biased by under-reporting, we would expect the trend estimates across stringency periods to overstate the correlation between NPIs and case growth. However, the fact that we estimate incrementally small associations between the stringency index and case growth suggests that the extent of this bias may also be small.
      Second, the back-projection method we used is an imperfect recreation of the date of infection because it imparts a distributional form on the incubation period, generation time, and reporting delay that could vary (
      • Küchenhoff H
      • Günther F
      • Höhle M
      • Bender A.
      Analysis of the early COVID-19 epidemic curve in Germany by regression models with change points.
      ), and may have led to temporal inaccuracies in our back-projected infections. Although back-projection methods have been used previously to reconstruct incidence curves of other infections, such as HIV (
      • Mallitt KA
      • Wilson DP
      • McDonald A
      • Wand H.
      Is back-projection methodology still relevant for estimating HIV incidence from national surveillance data?.
      , for a discussion), there are currently no suitable SARS-CoV-2 data which establish the reliability of these methods, particularly given the correlation between testing volumes and case counts mentioned above. However, given that back-projection explicitly accounts for delays between infection and diagnosis, we think it lends itself as a reasonable illustration of how such delays can be the crucial difference between well-intentioned policies missing their target and having their desired effect (
      • Roberts N
      • Andersen D
      • Deal RM
      • Shaffer WA.
      An introduction to delays, in An Introduction to computer simulation: the system dynamics approach.
      ,
      • Meadows D.
      Why systems surprise us.
      ).
      Third, by using the stringency index as the measure of NPI dose, we cannot disentangle which individual NPI is most associated with case growth. However, by measuring several NPIs, the stringency index mitigates the possibility that any one NPI is over- or misinterpreted, while supporting systematic comparisons across sub-national regions (
      • Hale T
      • Angrist N
      • Goldszmidt R
      • Kira B
      • Petherick A
      • Phillips T
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
      ). The stringency index does not measure the effectiveness of NPIs nor their costs. And, like all indices, it makes assumptions about what information counts and may introduce measurement bias (
      • Hale T
      • Angrist N
      • Goldszmidt R
      • Kira B
      • Petherick A
      • Phillips T
      • et al.
      A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
      ). For example, it focuses on closures and movement restrictions to contain the spread of infection, and thus does not include more “traditional” NPIs, such as testing, contact tracing, and isolation (
      • Fraser C
      • Riley S
      • Anderson RM
      • Ferguson NM.
      Factors that make an infectious disease controllable.
      ). Although this was a limitation, there were consistencies across the 5 provinces, given the federal nature of some laws and recommendations for NPIs including international travel, testing, and contact tracing.
      Fourth, the stringency index does not explicitly include information on vaccine programs that started in Canada in mid-December 2020. Although not an NPI, vaccination represents a potential confounder when examining the association between NPI stringency and case growth. However, given that only 2 to 4% of each province's population had received at least 1 dose by early February 2021 (
      Public Health Agency of Canada
      Canadian COVID-19 vaccination coverage report.
      ), the larger effect of the vaccine programs was likely to have been on decreased mortality, not the spread of infection in the second wave.
      A fifth concern is that we have not evaluated the impact of NPIs through other quasi-experimental approaches, like difference-in-differences. The most notable challenge to this approach is that inferences would have been made in the absence of sufficient amounts of baseline data in the growth of SARS-CoV-2 cases, and that there is direct feedback between intervention and outcome. This means that at least 2 key assumptions of a standard difference-in-differences design are questionable. Finally, because we regard this study as an ecological time series analysis, our findings do not necessarily imply causation. And although our results are supported elsewhere (
      • Herby J
      • Jonung L
      • Hanke SH.
      A literature review and meta-analysis of the effects of lock-downs on COVID-19 mortality.
      ), we cannot definitively conclude that incremental increases in the stringency index did not have some independent effect on case growth.
      Overall, our analysis provides little evidence that banning public gatherings, closing schools and universities, placing stay-at-home orders, controlling travel, and restricting business hours had any obvious association with reducing the growth of SARS-CoV-2 cases in Canada—an observation that aligns with those of previous pandemics (
      World Health Organization Writing Group
      Nonpharmaceutical interventions for pandemic influenza, national and community measures.
      ;
      • Whitelaw TH.
      The practical aspects of quarantine for influenza.
      ). However, we emphasize that estimating the effects of NPIs is extremely challenging, and no single approach produces irrefutable results. When placed in the context of other recent large-scale analyses (
      • Herby J
      • Jonung L
      • Hanke SH.
      A literature review and meta-analysis of the effects of lock-downs on COVID-19 mortality.
      ), our results have led to similar insights, which is reassuring. Nevertheless, we do not discount the possibility that alternative approaches may yield different results. Interpreting the lack of association between greater NPI stringency and reduced case growth could be bolstered through de novo data collection exploring voluntary behavior changes and the network structure of human interactions, rather than the use of routinely collected surveillance data.
      Ultimately, these findings suggest that the diminished returns associated with a higher stringency index should signal for better understanding of what “works” and “for whom” when it comes to mitigating the spread of infection. Although there exist alternative explanations for the equivocal relationship between stringency index and case growth (particularly in the second wave), the onus of providing evidence shifts to demonstrating how NPIs consistently have flat associations despite intensified stringency. Continued study of the long-term outcomes of many NPIs, including any adverse costs to society, is needed before similarly aggressive measures can be endorsed for future epidemic control.

      Conflict of Interests

      Mathieu Maheu-Giroux reports a contractual agreement with the Institut national de santé publique du Québec and Institut d'excellence en sante et en service sociaux. All remaining authors reported no competing interests.

      Funding

      This work was supported by the Canadian Institutes of Health Research (grant no. VR5-172683).

      Ethical Approval Statement

      The data used in this study are publicly available aggregated (i.e., non-identifiable) time series from provincial COVID-19 surveillance dashboards and the Oxford COVID-19 Government Response Tracker. Under Article 2.2 of Canada's Tri-Council 2018 Policy Statement on the Ethical Conduct for Research Involving Humans, these publicly available datasets do not require Ethics approval for access or analysis (https://ethics.gc.ca/eng/tcps2-eptc2_2018_chapter2-chapitre2.html).

      Acknowledgements

      This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and the Ministry of Long-Term Care. The analyses, conclusions, opinions, and statements expressed herein are solely those of the authors and do not reflect those of the funding or data sources; no endorsement is intended or should be inferred.

      References

        • Abbott S
        • Hellewell J
        • Thompson RN
        • Sherratt K
        • Gibbs HP
        • Bosse NI
        • et al.
        Estimating the time-varying reproduction number of SARS-CoV-2 using national and subnational case counts [version 2; peer review: 1 approved with reservations].
        Wellcome Open Res. 2020; 5: 112https://doi.org/10.12688/wellcomeopenres.16006.2
      1. Alberta Health. COVID-19 Alberta Statistics: https://www.alberta.ca/stats/covid-19-alberta-statistics.htm (Last accessed: 24 August 2021).

        • Bendavid E
        • Oh C
        • Bhattacharya J
        • Ioannidis JPA
        Assessing mandatory stay-at-home and business closure effects on the spread of COVID-19.
        Eur J Clin Invest. 2020; : e13484https://doi.org/10.1111/eci.13484
        • Berry CR
        • Fowler A
        • Glazer T
        • Handel-Meyer S
        • MacMillen A.
        Evaluating the effects of shelter-in-place policies during the COVID-19 pandemic.
        PNAS. 2021; 118e2019706118https://doi.org/10.1073/pnas.2019706118
      2. British Columbia Centre for Disease Control. BC COVID-19 Data: http://www.bccdc.ca/health-info/diseases-conditions/covid-19/data (Last accessed: 24 August 2021).

        • Brown KA
        • Soucy J-PR
        • Buchan SA
        • Sturrock SL
        • Berry I
        • Stall NM
        • et al.
        The mobility gap: estimating mobility thresholds required to control SARS-CoV-2 in Canada.
        CMAJ. 2021; 26: E592-E600https://doi.org/10.1503/cmaj.210132
      3. Cameron-Blake E, Breton C, Sim P, Tatlow H, Hale T, Wood A, et al. Variation in the Canadian provincial and territorial responses to COVID- 19. Blavatnik School of Government Working Paper 2021. www.bsg.ox.ac.uk/covidtracker.

        • Cevik M
        • Baral SD.
        Networks of SARS-CoV-2 transmission.
        Science. 2021; 373: 162-163https://doi.org/10.1126/science.abg0842
      4. COVID-19 Community Mobility Reports. Google. Available: www.google.com/covid19/mobility/(Last accessed: 24 August 2021).

        • Ferguson NM
        • Laydon D
        • Nedjati-Gilani G
        • Imai N
        • Ainslie K
        • Baguelin M
        • et al.
        Impact of non-pharmaceutical interventions (NPIs) to reduce COVID-19 mortality and healthcare demand.
        Imperial College London, London (UK)2020https://doi.org/10.25561/77482
        • Flaxman S
        • Mishra S
        • Gandy A
        • Unwin HJT
        • Mellan TA
        • Coupland H
        • et al.
        Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe.
        Nature. 2020; 584: 257-261https://doi.org/10.1038/s41586-020-2405-7
        • Fraser C
        • Riley S
        • Anderson RM
        • Ferguson NM.
        Factors that make an infectious disease controllable.
        PNAS. 2004; 101: 6146-6151https://doi.org/10.1073/pnas.0307506101
        • Ganyani T
        • Kremer C
        • Chen D
        • Torneri A
        • Faes C
        • Wallinga J
        • et al.
        Estimating the generation interval for coronavirus disease (COVID-19) based on symptom onset data.
        Euro Surveil. 2020; 252000257https://doi.org/10.2807/1560-7917.ES.2020.25.17.2000257
        • Gomes MGM
        • Corder RM
        • King JG
        • Langwig KE
        • Souto-Maior C
        • Carneiro J
        • et al.
        Individual variation in susceptibility or exposure to SARS-CoV-2 lowers the herd immunity threshold.
        medRxiv. 2021; https://doi.org/10.1101/2020.04.27.20081893
        • Goolsbee A
        • Syverson C.
        Fear, lockdown, and diversion: comparing drivers of pandemic economic decline 2020.
        J Public Econ. 2021; 193104311https://doi.org/10.1016/j.jpubeco.2020.104311
        • Gozzi N
        • Bajardi P
        • Perra N.
        The importance of non-pharmaceutical interventions during the COVID-19 vaccine rollout.
        PLoS Comput Biol. 2021; 17e1009346https://doi.org/10.1371/journal.pcbi.1009346
        • Graham MH.
        Confronting multicollinearity in ecological multiple regression.
        Ecology. 2003; 84: 2809-2815
        • Hale T
        • Angrist N
        • Goldszmidt R
        • Kira B
        • Petherick A
        • Phillips T
        • et al.
        A global panel database of pandemic policies (Oxford COVID-19 Government Response Tracker).
        Nat Hum Behav. 2021; 5: 529-538https://doi.org/10.1038/s41562-021-01079-8
        • Haug N
        • Geyrhofer L
        • Londei A
        • Dervic E
        • Desvars-Larrive A
        • Loreto V
        • et al.
        Ranking the effectiveness of worldwide COVID-19 government interventions.
        Nat Hum Behav. 2020; 4: 1303-1312https://doi.org/10.1038/s41562-020-01009-0
        • Herby J
        • Jonung L
        • Hanke SH.
        A literature review and meta-analysis of the effects of lock-downs on COVID-19 mortality.
        Stud Appl Econom. 2022; 200: 1-62
        • Hsiang S
        • Allen D
        • Annan-Phan S
        • Bell K
        • Bolliger I
        • Chong T
        • et al.
        The effect of large-scale anti-contagion policies on the COVID-19 pandemic.
        Nature. 2020; 584: 262-267
        • R Core Team
        R: A language and environment for statistical computing.
        R Foundation for Statistical Computing, Vienna, Austria2022
        • Institut national de santé publique du Québec
        Données COVID-19 au Québec.
        2021 (Last accessed: 24 August)
        • Kermack WO
        • McKendrick AG
        • Walker GT.
        A contribution to the mathematical theory of epidemics.
        Proc R Soc Lond A Math Phys Sci. 1927; 115: 700-721https://doi.org/10.1098/rspa.1927.0118
        • Koo JR
        • Cook AR
        • Park M
        • Sun Y
        • Sun H
        • Tim JT
        • et al.
        Interventions to mitigate early spread of SARS-CoV-2 in Singapore: a modelling study.
        Lancet Infect Dis. 2020; 20: 678-688https://doi.org/10.1016/S1473-3099(20)30162-6
        • Küchenhoff H
        • Günther F
        • Höhle M
        • Bender A.
        Analysis of the early COVID-19 epidemic curve in Germany by regression models with change points.
        Epidemiol Infect. 2021; 149 (1–7): e68https://doi.org/10.1017/S0950268821000558
        • Lauer SA
        • Grantz KH
        • Bi Q
        • Jones FK
        • Zheng Q
        • Meredith HR
        • et al.
        The incubation period of coronavirus disease 2019 (COVID-19) from publicly reported confirmed cases: Estimation and application.
        Ann Int Med. 2020; 172: 577e582https://doi.org/10.7326/M20-0504
        • Li Y
        • Campbell H
        • Kulkarni D
        • Harpur A
        • Nundy M
        • Wang X
        • et al.
        The temporal association of introducing and lifting non-pharmaceutical interventions with the time-varying reproduction number (R) of SARS-CoV-2: a modelling study across 131 countries.
        Lancet Infect Dis. 2020; 21: 193-202https://doi.org/10.1016/S1473-3099(20)30785-4
        • Liu Y
        • Morgenstern C
        • Kelly J
        • Lowe R
        • et al.
        The impact of non-pharmaceutical interventions on SARS-CoV-2 transmission across 130 countries and territories.
        BMC Med. 2021; 19: 40https://doi.org/10.1186/s12916-020-01872-8
        • Mallitt KA
        • Wilson DP
        • McDonald A
        • Wand H.
        Is back-projection methodology still relevant for estimating HIV incidence from national surveillance data?.
        Open AIDS J. 2012; 6: 108-111
      5. Manitoba Health. Cases and Risk of COVID-19 in Manitoba: https://www.gov.mb.ca/covid19/updates/cases.html (Last accessed: 24 August 2021).

        • McCoy LG
        • Smith J
        • Anchuri K
        • Berry I
        • Pineda J
        • Harish V
        • et al.
        Characterizing early Canadian federal, provincial, territorial and municipal nonpharmaceutical interventions in response to COVID-19: a descriptive analysis.
        CMAJ Open. 2020; https://doi.org/10.9778/cmajo.20200100
        • Meadows D.
        Why systems surprise us.
        Thinking in systems. Chelsea Green Publishing, 2008
        • Mezencev R
        • Klement C
        Stringency of the containment measures in response to COVID-19 inversely correlates with the overall disease occurrence over the epidemic wave.
        medRxiv preprint. 2021; https://doi.org/10.1101/2021.01.26.21250501
        • Mishra S
        • Kwong JC
        • Chan AK
        • Baral SD.
        Understanding heterogeneity to inform the public health response to COVID-19 in Canada.
        CMAJ. 2020; 192: E684-E685https://doi.org/10.1503/cmaj.201112
        • Moore S
        • Hill EM
        • Tildesley MJ
        • Dyson L
        • Keeling MJ.
        Vaccination and non-pharmaceutical interventions for COVID-19: a mathematical modelling study.
        Lancet Infect Dis. 2021; 21: 793-802https://doi.org/10.1016/S1473-3099(21)00143-2
        • Nakagawa S
        • Shielzeth H.
        A General and simple method for obtaining R-squared from general linear mixed models.
        Meth Ecol Evol. 2013; 4: 133-142https://doi.org/10.1111/j.2041-210x.2012.00261.x
        • Paul LA
        • Daneman N
        • Brown KA
        • Johnson J
        • van Ingen T
        • Joh E
        • et al.
        Characteristics associated with household transmission of SARS-CoV-2 in Ontario, Canada: a cohort study.
        Clin Infect Dis. 2021; (In Press)https://doi.org/10.1093/cid/ciab186
        • Philipson T.
        Economic epidemiology and infectious diseases.
        Handbook of Health Economics. Elsevier, 2000: 1761-1799https://doi.org/10.1016/S1574-0064(00)80046-3
        • Pinheiro JC
        • Bates DM.
        Mixed-effects models in S and SPLUS.
        Springer, New York2000
        • Public Health Agency of Canada
        Canadian COVID-19 vaccination coverage report.
        Public Health Agency of Canada, Ottawa2021
        • Public Health England Transmission Group
        Factors contributing to risk of SARS-CoV2 transmission in various settings.
        2020
        • Rao A
        • Ma H
        • Moloney G
        • Kwong JC
        • Juni P
        • Sander B
        • et al.
        A disproportionate epidemic: COVID-19 cases and deaths among essential workers in Toronto.
        Canada. Ann Epidemiol. 2021; (In Press)https://doi.org/10.1016/j.annepidem.2021.07.010
        • Roberts N
        • Andersen D
        • Deal RM
        • Shaffer WA.
        An introduction to delays, in An Introduction to computer simulation: the system dynamics approach.
        Productivity Press, 1994
        • Rowbothum S
        • Conte K
        • Hawe P.
        Variation in the operationalisation of dose in implementation of health promotion interventions: insights and recommendations from a scoping review.
        Implement Sci. 2019; 14: 56https://doi.org/10.1186/s13012-019-0899-x
        • Sun K
        • Wang W
        • Gao L
        • Wang Y
        • Luo K
        • Ren L
        • et al.
        Transmission heterogeneities, kinetics, and controllability of SARS-CoV-2.
        Science. 2021; 371: eabe2424https://doi.org/10.1126/science.abe2424
        • Sundaram ME
        • Calzavara A
        • Mishra S
        • Kustra R
        • Chan AK
        • Hamilton MA
        • et al.
        The individual and social determinants of COVID-19 in Ontario, Canada: a population-wide study.
        CMAJ. 2021; 193: E723-E734https://doi.org/10.1503/cmaj.202608
        • Turbé H
        • Bjelogrlic M
        • Robert A
        • Gaudet-Blavignac C
        • Goldman JP
        • Lovis C.
        Adaptive time-dependent priors and Bayesian inference to evaluate SARS-CoV-2 public health measures validated on 31 countries.
        Front Public Health. 2021; 8583401https://doi.org/10.3389/fpubh.2020.583401
        • Whitelaw TH.
        The practical aspects of quarantine for influenza.
        CMAJ. 1919; 9: 1070-1074
        • World Health Organization Writing Group
        Nonpharmaceutical interventions for pandemic influenza, national and community measures.
        Emerg Infect Dis. 2006; 12: 88-94https://doi.org/10.3201/eid1201.051371