Stochastic interventional approach to assessing immune correlates of protection: Application to the COVE messenger RNA-1273 vaccine trial

Background: Stochastic interventional vaccine efficacy (SVE) analysis is a new approach to correlate of protection (CoP) analysis of a phase III trial that estimates how vaccine efficacy (VE) would change under hypothetical shifts of an immune marker. Methods: We applied nonparametric SVE methodology to the COVE trial of messenger RNA-1273 vs placebo to evaluate post-dose 2 pseudovirus neutralizing antibody (nAb) titer against the D614G strain as a CoP against COVID-19. Secondly, we evaluated the ability of these results to predict VE against variants based on shifts of geometric mean titers to variants vs D614G. Prediction accuracy was evaluated by 13 validation studies, including 12 test-negative designs. Results: SVE analysis of COVE supported post-dose 2 D614G titer as a CoP: estimated VE ranged from 66.9% (95% confidence interval: 36.2, 82.8%) to 99.3% (99.1, 99.4%) at 10-fold decreased or increased titer shifts, respectively. The SVE estimates only weakly predicted variant-specific VE estimates (concordance correlation coefficient 0.062 for post 2-dose VE). Conclusion: SVE analysis of COVE supports nAb titer as a CoP for messenger RNA vaccines. Predicting variant-specific VE proved difficult due to many limitations. Greater anti-Omicron titers may be needed for high-level protection against Omicron vs anti-D614G titers needed for high-level protection against pre-Omicron COVID-19.


Introduction
Stochastic interventional vaccine efficacy (SVE) analysis is a new approach to the assessment of immune correlates of protection (CoPs) based on a phase III vaccine efficacy (VE) trial dataset, recently published in the statistical literature [1].The SVE method is a causal inference-based technique for investigating how VE in an efficacy trial would be expected to shift based on hypothetical perturbations of an immune marker measured at a fixed time point post-vaccination.The degree of hypothetical perturbation is selected by the analyst, facilitating exploration of the immune marker-VE dose-response relationship.This technology transfer article introduces the SVE framework to a non-statistician vaccine development audience, with application to the COVE trial.To the best of our knowledge, this is the first application of the SVE approach to a phase III VE dataset to assess an immune CoP.
Previous immune correlates analyses of phase III VE trials by the US government's COVID-19 Vaccine Correlates of Protection Program [2][3][4][5] used controlled vaccine efficacy (CVE) analysis [6], which assesses how assignment of all participants to vaccination and to a fixed immune marker value (a deterministic intervention) impacts VE (vs placebo).SVE analysis instead considers a stochastic intervention, which specifies that the immune marker of vaccine recipients is shifted from its observed value by a user-supplied shifting function.By framing the hypothetical intervention as a change (or shift) relative to observed immune marker levels of individual vaccine recipients, the SVE approach delineates how VE would be expected to change upon perturbation of an immune marker.The SVE approach recognizes immune response heterogeneity by considering only relative, individual-level shifts while remaining potentially informative for refined vaccines designed to elicit higher immune marker values.
A validated immune CoP [21] is a biomarker that can be used to reliably infer the degree of vaccine protection against a clinical endpoint and applied to establish a basis of vaccine approval (e.g., variant-adapted vaccines) when RCT data are unavailable.A body of evidence supports nAb titer against the index or D614G strain as a CoP for COVID ).These results provide a basis for nAb titers against contemporary SARS-CoV-2 strains for recipients of a variant-adapted vaccine to be used for its authorization or approval, typically through immunobridging studies.In addition, a nAb titer CoP can be used for estimating VE against variants by incorporating data on nAb titers to the variants.Moreover, variant-adapted boosters that better-match newer circulating strains may provide higher protection (vs the original vaccine-strain booster) against symptomatic COVID-19 caused by the newer strains [23].
Using a population-level approach, Cromer et al. [24] modeled how well nAb titers against variants could be used to predict VE against those variants, reporting good prediction based on the mean decrease in nAb titer against the variant strain vs the ancestral strain.Here, we apply the SVE approach using individual-level data from a single RCT, combined with data on nAb titers against variants.Our objectives are, first, to show how the SVE framework helps define nAb titer as a correlate of protection against ancestral strain COVID-19, and, secondly, to evaluate how well the SVE framework can be used to predict two-dose and three-dose VE against variant-specific symptomatic infection based on concordance analysis with direct estimates of VE obtained from TND studies and one RCT.

Stochastic interventional vaccine efficacy analysis
The SVE approach estimates the counterfactual VE in a RCT under relative, individuallevel perturbations of an immune marker measured at a fixed post-vaccination timepoint.The stochastic intervention considers a perturbation δ to the observed immune marker S for vaccine recipients, shifting S to generate a counterfactual immune response S + δ, for an analyst-specified value δ.Under this hypothetical intervention, with A the indicator of receiving vaccine as opposed to placebo and X a set of baseline adjustment covariates (minority demographics indicator, high risk-of-infection indicator, baseline risk score), and under causal inference assumptions noted below, the counterfactual risk of infection or disease outcome (event Y = 1) can be expressed as E[P(Y = 1 S = S + δ, A = 1, X = x) A = 1, X], where counterfactual immune response S + δ is higher (for δ > 0) or lower (for δ < 0) than the observed immune response S. The SVE, itself a function of δ, equals where the term in the denominator is the placebo arm risk (E[] and P[] are expectation and probability operators).Intuitively, the SVE, which depends on the magnitude of the perturbation δ, is one minus a proportion of the mean risk for vaccine recipients assigned the given perturbation and the mean risk for placebo recipients.When the SVE curve increases/ decreases with the degree of perturbation, it indicates that increasing/decreasing immune marker levels increases/decreases vaccine efficacy.
When δ = 0, indicating that the immune marker S is not modified, SVE(δ) equals the overall VE, readily estimable from randomization to vaccine vs placebo.The SVE approach formalizes a framework for considering how hypothetical changes (represented by δ) to an immune marker's level impact VE.Using phase I study data collected on the PsV-nAb ID50 titer against SARS-CoV-2 variants, we use this anchoring in our SVE analysis by setting δ as the mean difference in response between said variants and the D614G virus.Evaluation of SVE(δ) across multiple choices of δ describes how VE is expected to change under different magnitude shifts in an immune marker.For example, when δ is measured as log 10 fold-change in geometric mean PsV-nAb ID50, comparison of SVE(δ = 0) to SVE(δ = 1) informs on how much VE would be expected to change with a refined version of the vaccine regimen that increased PsV-nAb ID50 titer by an order of magnitude.Repeating this process across several choices of δ, then, "traces out" a curve of SVE(δ) vs δ [1].We perform such an analysis, interpreting δ based on differences in PsV-nAb ID50 titers against variants relative to against D614G.The assumptions made in applying the SVE approach to estimate post-dose 2 VE against the COVID-19 primary endpoint of the COVE study under a given mean shift in log10 PsV-nAb ID50 titer are detailed in the Supplementary Material.
The estimated curve SVE(δ) provides a way to predict VE against COVID-19 caused by SARS-CoV-2 variants other than the ancestral virus.Specifically, for a given SARS-CoV-2 variant, let δ be the difference in mean log 10 PsV-nAb ID50 titer against the variant compared to against the D614G strain.The estimate of VE at this shift value provides an estimate of VE against the variant-specific COVID-19.How can this estimate be interpreted?Envisage a 'hypothetical variant trial' that is the same as the actual COVE study in all respects except that we suppose that the variant had been the sole circulating viral lineage during the trial (instead of the ancestral lineage) and that its placebo-arm incidence of COVID-19 was the same as the placebo-arm incidence of ancestral-virus COVID-19 in the actual COVE study (this is a placebo arm scenario for interpretation of results, not an assumption).Then, the estimate of VE against the variant is interpreted as the VE that would have resulted in the hypothetical variant trial under a 'variant-invariant CoP model' assumption (terminology from Jerry Sadoff, personal communication), which can be posited as follows for use of the SVE method: the mean counterfactual risk of D614G-virus-specific COVID-19 when assigning all participants to two doses of the mRNA-1273 vaccine and the δ mean shift in anti-D614G log 10 PsV-nAb ID50 titer in the COVE trial equals the mean risk of variant-specific COVID-19 when assigning all participants to two doses of the mRNA-1273 vaccine (without a shift) in the two-dose hypothetical variant trial.In other words, the ancestral lineage-specific VE at a particular titer level of antibodies against ancestral lineage virus is the same as variant-specific VE against a given variant at the same titer level of antibodies against that variant.We also consider prediction of post-dose 3 VE (vs three doses of placebo) against COVID-19 with a specific variant (see Supplementary Material for assumptions).

Inclusion of vaccine efficacy/effectiveness estimates in the validation analysis
The literature search process and criteria for inclusion of a variant-specific VE estimate in the validation analysis are described in the Supplementary Material.In brief, the validation analysis considered all available studies of an mRNA vaccine that measured SARS-CoV-2 sequences from COVID-19 cases, estimated lineage-specific vaccine effectiveness (vs unvaccinated), and met other criteria for feasibility of comparing results with those from the COVE study.25] and one RCT [8] qualified and were used as comparators.

Neutralizing antibody titers against SARS-CoV-2 variants
Table 1 describes the studies from which mRNA-1273 vaccine recipient serum samples were obtained for nAb assays and the spike variant assayed (Supplementary Table 1).nAb data were measured against 18 different variants, with data from between 10 and 58 participants per variant.Fig. 1a and b show the distribution of PsV-nAb ID50 titers against each variant, for two-dose and three-dose vaccine recipients, respectively; Supplementary Table 2 provides summary statistics of the geometric mean PsV-nAb ID50 titers.In general PsV-nAb ID50 geometric mean titer (GMT) against each variant was lower than corresponding dosespecific GMT against D614G.GMT generally decreased with chronological time of variant emergence defined by date of World Health Organization-Variant Classification, such that antigenic distance increased with successive variant emergence.

Applying stochastic interventional vaccine efficacy analysis to modeling neutralizing antibody titer-predicted 2-dose vaccine efficacy against SARS-CoV-2 variants
Fig. 2a can be applied to predict two-dose VE against variant-specific COVID-19 for each of the nine (non-ancestral) variants characterized against two-dose vaccine recipient sera studied in Table 1.This is achieved by locating the GMT of two-dose vaccine-recipient sera against a given variant on the x-axis and taking the SVE estimate on the y-axis as the predicted VE against that variant.Each of the nine SVE estimates can be interpreted as the estimated vaccine efficacy had that specific variant been the only lineage circulating in the COVE trial and the background/placebo-arm risk of this variant were the same as that for ancestral observed in COVE.VE estimates against variants ranged from 67% (mu) to 86% (lambda) and were all lower than the estimate (93%) against the ancestral strain.VE estimates against kappa and beta were between 73% and 75%, while those against iota, gamma, delta, epsilon, and alpha, ranged between 82% and 84%.
An application of Fig. 3a is predicting three-dose VE against variant-specific COVID-19 for each of the seven variants characterized against three-dose vaccine recipient sera studied in Table 1.Each of the seven SVE estimates can be interpreted as the estimated VE had (1) that specific variant been the only lineage circulating in the COVE trial; (2) the background/placebo-arm risk of this variant been the same as for that observed for ancestral in COVE; and (3) the blinded period of the COVE trial studied three doses of vaccine vs three doses of placebo.The three-dose VE estimates against variants were all lower than the hypothetical three-dose VE estimate (97.3%) against ancestral.The lowest VE estimate (82%) was against omicron BA.4/BA.5.For other omicron variants, the VE estimates ranged from 90% to 85% for BA.1, BA.3, BA.2, and BA.2.12.1 in decreasing order.While these VE estimates may appear to reflect the stronger degree of protection conferred by three doses, this analysis is exploratory, requiring strong comparability (i.e., ceteris paribus) assumptions operationalized under the hypothetical of COVE studying a three-dose regimen during the blinded phase rather than the actual two-dose regimen and an analogous variant-invariant CoP model.

Evaluation of prediction accuracy using vaccine efficacy/effectiveness estimates against variants
We next performed a validation analysis using empirical vaccine efficacy/effectiveness estimates (vs unvaccinated) against variants from external studies (Table 2).For two-dose vaccination (Fig. 2b), the Spearman rank correlation between predicted and empirical VE estimates was 0.143.The predictions tend to be underestimates, with low concordance: Concordance Correlation Coefficient (CCC) equal to 0.062.For three-dose vaccination (Fig. 3b), the Spearman rank correlation was 0.894; while this value is high, concordance was weak, and CCC was 0.017.The predictions tend to be overestimates, with predicted VEs ranging from 82-90% and empirical VE estimates from 40-60%.

Reproducibility statement
All aspects of the statistical analysis were conducted using the open-source R language and environment for statistical computing and graphics (version 4.0.4).Code for replicating the analysis will be posted publicly on GitHub.

Discussion
Our first objective was to evaluate post-dose 2 pseudovirus neutralizing antibody titer (PsV-nAb ID50) against D614G as a correlate of protection for the COVID-19 primary endpoint in the COVE study via the SVE approach.SVE sharply increased under stochastic shifts of geometric mean PsV-nAb ID50 titer upwards, and sharply decreased under downward shifts, further supporting the PsV-nAb ID50 biomarker as a correlate of protection [2].The SVE approach has the advantage that it is readily possible to define meaningful potential population shifts of an immune marker distribution, because clinical studies of vaccine recipients can be used to define shifts that can be potentially produced by modifications of the vaccine regimen.Moreover, while our data analysis only considered shifts applied uniformly to all study participants, these shifts can depend on participant factors, such that the framework can accommodate situations where some shifts are possible for some sub-populations but not others.For example, higher immune marker levels may be attainable for younger vs older individuals, or for individuals seropositive vs seronegative to the pathogen under study.Given a division of the cohort into two subgroups, separate specified shifts δ 1 and δ 2 can be assigned to the subgroups, and a similar statistical analysis applied to estimate SVE(δ 1 , δ 2 ), which is vaccine efficacy under the δ 1 perturbation assigned subgroup 1 and the δ 2 perturbation assigned subgroup 2.
While our analysis focused on a COVID-19 VE trial, the SVE framework is equally relevant and applicable to an arbitrary VE trial for a given vaccine regimen and pathogen.However, details of the vaccine regimen, pathogen, immunoassay, and most importantly the biomarker derived from the immunoassay provide critical context for specifying the relevant marker shifts or shift functions for addressing particular objectives.For example, consider one biomarker with very broad dynamic range across vaccine recipients with quantitative values for all vaccine recipients, compared to another biomarker for which 50% of vaccine recipients have marker value before the assay lower limit of detection (LOD).For the latter biomarker, applying SVE analysis for a grid of constant shifts ranging from a decrease to the 5 th percentile to an increase to the 95 th percentile may be of interest, whereas for the latter biomarker, this analysis would not be useful and a more relevant approach would specify separate shift functions to the two subgroups with marker value below vs above the LOD.
For another example, for a pathogen with limited diversity, the marker typically is a response to the vaccine antigen, where if the pathogen has high diversity, then the marker may be a cross-reactivity score to a panel of antigens.
Our second, totally distinct, objective evaluated how well SVE estimates from COVE could predict 2-dose mRNA VE against COVID-19 caused by different SARS-CoV-2 variants in the hypothetical context that a given variant had circulated in COVE.The SVE estimates were weakly concordant with empirical two-dose variant-specific VE estimates from external studies and tended to underestimate empirical VE.The relative lack of randomized, placebo-controlled trials (RCTs) providing empirical estimates of VE for this validation analysis partly explains the underestimation, with all but one of the studies being an observational test-negative design (TND), which, compared to RCTs, have been found to produce biased-upward estimates of VE against 26].The ability of TND studies to unbiasedly estimate VE requires assumptions including that vaccination does not impact risk of other infectious diseases resulting in similar symptomatic illness, that individuals meeting the COVID-19 endpoint symptom criteria would not be more or less likely to seek COVID-19 testing if they were vaccinated, and that there is correct adjustment for confounding factors occurring in the absence of randomization to vaccine vs placebo.Especially for the three vaccine-dose TND studies [11,14,20], the comparator group of unvaccinated persons typically made up a small and highly selected proportion of the relevant population for inference.Moreover, the 95% CIs for vaccine effectiveness estimates from the TND studies are generally too narrow, reflecting uncertainty due to sampling variability but not accounting for other sources of uncertainty [27].Other reasons for weak predictions are discussed in the Supplementary Material.
Our evaluation of how well SVE estimates could predict three-dose VE against omicron COVID-19 yielded overestimates.This over-estimation cannot be explained by the lack of RCTs.One potential explanation is that omicron may have a greater average challenge dose and/or greater viral infectivity (discussed in the Supplementary Material).However, we emphasize that the predictions of three-dose VE are only exploratory and make additional extrapolation assumptions beyond those made for the two-dose VE predictions.Cromer et al. [26] found that a neutralization-CoP modeling approach performed well in predicting empirical vaccine effectiveness/VE against symptomatic COVID-19 caused by variants in TND, case-control, and RCT studies.Their modeled VE estimates had high correlation with empirical VE, with slight underestimation against non-omicron variants and overestimation against omicron beyond 3 months post-vaccination, consistent with our results.While both approaches are based on fold-change in geometric mean neutralization titer of vaccine recipient sera against a variant vs the index strain, our approach is based on one phase III study and one neutralization assay, while Cromer et al. [26]'s is based on different neutralization assays and used the geometric mean of convalescent sera to standardize readouts across studies.Moreover, we used nonparametric modeling of individual-level data while Cromer et al. used parametric modeling of study-level data.
Additional differences are detailed in the Supplementary Material.While our approach is theoretically statistically optimal for robustness and efficiency [1], the limited sample size of the omicron validation studies hinders performance comparison.
We draw two conclusions for the second objective, the first being the above-noted evidence against the postulated "variant-invariant CoP model."Based on the low concordance of SVE estimates with empirical variant-specific VE estimates, our second conclusion is that the prediction modeling ventured a "bridge too far," caused in part by the hypothetical context for the predictions, the limited accuracy and precision of variant-specific VE estimates in the TND observational studies as noted above, and variations among the TND studies.Another reason may be that our study did not directly assess a 'variant-matched correlate' that would associate variant-specific titer with incidence of variant-matched COVID-19, as nAb assay data against D614G may have limited ability to predict efficacy against other variants.This hypothesis will be tested in future work.In contrast, it was much easier to attain success for our first objective, given that it is based wholly on data directly observed in COVE with fewer assumptions and avoiding extrapolations.We conclude that the SVE framework is a promising new approach for predicting how VE would change under specified perturbations of an immune marker's distribution, where prediction domain application details may determine success.One potential future application is modeling predicted improvement in VE with updated vaccine-insert sequence(s).PsV-nAb ID50 titers of serum from recipients of (a) two or (b) three doses of mRNA-1273 against different SARS-CoV-2 variants.PsV-nAb ID50 titers of serum samples drawn 4 weeks after two or three doses of the mRNA-1273 vaccine (clinical studies detailed information in Table 1) were assessed against variant spike-pseudotyped viruses.The dotted horizontal line represents the geometric mean PsV-nAb ID50 against the ancestral strain D614G.Each dot represents one participant.Numbers in parentheses after each variant name are the sample sizes.Supplementary Table 2 provides summary statistics of the geometric mean PsV-nAb ID50 titers.GM, geometric mean; LLOD, lower limit of detection; mRNA, messenger RNA; PsV-nAB, pseudovirus neutralizing antibody; ULOQ, upper limit of quantitation.IU50/ml, International Units 50/ml (calibrated to the D614G strain World Health Organization International Standard 20/136 for anti-SARS-CoV-2 immunoglobulin).For two-dose mRNA-1273 recipients, stochastic interventional vaccine efficacy estimates under hypothetical shifts of neutralizing antibody titer, with application to predict vaccine efficacy against different SARS-CoV-2 variants, with validation analysis from external studies of two vaccine doses.(a) Y-axis: Estimated vaccine efficacy against virologically confirmed, symptomatic COVID-19 disease for a vaccine that elicits geometric mean 4week post dose 2 PsV-nAb ID50 titer against the ancestral strain with the value indicated on the x-axis, estimated using the method in Hejazi et al. [1].ID50 titer = IU/ml because ancestral strain titers were calibrated to the World Health Organization International Standard 20/136 D614G strain for anti-SARS-CoV-2 immunoglobulin.The circles, squares, and diamonds are the SVE estimates, with 95% confidence intervals (dashed lines), for a vaccine that hypothetically shifts each vaccine recipient's log 10 nAb ID50 titer by a constant amount (for the black square, δ = 0) from their observed value against D614G that yields the given geometric mean D57 PsV-nAb ID50 titer against the designated variant (see Methods).For the diamonds, antigen panel data (Table 1) were used to estimate the actual mean shift in log 10 PsV-nAb ID50 titer for a response against the D614G strain as opposed to against the designated SARS-CoV-2 variant.SVE estimates are reported for the follow-up period 7 through 100 days post day 57.2. The single reported VE estimate from an RCT (Moderna COVE), whose symbol is differentiated by a dark outline, is the average (taken on the log scale and back-transformed to the VE scale) of the three variant-specific VE estimates in the Moderna COVE study.The solid black line is the y = x line.The Andrews et al. [11,12] TND estimates, which did not specify Omicron sublineage, were plotted against SVE results for Omicron BA.1, based on available genomic epidemiology data (see the Supplementary Material).BNT162b2 vaccine efficacy/ effectiveness estimates were rescaled based on the difference from mRNA-1273 vaccine efficacy/effectiveness estimates (see details in Methods).For simplicity in plotting, both mRNA doses in the "any mRNA" regimens in Skowronski et al. [19] were assumed to be mRNA-1273, and no rescaling was performed.LLOD, lower limit of detection; mRNA, messenger RNA; PsV-nAB, pseudovirus neutralizing antibody; RCT, randomized, placebo-controlled efficacy trial; SVE, stochastic interventional vaccine efficacy; TND, test-negative design; VE, vaccine efficacy/ effectiveness.For three-dose mRNA-1273 recipients, stochastic interventional vaccine efficacy modeling of neutralization titer-predicted vaccine efficacy against different SARS-CoV-2 variants and validation analysis from external studies of three vaccine doses.(a) The y-axis plots the estimated vaccine efficacy against virologically confirmed, symptomatic COVID-19 disease for a vaccine that elicits geometric mean D29 post dose 3 PsV-nAb ID50 titer against the ancestral strain with the value indicated on the x-axis, estimated using the method in Hejazi et al. [1].ID50 titer = IU/ml because ancestral strain titers were calibrated to the World Health Organization International Standard 20/136 D614G strain for anti-SARS-CoV-2 immunoglobulin.The black dots are the SVE estimates, with 95% confidence intervals (dashed lines), for a vaccine that hypothetically shifts each vaccine recipient's log 10 nAb ID50 titer by a constant amount from their observed value against D614G that yields the given geometric mean D57 PsV-nAb ID50 titer against the designated variant (see Methods).For the colored diamonds, available phase I data (Table 1) are used to estimate the actual mean shift in geometric mean PsV-nAb ID50 titer for a response against D614G as opposed to against the designated SARS-CoV-2 variant.SVE estimates are reported for the follow-up period 7 through 100 days post day 57.The vertical green bars indicate the observed distribution of PsV-nAb ID50 titer as measured from participants post dose 2 in the COVE trial.The diagonal dashed line reports a linear model summary of how SVE(δ) is expected to change with log 10 geometric mean titer shifts (δ) (see the Supplementary Text for further details).(b) Comparison of VE estimates against SARS-CoV-2 infection or symptomatic COVID-19 caused by different SARS-CoV-2 variants obtained in TND studies of index-strain COVID-19 mRNA vaccines compared to the SVE estimates in panel A. Estimates are for after three doses of mRNA vaccine (mRNA-1273 and/or BNT162b2) vs placebo.The reported VE estimate for each plotted symbol corresponds to the average (taken on the log scale and back-transformed to the VE scale) of all three-dose variantspecific VE estimates from the three studies listed in Table 2 that reported these estimates.The solid black line is the y = x line.The Andrews et al. [11,12] TND estimates, which did not specify Omicron sublineage, were plotted against the SVE results against Omicron BA.1, based on the available genomic epidemiology data for SARS-CoV-2 variants (see Supplementary Material).BNT162b2 vaccine efficacy/effectiveness estimates were rescaled based on the difference from mRNA-1273 vaccine efficacy/effectiveness estimates (see details in Methods).For simplicity in plotting, the heterologous three-dose regimens were treated as three-dose mRNA-1273 regimens, with no rescaling performed.LLOD, lower limit of detection; mRNA, messenger RNA; PsV-nAB, pseudovirus neutralizing antibody; SVE, stochastic interventional vaccine efficacy; TND, test-negative design; VE, vaccine efficacy/effectiveness.For the delta variant only, vaccine effectiveness of 2 doses of mRNA-1273 was reported for the time intervals 14-60 days, 61-90 days, 91-120 days post dose 2 (Table S12 in Bruxvoort et al.).The average of the VE point estimates and the average of the 95% confidence limits [averaged on the log(1-VE) scale] over the three time intervals is reported as the vaccine effectiveness estimate against delta (right hand column) of this table..

c
Data were combined to determine a single geometric mean titer shift for alpha in Fig.2a.d Data were combined to determine a single log 10 geometric mean titer shift value of delta in Fig.2a.
-negative design; RCT = randomized controlled trial.* Only variant-specific vaccine efficacy/vaccine effectiveness estimates meeting the eligibility criteria for inclusion in the validation analysis (detailed in the supplementary material) are listed here.* * Potential significant heterogeneity in the interval between first and second dose due to e.g.adoption of delayed second-dose strategy.The dose interval could extend up to 12 weeks in some cases.A Laboratory-confirmed SARS-CoV-2 infection detected by real-time reverse transcription polymerase chain reaction.B FromTable 2 in Pajon et al.C PCR-confirmed SARS-CoV-2 infection with symptoms consistent with COVID-19 (high temperature, new continuous cough, or loss or change in sense of smell or taste).D See Figure S3 in Lopez Bernal et al.; all COVID-19 endpoints were between 14 days to 140 days post dose 2. E Overall reported vaccine effectiveness estimates from Table 2 in Lopez Bernal et al.F With or without symptoms; positive SARS-CoV-2 RT-PCR test.G mRNA-1273 was first made available on Dec 18, 2020.With second doses on approximately Jan 15, 2021 and data on tests from Mar 1, 2021 through Jul 27, 2021, follow up period post dose 2 extended from 45 days through approximately 6.5 months.H The diagonal dashed line gives a linear model summary of how SVE(δ) is expected to change with log 10 geometric mean titer shifts (δ) (see the Supplementary Text for further details).The vertical green bars indicate the observed distribution of PsV-nAb ID50 titer as measured from participants post dose 2 in the COVE trial.(b) Comparison of VE estimates against SARS-CoV-2 infection or symptomatic COVID-19 caused by different SARS-CoV-2 variants obtained in a phase III RCT or in TND studies of index-strain COVID-19 mRNA vaccines compared to SVE estimates in panel A. Estimates are for after two doses of mRNA vaccine (mRNA-1273 and/or BNT162b2) vs placebo.The reported VE estimate for each plotted symbol corresponds to the average (taken on the log scale and back-transformed to the VE scale) of the available variant-specific VE estimates from the 13 studies listed in Table

Table 1
Studies from which serum samples were obtained for nAb assays, variants assayed, and sample sizes.

with serum samples Trial phase Vaccine (no. doses) Sampling timepoint for assay Spike variant/subvariant used in the nAb assay a No. participants with nAb ID50 data
The Beta version used for testing phase III samples contains 246R in Spike, whereas the Beta used to test phase I samples contains an R246I mutation.No difference was found in neutralizing susceptibility between the two versions of Beta (246R vs 246I) (Montefiori lab, unpublished).

Table 2
Empirical variant-specific vaccine efficacy/vaccine effectiveness estimates obtained from test-negative design observational studies and from one randomized controlled trial used in the validation analysis *