Advertisement

Prediction of 28-days mortality with sequential organ failure assessment (SOFA), quick SOFA (qSOFA) and systemic inflammatory response syndrome (SIRS) — A retrospective study of medical patients with acute infectious disease

  • Author Footnotes
    1 Shahin Gaini and Mette Marie Relster share 1st authorship.
    Shahin Gaini
    Correspondence
    Corresponding author at: Infectious Diseases Division, Medical Department, National Hospital Faroe Islands, JC. Svabosgøta 41-49, FO-100 Tórshavn, Faroe Islands.
    Footnotes
    1 Shahin Gaini and Mette Marie Relster share 1st authorship.
    Affiliations
    Department of Infectious Diseases, Odense University Hospital and University of Southern Denmark, Odense, Denmark

    Infectious Diseases Division, Medical Department, National Hospital Faroe Islands, Tórshavn, Faroe Islands

    Centre of Health Research and Department of Science and Technology, University of the Faroe Islands, Tórshavn, Faroe Islands
    Search for articles by this author
  • Author Footnotes
    1 Shahin Gaini and Mette Marie Relster share 1st authorship.
    Mette Marie Relster
    Footnotes
    1 Shahin Gaini and Mette Marie Relster share 1st authorship.
    Affiliations
    Department of Infectious Diseases, Odense University Hospital and University of Southern Denmark, Odense, Denmark
    Search for articles by this author
  • Court Pedersen
    Affiliations
    Department of Infectious Diseases, Odense University Hospital and University of Southern Denmark, Odense, Denmark
    Search for articles by this author
  • Isik Somuncu Johansen
    Affiliations
    Department of Infectious Diseases, Odense University Hospital and University of Southern Denmark, Odense, Denmark
    Search for articles by this author
  • Author Footnotes
    1 Shahin Gaini and Mette Marie Relster share 1st authorship.
Open AccessPublished:September 26, 2018DOI:https://doi.org/10.1016/j.ijid.2018.09.020

      Highlights

      • SIRS versus Sepsis-3 regarding mortality in medical patients with acute infectious disease.
      • SOFA score of at least 2 predicts 28-day mortality in medical patients with acute infectious disease.
      • SOFA score of at least 2 is a better prognostic marker than qSOFA and SIRS in medical patients with acute infectious disease.

      Abstract

      Aims

      Evaluating the use of sequential organ failure assessment (SOFA) ≥ 2 compared to quick SOFA (qSOFA) and to systemic inflammatory response syndrome (SIRS) in assessing 28-days mortality in medical patients with acute infection.

      Methods

      In total, 323 patients with verified infection were stratified in accordance to Sepsis-3. SOFA, qSOFA and SIRS were calculated using registered variables. Adverse outcome was death within 28-days of admission.

      Results

      In total, 190 (59%) patients had a SOFA score ≥ 2 and the overall in-hospital mortality was 21 (6%). Scores of SOFA and qSOFA were both significantly elevated in non-survivors. SOFA showed good accuracy (Area under the receiver operating characteristic (AUROC) = 0.83, 95% CI, 0.76 - 0.90) for 28-days mortality compared with qSOFA (AUROC = 0.67, 95% CI, 0.54 - 0.80) and SIRS (AUROC = 0.62, 95% Cl 0.49 - 0.74). SOFA was ≥ 2 in all patients who died, while qSOFA and SIRS was ≥ 2 in 8 (38%) and 17 (81%) of the patients who died, respectively.

      Conclusion

      SOFA score ≥ 2 was better than SIRS and qSOFA to predict mortality within 28-days of admission among patients with acute infectious disease.

      Abbreviations:

      ALAT (alaninaminotransferase), AUROC (area under the receiver operating characteristic), BP (blood pressure), Cl (confidence level), CRP (C-reactive protein), ED (emergence department), GCS (Glasgow coma scale), HW (hospital wards), ICU (intensive care unit), IQR (inter quartile range), n (number of patients), ND (not determined), ns (not significant), OD (organ dysfunction), P (prospective), PCT (procalcitonin), qSOFA (quick SOFA), R (retrospective), RR (respiratory rate), SatO2 (oxygen saturation), SEN (sensitivity), SI (suspected infection), SIRS (systemic inflammatory response syndrome), SOFA (sequential organ failure assessment), SPE (specificity), VI (verified infection)

      Keywords

      Introduction

      Sepsis is a life-threatening and complex disease with a pathobiology formed by factors relating to both the pathogen involved and the host (
      • Angus D.C.
      • van der Poll T.
      Severe sepsis and septic shock.
      ,
      • Deutschman C.S.
      • Tracey K.J.
      Sepsis: current dogma and new perspectives.
      ,
      • Mayr F.B.
      • Yende S.
      • Angus D.C.
      Epidemiology of severe sepsis.
      ,
      • Wiersinga W.J.
      • Leopold S.J.
      • Cranendonk D.R.
      • et al.
      Host innate immune responses to sepsis.
      ). For more than two decades, characterization of sepsis has focused on inflammatory excess and been defined as an infection with at least two of four systemic inflammatory response syndrome (SIRS) criteria (
      • Bone R.C.
      • Balk R.A.
      • Cerra F.B.
      • et al.
      Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine.
      ). Yet, the current use of SIRS might not necessarily identify patients with severe disease (
      • Churpek M.M.
      • Zadravecz F.J.
      • Winslow C.
      • et al.
      Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients.
      ).
      New international definitions termed Sepsis-3, defines sepsis as a life-threatening organ dysfunction caused by a dysregulated host response to infection, and septic shock as a subset of sepsis in which underlying circulatory and cellular metabolism abnormalities are profound enough to substantially increase mortality (
      • Shankar-Hari M.
      • Phillips G.S.
      • Levy M.L.
      • et al.
      Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Singer M.
      • Deutschman C.S.
      • Seymour C.W.
      • et al.
      The third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ). Along with the definitions, clinical guidelines proposed by Sepsis-3 recommend the use of a total sequential organ failure assessment (SOFA) score above or equal to two in patients with verified infection to identify a patient with sepsis. Acknowledging the need to identify patient with sepsis prior to available laboratory test results required for calculation of SOFA, a new readily and inexpensively assessed bedside clinical score, termed quick SOFA (qSOFA), was presented in the Sepsis-3. The qSOFA score ranges from zero to three points given for each of the clinical variables, respiratory rate (RR) ≥ 22 breaths/min, Glasgow Coma Scale (GCS) < 15 and systolic blood pressure ≤ 100 mm Hg (
      • Shankar-Hari M.
      • Phillips G.S.
      • Levy M.L.
      • et al.
      Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Singer M.
      • Deutschman C.S.
      • Seymour C.W.
      • et al.
      The third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ).
      The initial and large retrospective study, which correlated patients hospitalized with suspicion of infection and a SOFA score ≥ 2, found a 2- to 25-fold increased risk of in-hospital mortality for patients outside the ICU, depending on the patients’ baseline level of risk, for SOFA ≥ 2 compared with SOFA < 2 (
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ). For patients in the ICU, in-hospital mortality increased with 3- to 11-fold for SOFA ≥ 2 compared with SOFA < 2.
      Moreover, the study showed that qSOFA ≥ 2 was related to in-hospital mortality, but was unable to predict mortality of patients in ICU (
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ). Because qSOFA seems less robust than SOFA in ICU, Sepsis-3 recommends the use of the qSOFA outside the ICU as an early score to initiate investigation for organ dysfunction and to direct appropriate clinical management (
      • Singer M.
      • Deutschman C.S.
      • Seymour C.W.
      • et al.
      The third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ).
      The objective of the present study was to retrospectively evaluate the Sepsis-3 recommended use of SOFA ≥ 2 to evaluate risk of mortality within 28-days of admission in a broad spectrum of prospectively included patients with verified infection from a medical non-ICU ward after initial assessment between 24 and 48 h after admission.

      Patients and methods

      Study population

      The present study was based on a post hoc retrospective analysis of patient data (n = 323) from three previously conducted prospective studies regarding patients with varying degrees of infection ranging from infection without sepsis, sepsis, severe sepsis and septic shock according to the SIRS criteria (Figure 1) (
      • Gaini S.
      • Koldkjaer O.G.
      • Moller H.J.
      • et al.
      A comparison of high-mobility group-box 1 protein, lipopolysaccharide-binding protein and procalcitonin in severe community-acquired infections and bacteraemia: a prospective study.
      ,
      • Gaini S.
      • Koldkjaer O.G.
      • Pedersen C.
      • et al.
      Procalcitonin, lipopolysaccharide-binding protein, interleukin-6 and C-reactive protein in community-acquired infections and sepsis: a prospective study.
      ,
      • Gaini S.
      • Pedersen S.S.
      • Koldkaer O.G.
      • et al.
      New immunological serum markers in bacteraemia: anti-inflammatory soluble CD163, but not proinflammatory high mobility group-box 1 protein, is related to prognosis.
      ).
      Figure 1
      Figure 1Flowchart of the clinical criteria for identification and stratification of the study population according to Sepsis-3 guidelines. The diagnosis of infection was made by only one physician, who was blinded to all biochemical results. The state of infection was assessed by the clinical and para-clinical results and was categorized (1) no infection, (2) verified infection or (3) unknown. n: number of patients, SOFA: Sequential organ failure assessment.
      In short, patients were included at the Department of Internal Medicine at Odense University Hospital, a large tertiary Danish University Hospital, by referral either from a general practitioner or from the emergency ward. Enrolment criteria for two of the studies were: suspicion of infection by the doctor in charge and initiation of empirical treatment with antibiotics (
      • Gaini S.
      • Koldkjaer O.G.
      • Moller H.J.
      • et al.
      A comparison of high-mobility group-box 1 protein, lipopolysaccharide-binding protein and procalcitonin in severe community-acquired infections and bacteraemia: a prospective study.
      ,
      • Gaini S.
      • Koldkjaer O.G.
      • Pedersen C.
      • et al.
      Procalcitonin, lipopolysaccharide-binding protein, interleukin-6 and C-reactive protein in community-acquired infections and sepsis: a prospective study.
      ). In the third study, patients with bacteremia were enrolled after verified positive blood cultures (
      • Gaini S.
      • Pedersen S.S.
      • Koldkaer O.G.
      • et al.
      New immunological serum markers in bacteraemia: anti-inflammatory soluble CD163, but not proinflammatory high mobility group-box 1 protein, is related to prognosis.
      ). For all three studies, the main exclusion criteria were age <18 years, earlier participation in the study and prior hospitalization within seven days of admission. The patients received a standard of care in agreement with the departmental guidelines.

      Ethics

      Data and blood sample collection in the three original studies were done in accordance with the Declaration of Helsinki and ethical allowances were obtained from the Scientific Ethical Committee of Southern Denmark (Funen and Vejle Counties, file number 20010076) and from the Danish Data Security Board (2003). Written informed consent was obtained from the patients or closest relatives in each case.
      After completion of the original studies, all data and samples were completely and irrevocably anonymized. According to Danish law, research on anonymized data can be done without obtaining new approval from the Ethical Committee and data Protection Agency if the original data collection was done in accordance with all rules.

      Stratification and characterization of study population

      Patients with verified infection (
      • Gaini S.
      • Koldkjaer O.G.
      • Moller H.J.
      • et al.
      A comparison of high-mobility group-box 1 protein, lipopolysaccharide-binding protein and procalcitonin in severe community-acquired infections and bacteraemia: a prospective study.
      ,
      • Gaini S.
      • Koldkjaer O.G.
      • Pedersen C.
      • et al.
      Procalcitonin, lipopolysaccharide-binding protein, interleukin-6 and C-reactive protein in community-acquired infections and sepsis: a prospective study.
      ,
      • Gaini S.
      • Pedersen S.S.
      • Koldkaer O.G.
      • et al.
      New immunological serum markers in bacteraemia: anti-inflammatory soluble CD163, but not proinflammatory high mobility group-box 1 protein, is related to prognosis.
      ), were stratified in accordance with the Sepsis-3 guidelines (
      • Shankar-Hari M.
      • Phillips G.S.
      • Levy M.L.
      • et al.
      Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Singer M.
      • Deutschman C.S.
      • Seymour C.W.
      • et al.
      The third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ) in two subgroups: 1) patients with a SOFA score <2 (no sepsis) and 2) patients with a SOFA score ≥ 2 (sepsis). Information on demography, biochemical parameters, comorbidity, disease severity, hematology and outcome were available (
      • Gaini S.
      • Koldkjaer O.G.
      • Moller H.J.
      • et al.
      A comparison of high-mobility group-box 1 protein, lipopolysaccharide-binding protein and procalcitonin in severe community-acquired infections and bacteraemia: a prospective study.
      ,
      • Gaini S.
      • Koldkjaer O.G.
      • Pedersen C.
      • et al.
      Procalcitonin, lipopolysaccharide-binding protein, interleukin-6 and C-reactive protein in community-acquired infections and sepsis: a prospective study.
      ,
      • Gaini S.
      • Pedersen S.S.
      • Koldkaer O.G.
      • et al.
      New immunological serum markers in bacteraemia: anti-inflammatory soluble CD163, but not proinflammatory high mobility group-box 1 protein, is related to prognosis.
      ). Primary outcome was mortality within 28-days of admission. Comorbidity was evaluated with the Charlson index (
      • Gaini S.
      • Koldkjaer O.G.
      • Moller H.J.
      • et al.
      A comparison of high-mobility group-box 1 protein, lipopolysaccharide-binding protein and procalcitonin in severe community-acquired infections and bacteraemia: a prospective study.
      ,
      • Gaini S.
      • Koldkjaer O.G.
      • Pedersen C.
      • et al.
      Procalcitonin, lipopolysaccharide-binding protein, interleukin-6 and C-reactive protein in community-acquired infections and sepsis: a prospective study.
      ,
      • Gaini S.
      • Pedersen S.S.
      • Koldkaer O.G.
      • et al.
      New immunological serum markers in bacteraemia: anti-inflammatory soluble CD163, but not proinflammatory high mobility group-box 1 protein, is related to prognosis.
      ). Disease severity was classified according to SOFA scores (<2 and ≥2) (
      • Shankar-Hari M.
      • Phillips G.S.
      • Levy M.L.
      • et al.
      Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Singer M.
      • Deutschman C.S.
      • Seymour C.W.
      • et al.
      The third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ), qSOFA scores (<2 and ≥2) (
      • Shankar-Hari M.
      • Phillips G.S.
      • Levy M.L.
      • et al.
      Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Singer M.
      • Deutschman C.S.
      • Seymour C.W.
      • et al.
      The third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ) and SIRS criteria (<2 and ≥2) (
      • Bone R.C.
      Toward an epidemiology and natural history of SIRS (systemic inflammatory response syndrome).
      ). Severe sepsis was defined as SIRS ≥ 2 in addition to one or several indices of organ dysfunction as described previously (
      • Gaini S.
      • Pedersen S.S.
      • Koldkaer O.G.
      • et al.
      New immunological serum markers in bacteraemia: anti-inflammatory soluble CD163, but not proinflammatory high mobility group-box 1 protein, is related to prognosis.
      ).

      Statistical analysis

      Data were analyzed with Graphpad Prism 5.01 (GraphPad software Inc., La Jolla, CA, USA). Distributions of datasets were determined by D'Agostino & Pearson omnibus normality test. Continuous variables for the stratified subgroups of patients were compared with non-parametric tests and described as medians with interquartile ranges (IQR). For multi-group comparisons, we used Kruskal-Wallis One-way analysis of variance and for two-group comparisons, Mann-Whitney U test were used. Categorical variables for stratified groups were described as percentages. The prognostic performance was assessed using area under the receiver operating characteristic (AUROC) analysis. We considered AUROCs to be poor at 0.6 to 0.70, adequate at 0.7 to 0.8, good at 0.8 to 0.9, and excellent for AUROCs ≥ 0.9. P-values less than 0.05 were considered statistically significant.

      Results

      Clinical characteristics of the study population

      A total of 190 (59%) patients were classified with sepsis (Figure 1). From patient characteristics, patients diagnosed with sepsis were significantly older, and showed increased comorbidity and numbers of prescribed drugs compared to those without sepsis (Table 1). Severity defined as SOFA scores ≥ 2 was significantly correlated with increasing percent of patients with qSOFA scores ≥ 2 (p < .0001) and SIRS criteria ≥ 2 (p < .01). Diagnosis of sepsis was associated with mortality within 28-days of admission, in addition to deterioration of both physiological, hematological, and biochemical parameters (Table 1).
      Table 1Patient Characteristics.
      Verified Infection (n = 323)No Sepsis

      SOFA < 2 (n = 133)
      Sepsis

      SOFA ≥ 2 (n = 190)
      P-value
      Demography and comorbidity
      Age [year]66 (50; 78)57 (42; 67)73 (59; 82)<0.0001
      Female, n [%]158 (49)74 (55)84 (44)<0.01
      Glasgow coma scale15 (15; 15)d15 (15; 15)d15 (15; 15)ens
      Charlson index score1 (0; 2)0 (0; 2)1 (0; 3)<0.0001
      Prescribed drugs*3 (0; 6)2 (0; 5)4 (1; 7)<0.0001
      Disease severity and Physiology
      SOFA ≥ 2 n [%]190 (59)
      Quick SOFA ≥ 2 n [%]41 (13)2 (2)39 (21)<0.0001
      SIRS ≥ 2 n [%]234 (72)88 (66)146 (77)<0.01
      Systolic BP, [mm Hg]130 (110; 140)a130 (120; 145)b121 (105; 140)a<0.01
      Diastolic BP, [mm Hg]70 (60; 80)b80 (70; 80)b70 (60; 80)b<0.01
      Respiratory rate, [beats/min]24 (20; 30)f22 (19; 28)f28 (20; 32)fns
      Temperature, [°C]39 (38; 39)a38 (38; 39)a38.6 (38; 39)ns
      Hematology and biochemistry
      Hemoglobin, [mmol/L]8 (7; 9)b8 (7; 9)c8 (7; 9)b
      Hematocrit, [%]0.4 (0.4; 0.4)e0.4 (0.4; 0.4)e0.4 (0.4; 0.4)e
      White blood cells, [x10^9/]12 (8; 16)a11 (8; 14)a13 (8; 17)<0.01
      Neutrophils, [x10^9/L]10 (7; 14)c8 (6; 13)c11 (7; 15)b<0.0001
      PCT, [mg/L]1 (0.2; 7)c0.2 (0.09; 2)c3 (0.5; 12)c<0.0001
      CRP, [mg/L]181 (99; 277)b142 (77; 230)c201 (107; 310)a<0.01
      Platelets, [x10^9/L]226 (169; 316)c269 (207; 367)d195 (140; 293)c<0.0001
      Factors II − VII − X, [Ux10^3/L]0.8 (0.6; 1.0)d0.9 (0.8; 1)d0.8 (0.6; 1)d<0.01
      ALAT, [U/L]26 (16; 44)c25 (14; 43)c26 (16; 49)cns
      Bilirubin, [μmol/L]11 (8; 16)d10 (7; 13)d13 (9; 20)d<0.0001
      Creatinine, [μmol/L]102 (87; 131)b88 (78; 96)c125 (103; 180)a<0.0001
      Outcome
      ICU admission, n [%]36 (11)6 (4.5)30 (16)<0.01
      28 – day mortality, n [%]21 (7)0 (0)21 (11)<0.0001
      Demography, comorbidity, disease severity, hematology, biomarker levels and outcome in 323 patients stratified according to Sepsis-3 guidelines. Data are presented as medians with 25th and 75th percentiles in parentheses and units in square brackets, unless otherwise stated. Missing values are indicated by roman numbers: a) <1%, b) <2%, c) <4%, d) <10%, e) <15% and f) >15%. Mann-Whitney test for equality: Sepsis – No sepsis. *: Number of prescribed drugs for chronic diseases prior to admission. n: number of patients, ns: not significant, SOFA: Sequential organ failure assessment, SIRS: Systemic inflammatory response syndrome, BP: Blood pressure, SatO2: Oxygen saturation, PCT: Procalcitonin, CRP: C-reactive protein, ALAT: Alaninaminotransferase, ICU: Intensive care unit.
      Data for RR and GCS was missing for 120 (37%) and 30 (9%) of the patients, respectively. 35%, 27% and 51% of patients included for this study from the three original studies had missing RR values. 9%, 14% and 4% of patients included for this study from the three original studies had missing GCS values. We found that 46 (14%) of the patients might have been misclassified as having qSOFA <2 based on either 1) missing values for both RR and GCS or 2) one missing value and a qSOFA score = 1. For SIRS criteria, 38 (12%) of the patients might have been misclassified as having SIRS < 2 based missing values for RR. The effect of misclassified values was evaluated by repetition of all analyses after exclusion of potentially misclassified patients (Supplementary File S1). In total, 66 (20%) patients were excluded, but the results did not change significantly. Except that the difference in SIRS between non-survivors and survivors only just reached significance with a p-value of 0.04, probably reflecting the lower power of the estimate.
      Comparison of the Sepsis-1 and Sepsis-3 definitions for classification of patients with verified infection.
      Forty-five (14%) of the entire study population met none of the criteria for sepsis, while 146 (45%) patients met the criteria in accordance with both Sepsis-1 and Sepsis-3. A total of 44 (14%) patients met only the Sepsis-3 criteria, and 88 (27%) met only the Sepsis-1 criteria. Among the 124 patients classified as having severe sepsis according to Sepsis-1 definition, the majority also met the criteria for Sepsis-3 (30%).

      Prognostic values of the SOFA, qSOFA and SIRS criteria for 28-days mortality among patients with verified infection

      The SOFA score and the qSOFA score were both significantly elevated in non-survivors, while there was no significant difference in SIRS criteria between non-survivors and survivors (Table 2). The predictive value for 28-days mortality of SOFA was good compared to both SIRS and qSOFA (Table 2). The sensitivity and specificity for predicting 28-days mortality showed that SOFA ≥ 2 had 100% sensitivity and 44% specificity, while qSOFA ≥ 2 had 38% sensitivity and 89% specificity, and SIRS ≥ 2 had 81% sensitivity and 28% specificity.
      Table 2Comparison of SOFA, qSOFA and SIRS, respectively, in patients with verified infection in relation to in-hospital mortality at the 28-day follow-up.
      Survivors

      n = 302
      Non-survivors

      n = 21
      SOFA score (1–24)Median with IQR2 (1; 3)4 (1; 6)
      p - value<0.0001
      AUROC

      95% (Cl)
      0.83; 0.76–0.90
      Sensitivity (SOFA ≥ 2)

      [%] (95% Cl)
      100.0 (83.9–100)
      Specificity (SOFA ≥ 2)

      [%] (95% Cl)
      44.0 (38.4–49.8)
      qSOFA score (1–3)Median with IQR0 (0; 1)1 (0; 2)
      p – value0.004
      AUROC

      95% Cl
      0.67; 0.54–0.80
      Sensitivity (qSOFA ≥ 2)

      [%] (95% Cl)
      38.1 (18.1–61.6)
      Specificity (qSOFA ≥ 2)

      [%] (95% Cl)
      89.1 (85.0–92.4)
      SIRS criteria (0–4)Median with IQR2 (1; 3)3 (2; 4)
      p – valuens
      AUROC

      95% (Cl)
      0.61; 0.49–0.74
      Sensitivity (SIRS ≥ 2)

      [%] (95% Cl)
      80.9 (58.1–94.6)
      Specificity (SIRS ≥ 2)

      [%] (95% Cl)
      28.1 (23.1–33.6)
      Data are presented as median with 25th and 75th percentiles in parentheses, p values and AUROC with 95% confidence interval (Cl). Sensitivity and specificity of SOFA, qSOFA and SIRS at the Sepsis-3 and sepsis-2 thresholds are presented with 95% Cl. Mann-Whitney test was used for equality. n: number of patients, IQR: Inter quartile range, SOFA: Sequential organ failure assessment, AUROC: Area under the receiver operating characteristic, qSOFA: quick SOFA, ns: not significant and SIRS: Systemic inflammatory response syndrome.
      The assessment of SOFA score ≥ 2, qSOFA score ≥ 2 and SIRS criteria ≥2 in relation to 28-days mortality are illustrated in Figure 2. The SOFA score was ≥ 2 in all 21 patients who died within 28-days of admission and in 169 (56%) of the survivors (Figure 2A), while the qSOFA score was ≥ 2 in 8 (38%) of the patients who died (Figure 2B). The distribution of patients with a SIRS criterion ≥ 2 included 17 (81%) non-survivors and 217 (72%) of the survivors (Figure 2C).
      Figure 2
      Figure 2Validation of the Sepsis-3 definitions of sepsis to predict outcome. (A) Percent of patients deceased at the 28-day follow-up classified with SOFA score < 2 and SOFA score ≥ 2, respectively. (B) Percent of patients deceased at the 28-day follow-up classified with qSOFA score < 2 and qSOFA score ≥ 2, respectively (C) Percent of patients deceased at the 28-day follow-up classified with SIRS score < 2 and SIRS score ≥ 2, respectively. Boxes represent percent of patients. SOFA: Sequential organ failure assessment, qSOFA: quick SOFA and SIRS: Systemic inflammatory response syndrome.

      Discussion

      Our results support the use of SOFA score ≥ 2 to identify patients with verified infection who have increased risk of mortality within 28-days of admission. We found that a SOFA score ≥ 2 measured after initial assessment was superior compared to both SIRS and qSOFA to classify patients according to risk of 28-days mortality.
      In our study, the predictive value of SOFA ≥ 2 expressed as AUROC was slightly higher than in retrospective studies including ICU patients. The studies are summarized in Table 3 (
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      ,
      • Raith E.P.
      • Udy A.A.
      • Bailey M.
      • et al.
      Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit.
      ,
      • Giamarellos-Bourboulis E.J.
      • Tsaganos T.
      • Tsangaris I.
      • et al.
      Validation of the new Sepsis-3 definitions: proposal for improvement in early risk identification.
      ,
      • Cheng B.
      • Li Z.
      • Wang J.
      • et al.
      Comparison of the performance between sepsis-1 and sepsis-3 in ICUs in China: a retrospective multicenter study.
      ,
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      ). Despite that our analyses were done retrospectively, to our knowledge, our study is among the first to evaluate the use of SOFA score ≥ 2 to assess risk of 28-days mortality in a population of patients prospectively included in a non-ICU setting. Only two retrospective studies by Seymour et al. and Donnelly et al., respectively, have evaluated the prognostic value of SOFA in a non-ICU setting. Both studies showed AUROC values of SOFA comparable to our results (
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • Donnelly J.P.
      • Safford M.M.
      • Shapiro N.I.
      • et al.
      Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study.
      ). The sensitivity and specificity of SOFA ≥ 2 for mortality within 28-days have, to our knowledge, not been published previously. The high sensitivity for identifying patients who died within 28 days (100% in our study population) emphasizes the potential clinical value of SOFA ≥ 2 or SOFA < 2 to guide management of patients with supposed infection.
      Table 3Comparison of AUROC and sensitivity and specificity of different selected studies for validation of sepsis-3.
      PatientsSettingInclusion severityDesignPrimary outcomeSOFA

      AUROC

      SEN

      SPE
      qSOFA

      AUROC

      SEN

      SPE
      SIRS

      AUROC

      SEN

      SPE
      Reference
      214ICUSI or VIRIn-hospital mortality0.70

      ND

      ND
      0.66

      qSOFA ≥ 2:

      89.7% (SEN)

      27.4% (SPE)
      0.65

      SIRS ≥ 2:

      97.4% (SEN)

      2.3% (SPE)
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      3346 (ED)

      1058 (ICU)
      VI + Sepsis-2PIn-hospital mortalityNDND

      qSOFA ≥ 2 (ICU):

      87.5% (SEN)

      ND
      ND
      • Giamarellos-Bourboulis E.J.
      • Tsaganos T.
      • Tsangaris I.
      • et al.
      Validation of the new Sepsis-3 definitions: proposal for improvement in early risk identification.
      148907ED (66617)

      ICU (7836)
      SIRIn-hospital mortality0.74 (ICU)

      0.79 (no ICU)

      ND

      ND
      0.66 (ICU)

      0.81 (no ICU)

      ND

      ND
      0.64 (ICU)

      0.76 (no ICU)

      ND

      ND
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      496ICUSIR28-day mortality0.69

      ND

      ND
      ND0.55

      ND

      ND
      • Cheng B.
      • Li Z.
      • Wang J.
      • et al.
      Comparison of the performance between sepsis-1 and sepsis-3 in ICUs in China: a retrospective multicenter study.
      30239EDSIRIn-hospital mortality0.77 (SOFA ≥ 2)

      ND

      ND
      0.76 (qSOFA ≥ 2)

      ND

      ND
      0.72 (SIRS ≥ 2)

      ND

      ND
      • Donnelly J.P.
      • Safford M.M.
      • Shapiro N.I.
      • et al.
      Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study.
      152ICU (102)

      HW (50)
      SIPIn-hospital mortalityND0.74

      qSOFA ≥ 2:

      90% (SEN)

      42% (SPE)
      0.59

      SIRS ≥ 2:

      93% (SEN)

      12% (SPE)
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      879EDSIPIn-hospital mortalityND0.80

      ND

      ND
      0.65

      ND

      ND
      • Freund Y.
      • Lemachatti N.
      • Krastinova E.
      • et al.
      Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department.
      7637EDSIPIn-hospital mortalityNDND

      qSOFA ≥ 2:

      52% (SEN)

      86% (SPE)
      ND

      SIRS ≥ 2:

      83% (SEN)

      50% (SPE)
      • Henning D.J.
      • Puskarich M.A.
      • Self W.H.
      • et al.
      An emergency department validation of the SEP-3 sepsis and septic shock definitions and comparison with 1992 consensus definitions.
      184875ICUSIRIn-hospital mortality0.753

      ND

      ND
      0.61

      ND

      ND
      0,59

      ND

      ND
      • Raith E.P.
      • Udy A.A.
      • Bailey M.
      • et al.
      Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit.
      8871EDSIP30-day mortalityND0.78

      qSOFA ≥ 2:

      50% (SEN)

      91.3% (SPE)
      0.71

      SIRS ≥ 2:

      77% (SEN)

      54% (SPE)
      • Williams J.M.
      • Greenslade J.H.
      • McKenzie J.V.
      • et al.
      Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
      323HWVIPIn-hospital mortality0.83

      SOFA ≥ 2:

      100% (SEN)

      44% (SPE)
      0.67

      qSOFA ≥ 2:

      38% (SEN)

      89% (SPE)
      0.61

      SIRS ≥ 2:

      81% (SEN)

      28% (SPE)
      Gaini et al. (2018 present study)
      SIRS: Systemic inflammatory response syndrome, SOFA: Sequential organ failure assessment, qSOFA: quick SOFA, AUROC: Area under the receiver operating characteristic, SEN: Sensitivity, SPE: Specificity, ICU: Intensive care unit, SI: Suspected infection, VI: Verified infection, R: retrospective, ND: not determined, ED: Emergence department, P: prospective, HW: Hospital wards.
      The AUROC of SIRS criteria in our non-ICU setting was comparable to the previous published results in both prospectively and retrospectively included patients primarily from ICU settings (Table 3) (
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      ,
      • Raith E.P.
      • Udy A.A.
      • Bailey M.
      • et al.
      Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit.
      ,
      • Cheng B.
      • Li Z.
      • Wang J.
      • et al.
      Comparison of the performance between sepsis-1 and sepsis-3 in ICUs in China: a retrospective multicenter study.
      ,
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      ,
      • Donnelly J.P.
      • Safford M.M.
      • Shapiro N.I.
      • et al.
      Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study.
      ,
      • Freund Y.
      • Lemachatti N.
      • Krastinova E.
      • et al.
      Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department.
      ), while
      • Donnelly J.P.
      • Safford M.M.
      • Shapiro N.I.
      • et al.
      Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study.
      in their retrospective study including 30239 non-ICU patients and
      • Williams J.M.
      • Greenslade J.H.
      • McKenzie J.V.
      • et al.
      Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
      in their prospective study including 8871 non-ICU patients, both reported a slightly higher AUROC of SIRS criteria for mortality (0.72 and 0.71 respectively) (
      • Donnelly J.P.
      • Safford M.M.
      • Shapiro N.I.
      • et al.
      Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study.
      ,
      • Williams J.M.
      • Greenslade J.H.
      • McKenzie J.V.
      • et al.
      Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
      ). Only,
      • Williams J.M.
      • Greenslade J.H.
      • McKenzie J.V.
      • et al.
      Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
      have evaluated the sensitivity and specificity of SIRS ≥ 2 in non-ICU patients and showed a sensitivity of 77% and a specificity of 54% compared with 81% and 28%, respectively, in our population. In contrast,
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      and
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      in their study populations from ICU settings both reported sensitivities above 90% and very low specificities (2.3% and 12%, respectively). The predictive values of qSOFA scores have been more inconsistent (Table 3) (
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      ,
      • Raith E.P.
      • Udy A.A.
      • Bailey M.
      • et al.
      Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit.
      ,
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      ,
      • Donnelly J.P.
      • Safford M.M.
      • Shapiro N.I.
      • et al.
      Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study.
      ,
      • Freund Y.
      • Lemachatti N.
      • Krastinova E.
      • et al.
      Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department.
      ,
      • Williams J.M.
      • Greenslade J.H.
      • McKenzie J.V.
      • et al.
      Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
      ). The studies analyzing the predictive value of qSOFA score for in-hospital mortality in patients admitted to ICU for suspected infection found AUROC values ranging from 0.61 to 0.74 comparable to our results (
      • Seymour C.W.
      • Liu V.X.
      • Iwashyna T.J.
      • et al.
      Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
      ,
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      ,
      • Raith E.P.
      • Udy A.A.
      • Bailey M.
      • et al.
      Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit.
      ,
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      ), while the previous studies in a non-ICU setting have shown better predictive accuracy of qSOFA ranging from 0.76 to 0.81. Only, a few of the previous studies have provided data on sensitivity and specificity of qSOFA ≥ 2 (
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      ,
      • Giamarellos-Bourboulis E.J.
      • Tsaganos T.
      • Tsangaris I.
      • et al.
      Validation of the new Sepsis-3 definitions: proposal for improvement in early risk identification.
      ,
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      ,
      • Williams J.M.
      • Greenslade J.H.
      • McKenzie J.V.
      • et al.
      Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
      ,
      • Henning D.J.
      • Puskarich M.A.
      • Self W.H.
      • et al.
      An emergency department validation of the SEP-3 sepsis and septic shock definitions and comparison with 1992 consensus definitions.
      ). Interestingly, the results in ICU patients consistently report high sensitivity and low specificity (
      • April M.D.
      • Aguirre J.
      • Tannenbaum L.I.
      • et al.
      Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
      ,
      • Giamarellos-Bourboulis E.J.
      • Tsaganos T.
      • Tsangaris I.
      • et al.
      Validation of the new Sepsis-3 definitions: proposal for improvement in early risk identification.
      ,
      • Finkelsztein E.J.
      • Jones D.S.
      • Ma K.C.
      • et al.
      Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
      ), while the findings in previous non-ICU settings and in our study consistently and oppositely report lower sensitivity and high specificity (
      • Williams J.M.
      • Greenslade J.H.
      • McKenzie J.V.
      • et al.
      Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
      ,
      • Henning D.J.
      • Puskarich M.A.
      • Self W.H.
      • et al.
      An emergency department validation of the SEP-3 sepsis and septic shock definitions and comparison with 1992 consensus definitions.
      ). A large recently published meta-analysis pooling 406802 patients with suspected infection concluded that qSOFA was a poorly sensitive marker for in-hospital mortality in hospitalized patients not in intensive care units (sensitivity 48% and specificity 83%) (
      • Maitra S.
      • Som A.
      • Bhattacharjee S.
      Accuracy of quick Sequential Organ Failure Assessment (qSOFA) score and systemic inflammatory resposyndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: a meta-analysis of observational studies.
      ). The same study showed a sensitivity of 56% and specificity of 78% for in-hospital mortality among all patients including those treated at ICU units (
      • Maitra S.
      • Som A.
      • Bhattacharjee S.
      Accuracy of quick Sequential Organ Failure Assessment (qSOFA) score and systemic inflammatory resposyndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: a meta-analysis of observational studies.
      ). The poor accuracy for mortality of qSOFA score compared with the superior accuracy of SOFA score reported in our study were supported by previous studies demonstrating that expanding of the qSOFA score with measures of either arterial pH, heart rate, age or fever increased the sensitivity for severe sepsis and septic shock, at the expense of specificity (
      • Giamarellos-Bourboulis E.J.
      • Tsaganos T.
      • Tsangaris I.
      • et al.
      Validation of the new Sepsis-3 definitions: proposal for improvement in early risk identification.
      ,
      • Dorsett M.
      • Kroll M.
      • Smith C.S.
      • et al.
      qSOFA has poor sensitivity for prehospital identification of severe sepsis and septic shock.
      ). Although neither of the studies included AUROC analysis of SOFA score, the results are supportive of the use of SOFA, as expansion of the qSOFA score with new parameters approaches the SOFA score.
      Our main finding, that a SOFA ≥ 2 predicts mortality better than qSOFA, combined with previous observations by others support the proposed guideline in Sepsis-3; that qSOFA are less robust than SOFA in ICU. Moreover, our main results complement the idea that the use of SIRS criteria does not necessarily identify patients with elevated risk of in-hospital mortality (
      • Churpek M.M.
      • Zadravecz F.J.
      • Winslow C.
      • et al.
      Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients.
      ).
      Thus, our findings support the clinical use of Sepsis-3 in initial assessment of patients hospitalized with infection, to discriminate between patients with high and low risk of in-hospital mortality. The strength of this support must however be interpreted considering the limitations of our study.
      Firstly, our study was done retrospectively and included a limited number of patients. This means that some of the subgroups were small with increased risk of false-negative results due to low statistical power. Therefore, our results should preferably be confirmed in larger, prospective studies. Secondly, serum lactate level was unavailable for many patients, and consequently we were unable to identify patients with septic shock according to Sepsis-3. Thirdly, missing values for RR and GCS might have led to an underestimation of both qSOFA and SIRS leading to underestimation of their predictive power. Yet reiteration of all the analyses with exclusion of potentially misclassified patients did not change the obtained results significantly (see web-only Supplementary File S1), and the missing values did not affect the main conclusion that SOFA ≥ 2 seems to be a valid predictor of increased in-hospital mortality. However, it may have underestimated the predictive power of qSOFA ≥ 2.
      Our study supports the use of SOFA score ≥ 2 as an aid to identify patients with increased risk of in-hospital mortality among non-ICU patients hospitalized with infection, but neither SOFA nor qSOFA scores ≥ 2 should be used as a single indicator to guide clinical management.

      Funding

      Region of Southern Denmark’s Research foundation supported the study.

      Conflict of interest

      The authors declare no potential conflict of interest.

      Ethics

      Data and blood sample collection in the three original studies were done in accordance with the Declaration of Helsinki and ethical allowances were obtained from the Scientific Ethical Committee of Southern Denmark (Funen and Vejle Counties, file number 20010076) and from the Danish Data Security Board (2003). Written informed consent was obtained from the patients or closest relatives in each case.
      After completion of the original studies, all data and samples were completely and irrevocably anonymized, so that the material cannot be directly or indirectly attributed to the donor with the means that may reasonably be used to identify the person concerned. According to Danish law, research on anonymized data can be done without obtaining new approval from the Ethical Committee and data Protection Agency if the original data collection was done in accordance with all rules.

      Availability of data and material

      The datasets used are available from the corresponding author on reasonable request.

      Authors’ contributions

      SG: Collected the original data, participated in the design of the study and helped to write and revise the manuscript.
      MMR: Participated in the design of the study, carried out the analyses, performed the statistical analysis, interpreted the patient data and wrote the manuscript.
      CP: Participated in the design of the study, gave valuable suggestions and helped to revise the manuscript.
      ISJ: Participated in the design of the study, gave valuable suggestions and helped to revise the manuscript.
      All authors read and approved the final manuscript.

      Appendix A. Supplementary data

      The following is Supplementary data to this article:

      References

        • Angus D.C.
        • van der Poll T.
        Severe sepsis and septic shock.
        N Engl J Med. 2013; 369 (PubMed PMID: 23984731): 840-851https://doi.org/10.1056/NEJMra1208623
        • April M.D.
        • Aguirre J.
        • Tannenbaum L.I.
        • et al.
        Sepsis clinical criteria in emergency department patients admitted to an intensive care unit: an external validation study of quick sequential organ failure assessment.
        J Emerg Med. 2017; 52 (PubMed PMID: 27823893): 622-631https://doi.org/10.1016/j.jemermed.2016.10.012
        • Bone R.C.
        • Balk R.A.
        • Cerra F.B.
        • et al.
        Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine.
        Chest. 1992; 101 (PubMed PMID: 1303622): 1644-1655
        • Bone R.C.
        Toward an epidemiology and natural history of SIRS (systemic inflammatory response syndrome).
        JAMA. 1992; 268 (PubMed PMID: 1460735): 3452-3455
        • Cheng B.
        • Li Z.
        • Wang J.
        • et al.
        Comparison of the performance between sepsis-1 and sepsis-3 in ICUs in China: a retrospective multicenter study.
        Shock. 2017; 48 (PubMed PMID: 28448400): 301-306https://doi.org/10.1097/SHK.0000000000000868
        • Churpek M.M.
        • Zadravecz F.J.
        • Winslow C.
        • et al.
        Incidence and prognostic value of the systemic inflammatory response syndrome and organ dysfunctions in ward patients.
        Am J Respir Crit Care Med. 2015; 192 (PubMed PMID: 26158402; PubMed Central PMCID: PMC4642209): 958-964https://doi.org/10.1164/rccm.201502-0275OC
        • Deutschman C.S.
        • Tracey K.J.
        Sepsis: current dogma and new perspectives.
        Immunity. 2014; 40 (PubMed PMID: 24745331): 463-475https://doi.org/10.1016/j.immuni.2014.04.001
        • Donnelly J.P.
        • Safford M.M.
        • Shapiro N.I.
        • et al.
        Application of the Third International Consensus Definitions for Sepsis (Sepsis-3) Classification: a retrospective population-based cohort study.
        Lancet Infect Dis. 2017; 17 (PubMed PMID: 28268067): 661-670https://doi.org/10.1016/S1473-3099(17)30117-2
        • Dorsett M.
        • Kroll M.
        • Smith C.S.
        • et al.
        qSOFA has poor sensitivity for prehospital identification of severe sepsis and septic shock.
        Prehosp Emerg Care. 2017; 21 (PubMed PMID: 28121217): 489-497https://doi.org/10.1080/10903127.2016.1274348
        • Finkelsztein E.J.
        • Jones D.S.
        • Ma K.C.
        • et al.
        Comparison of qSOFA and SIRS for predicting adverse outcomes of patients with suspicion of sepsis outside the intensive care unit.
        Crit Care. 2017; 21 (PubMed PMID: 28342442; PubMed Central PMCID: PMC5366240): 73https://doi.org/10.1186/s13054-017-1658-5
        • Freund Y.
        • Lemachatti N.
        • Krastinova E.
        • et al.
        Prognostic accuracy of sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department.
        JAMA. 2017; 317 (PubMed PMID: 28114554): 301-308https://doi.org/10.1001/jama.2016.20329
        • Gaini S.
        • Koldkjaer O.G.
        • Pedersen C.
        • et al.
        Procalcitonin, lipopolysaccharide-binding protein, interleukin-6 and C-reactive protein in community-acquired infections and sepsis: a prospective study.
        Crit Care. 2006; 10 (PubMed PMID: 16569262; PubMed Central PMCID: PMC1550885): R53https://doi.org/10.1186/cc4866
        • Gaini S.
        • Koldkjaer O.G.
        • Moller H.J.
        • et al.
        A comparison of high-mobility group-box 1 protein, lipopolysaccharide-binding protein and procalcitonin in severe community-acquired infections and bacteraemia: a prospective study.
        Crit Care. 2007; 11 (PubMed PMID: 17625012; PubMed Central PMCID: PMC2206511): R76https://doi.org/10.1186/cc5967
        • Gaini S.
        • Pedersen S.S.
        • Koldkaer O.G.
        • et al.
        New immunological serum markers in bacteraemia: anti-inflammatory soluble CD163, but not proinflammatory high mobility group-box 1 protein, is related to prognosis.
        Clin Exp Immunol. 2008; 151 (PubMed PMID: 18190604; PubMed Central PMCID: PMC2276958): 423-431https://doi.org/10.1111/j.1365-2249.2007.03586.x
        • Giamarellos-Bourboulis E.J.
        • Tsaganos T.
        • Tsangaris I.
        • et al.
        Validation of the new Sepsis-3 definitions: proposal for improvement in early risk identification.
        Clin Microbiol Infect. 2017; 23 (PubMed PMID: 27856268): 104-109https://doi.org/10.1016/j.cmi.2016.11.003
        • Henning D.J.
        • Puskarich M.A.
        • Self W.H.
        • et al.
        An emergency department validation of the SEP-3 sepsis and septic shock definitions and comparison with 1992 consensus definitions.
        Ann Emerg Med. 2017; S0196-0644 (PubMed PMID: 28262318): 30023-30029https://doi.org/10.1016/j.annemergmed.2017.01.008
        • Maitra S.
        • Som A.
        • Bhattacharjee S.
        Accuracy of quick Sequential Organ Failure Assessment (qSOFA) score and systemic inflammatory resposyndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: a meta-analysis of observational studies.
        Clin Microbiol Infect. 2018; (pii: S1198-743X(18)30294-5. [Epub ahead of print] ReviewPMID: 29605565)https://doi.org/10.1016/j.cmi.2018.03.032
        • Mayr F.B.
        • Yende S.
        • Angus D.C.
        Epidemiology of severe sepsis.
        Virulence. 2014; 5 (PubMed PMID: 24335434; PubMed Central PMCID: PMC3916382): 4-11https://doi.org/10.4161/viru.27372
        • Raith E.P.
        • Udy A.A.
        • Bailey M.
        • et al.
        Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit.
        JAMA. 2017; 317 (PubMed PMID: 28114553): 290-300https://doi.org/10.1001/jama.2016.20328
        • Seymour C.W.
        • Liu V.X.
        • Iwashyna T.J.
        • et al.
        Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
        JAMA. 2016; 315 (PubMed PMID: 26903335): 762-774https://doi.org/10.1001/jama.2016.0288
        • Shankar-Hari M.
        • Phillips G.S.
        • Levy M.L.
        • et al.
        Developing a new definition and assessing new clinical criteria for septic shock: for the third international consensus definitions for sepsis and septic shock (Sepsis-3).
        JAMA. 2016; 315 (PubMed PMID: 26903336): 775-787https://doi.org/10.1001/jama.2016.0289
        • Singer M.
        • Deutschman C.S.
        • Seymour C.W.
        • et al.
        The third international consensus definitions for sepsis and septic shock (Sepsis-3).
        JAMA. 2016; 315 (PubMed PMID: 26903338): 801-810https://doi.org/10.1001/jama.2016.0287
        • Wiersinga W.J.
        • Leopold S.J.
        • Cranendonk D.R.
        • et al.
        Host innate immune responses to sepsis.
        Virulence. 2014; 5 (PubMed Central PMCID: PMC3916381): 36-44https://doi.org/10.4161/viru.25436
        • Williams J.M.
        • Greenslade J.H.
        • McKenzie J.V.
        • et al.
        Systemic inflammatory response syndrome, quick sequential organ function assessment, and organ dysfunction: insights from a prospective database of ED patients with infection.
        Chest. 2017; 151 (PubMed PMID: 27876592): 586-596https://doi.org/10.1016/j.chest.2016.10.057