Using Machine Learning to Develop and Validate an In‐Hospital Mortality Prediction Model for Patients with Suspected Sepsis

Background: Early recognition of sepsis and the prediction of mortality in patients with infection are important. This multi‐center, ED‐based study aimed to develop and validate a 28‐day mortality prediction model for patients with infection using various machine learning (ML) algorithms. Methods: Patients with acute infection requiring intravenous antibiotic treatment during the first 24 h of admission were prospectively recruited. Patient demographics, comorbidities, clinical signs and symptoms, laboratory test data, selected sepsis‐related novel biomarkers, and 28‐day mortality were collected and divided into training (70%) and testing (30%) datasets. Logistic regression and seven ML algorithms were used to develop the prediction models. The area under the receiver operating characteristic curve (AUROC) was used to compare different models. Results: A total of 555 patients were recruited with a full panel of biomarker tests. Among them, 18% fulfilled Sepsis‐3 criteria, with a 28‐day mortality rate of 8%. The wrapper algorithm selected 30 features, including disease severity scores, biochemical parameters, and conventional and few sepsis‐related biomarkers. Random forest outperformed other ML models (AUROC: 0.96; 95% confidence interval: 0.93–0.98) and SOFA and early warning scores (AUROC: 0.64–0.84) in the prediction of 28‐day mortality in patients with infection. Additionally, random forest remained the best‐performing model, with an AUROC of 0.95 (95% CI: 0.91–0.98, p = 0.725) after removing five sepsis‐related novel biomarkers. Conclusions: Our results demonstrated that ML models provide a more accurate prediction of 28‐day mortality with an enhanced ability in dealing with multi‐dimensional data than the logistic regression model.