Real-time machine learning model to predict in-hospital cardiac arrest using heart rate variability in ICU – npj Digital Medicine

Recognizing the need for predicting in-hospital cardiac arrest in critically ill patients, we developed and validated an ML-based prediction model for in-hospital cardiac arrest using HRV measures in ICU patients. Our model leveraged HRV measures to overcome limitations encountered with conventional prediction models that rely on extensive EMR data. The proposed model not only simplifies the prediction process through a single data source but also facilitates real-time, continuous monitoring. The results demonstrated the potential of the LGBM model, which achieved good discrimination performance. This was paramount for the early detection and rapid prediction of in-hospital cardiac arrest, thereby improving patient outcomes in real-world clinical settings. This study highlights the (1) ability of the proposed model to predict the risk of in-hospital cardiac arrest using ECG data only, (2) usability of multiple HRV measures in the proposed ML-based model, and (3) explainability of the model through HRV measures.

In this study, only ECG data are used to predict the risk of in-hospital cardiac arrest, making our proposed model highly accessible and transferable to other healthcare settings that collect ECG data because continuous ECG monitoring is a standard practice in ICU settings. Unlike previous studies that employed multiple data sources, such as demographic information, vital signs, and laboratory results, to develop their prediction models^4,23,24,25, our model finds easy application in clinical practice because only ECG data are required to predict cardiac arrest in ICU settings. Additionally, we conducted a comparative analysis between our model and a clinical parameter-based model from a previous study, which utilized 43 features derived from six vital signs. While the clinical parameter-based model achieved an impressive AUROC of 0.94 for predicting in-hospital cardiac arrest within 1 h, our findings indicated that its performance was not consistently maintained when predicting events occurring within 24 h (Supplementary Table 1).

Since HRV quantifies dynamic changes in ECG signals, previous studies utilized HRV measures to develop models in various medical contexts, including the prediction of poor outcomes or treatment responses^26,27,28. However, such studies used traditional statistical models, such as a multivariable logistic regression model, which limited the number of HRV measures that could have been used owing to the linearity assumption between predictors and outcomes²⁹. Contrarily, ML-based models handle complex relationships among predictors and outcomes, thus offering the advantage of using numerous other HRV measures, including IALS, pNN50, TINN, and HTI, in the model development process, in addition to the traditional sets of HRV measures, such as the mean of the RR intervals (meanNN), SDNN, LF, and HF²⁹. Furthermore, ML-based models provide a distinct advantage while managing the inherently nonlinear and nonstationary fluctuations of HRV measures²⁹. In our study, we utilized nonlinear measures including IALS, TINN, and HTI, which have been proven effective in detecting diseases such as end-stage renal disease, primary aldosteronism, and pulmonary hypertension^30,31,32. The integration of these nonlinear HRV measures into ML algorithms proved to be of great potential in delivering superior discriminative performance. This observation was consistent with those of previous studies on different diseases³³, thereby further endorsing the effectiveness of the proposed approach.

In this study, the BorutaShap algorithm was employed to identify the most relevant HRV measure from 43 HRV measures, resulting in the selection of 33 HRV measures as input features for the model. Utilizing such a comprehensive set of HRV measures increased the accuracy and robustness of the prediction model. The feature importance analysis results determined using the SHAP method revealed that TINN, HTI, IALS, Prc20NN, MinNN, and IQRNN were the most critical HRV measures in the in-hospital cardiac arrest prediction model.

TINN, standing as the most pivotal feature in our study, was closely followed by HTI. Both TINN and HTI are time-domain HRV measures derived from geometric analysis, providing insights into the overall shape and distribution of the RR interval histogram¹⁰. TINN quantifies the baseline width of the distribution of RR intervals using triangular interpolation, where the triangle is determined by the least squares error. A larger TINN value typically signifies greater variability in the RR intervals. Conversely, HTI reflects the total number of RR intervals divided by the height of these intervals, shedding light on how the RR intervals are distributed. A lower HTI suggests that a higher proportion of intervals cluster around the mode, while a higher HTI indicates a wider spread of intervals. Notably, previous research has emphasized the importance of both HTI and TINN in cardiac risk assessment. Studies have shown that these values tend to be significantly lower in patients with sudden cardiac death compared to those with hypertrophic cardiomyopathy or healthy individuals³⁴. Additionally, in the context of developing prediction models for cardiac arrest in critically ill patients, TINN and HTI values have been found to be lower in patients experiencing cardiac arrest compared to those without²³. These values have also exhibited distinctions in patients with arrhythmias compared to healthy individuals, with a notable difference in HTI values between these groups³⁵. Furthermore, a previous study proved HTI to be an independent predictor of cardiovascular mortality in patients with AF¹⁵.

New HRV measures introduced in recent studies were applied to this study. A new HRV measure known as heart rate fragmentation or IALS was identified as one of the important features of our study. For IALS, acceleration, and deceleration segments were defined by a sequence of RR intervals between consecutive inflection points, for which the difference between the two RR intervals was <0 and >0, respectively. Segment length was determined as the number of RR intervals in that segment³⁶. A prior study revealed that IALS was significantly higher in patients with congestive heart failure (CHF), with a mean IALS of 0.78. This result is similar to that of our study (Fig. 5), suggesting that higher IALS can be associated with compromised cardiac conditions³⁷. Approximately 30–50% of the patients with CHF were estimated to be at risk of sudden cardiac arrest³⁸.

Few studies have used the other HRV measures included in our study, such as IQRNN, to study the relationship between those measures and cardiac arrest; however, our findings suggested that IQRNN has the potential as predictors of cardiac arrest. The values of IQRN, as well as TINN in patients experiencing sudden cardiac arrest, remained similar to those in patients without sudden cardiac arrest up until approximately 6 h prior to the event, after which dynamic changes occurred. Nevertheless, the causality between these HRV measures and cardiac arrest requires further investigation.

Changes in HRV measures were analyzed within the timeframe of 0.5 h to 24 h preceding the in-hospital cardiac arrest and compared with their median values in patients without in-hospital cardiac arrest, as shown in Fig. 5. The IALS values were consistently higher within 0.5 to 24 h preceding the cardiac arrest event compared to the patients without in-hospital cardiac arrest; however, there was a decreasing trend in these values leading up to the cardiac arrest event. Conversely, HTI values started low, but increased towards the event of cardiac arrest. These consecutive changes in HRV measures have not been documented in previous studies. Therefore, the analytical results of this study are expected to provide valuable insights into the real-time condition evaluation of a patient and facilitate the prompt initiation of interventions aimed at preventing events of cardiac arrest.

Furthermore, this study has the significant advantage of utilizing a large sample size of ~5000 patients, which adds to the representativeness and generalizability of the results to other patient populations. A large sample size is critical for accurately detecting rare events, such as in-hospital cardiac arrest, which is essential for developing reliable ML-based predictive models^39,40.

Nevertheless, the limitations of this study must be considered while interpreting the results. The binary classification model used has certain restrictions; the model can only predict whether a patient will experience a cardiac arrest but does not provide information on the timing of the event; however, we tried to evaluate our model on different time periods as secondary outcomes. Additionally, the model does not account for the influence of treatment interventions on outcomes and focuses solely on baseline predictors. The selection of the development and validation sets may also have been biased, which can affect the accuracy and generalizability of the results. Furthermore, the study was performed at a single center, limiting the transferability of the findings to other patient populations and healthcare systems.

Future research should focus on validating the findings of this study in larger multi-center studies to increase the generalizability of the results and confidence in the predictions made by the model. Open datasets with labels for cardiac arrest and ECG waveforms, such as the Medical Information Market for Intensive Care, can help validate our results before conducting a multi-center prospective study. Moreover, incorporating clinical factors such as comorbidities or medications may further assist the model⁴¹; however, we intentionally excluded these factors in this study considering variable availability across different hospital settings. Additionally, developing survival models that account for both the probability and timing of a cardiac arrest event is expected to provide valuable information for clinical decision-making and allow a better understanding of the long-term outcomes of patients who experience sudden cardiac arrest in ICU settings.

In conclusion, we developed and validated an ML-based real-time prediction model to predict in-hospital cardiac arrest in critically ill patients, focusing on the importance of HRV measures. If future prospective studies validate our results, they can potentially be used to detect in-hospital cardiac arrest in critically ill patients.