Saudi Journal of Gastroenterology
Home About us Instructions Submission Subscribe Advertise Contact Login    Print this page  Email this page Small font sizeDefault font sizeIncrease font size 
Users Online: 570 

Table of Contents   
Year : 2022  |  Volume : 28  |  Issue : 1  |  Page : 32-38
Development and validation of a predictive model for feeding intolerance in intensive care unit patients with sepsis

1 Department of Emergency Medicine, The First Clinical Medical College of Jinan University, Guangzhou, Guangdong Province; Department of Intensive Care Unit, the People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, China
2 Department of Environmental Health Sciences, University at Albany, State University of New York, USA
3 Department of Intensive Care Medicine, Yong wu Hospital of Guangxi Zhuang Autonomous Region, Nanning, China
4 Department of Intensive Care Unit, the People's Hospital of Guangxi Zhuang Autonomous Region, Nanning, China
5 Department of Intensive Care Medicine, The First Affiliated Hospital, Guangxi Medical University, Nanning, China

Click here for correspondence address and email

Date of Submission26-May-2021
Date of Acceptance18-Jul-2021
Date of Web Publication14-Sep-2021


Background: Feeding intolerance in patients with sepsis is associated with a lower enteral nutrition (EN) intake and worse clinical outcomes. The aim of this study was to develop and validate a predictive model for enteral feeding intolerance in the intensive care unit patients with sepsis.
Methods: In this dual-center, retrospective, case-control study, a total of 195 intensive care unit patients with sepsis were enrolled from June 2018 to June 2020. Data of 124 patients for 27 clinical indicators from one hospital were used to train the model, and data from 71 patients from another hospital were used to assess the external predictive performance. The predictive models included logistic regression, naive Bayesian, random forest, gradient boosting tree, and deep learning (multilayer artificial neural network) models.
Results: Eighty-six (44.1%) patients were diagnosed with enteral feeding intolerance. The deep learning model achieved the best performance, with areas under the receiver operating characteristic curve of 0.82 (95% confidence interval = 0.74–0.90) and 0.79 (95% confidence interval = 0.68–0.89) in the training and external sets, respectively. The deep learning model showed good calibration; based on the decision curve analysis, the model's clinical benefit was considered useful. Lower respiratory tract infection was the most important contributing factor, followed by peptide EN and shock.
Conclusions: The new prediction model based on deep learning can effectively predict enteral feeding intolerance in intensive care unit patients with sepsis. Simple clinical information such as infection site, nutrient type, and septic shock can be useful in stratifying a septic patient's risk of EN intolerance.

Keywords: Deep learning, enteral feeding intolerance, predictive model, sepsis

How to cite this article:
Hu K, Deng X, Han L, Xiang S, Xiong B, Pinhu L. Development and validation of a predictive model for feeding intolerance in intensive care unit patients with sepsis. Saudi J Gastroenterol 2022;28:32-8

How to cite this URL:
Hu K, Deng X, Han L, Xiang S, Xiong B, Pinhu L. Development and validation of a predictive model for feeding intolerance in intensive care unit patients with sepsis. Saudi J Gastroenterol [serial online] 2022 [cited 2022 Jul 2];28:32-8. Available from:

See accompanying Editorial

   Introduction Top

Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection,[1] and remains an important global health concern. Sepsis is associated with the breakdown of carbohydrate, lipid, and protein stores. Despite the increased nutritional requirements of patients with sepsis, their nutrition intake decreases, leading to a substantial energy shortage.[2] Enteral nutrition (EN) is recommended in critically ill patients by the European Society of Intensive Care Medicine (ESICM).[3] Unfortunately, intensive care unit (ICU) patients diagnosed with sepsis are likely to develop enteral feeding intolerance (FI).[4]

FI indicates intolerance to enteral feeding for clinical reasons such as vomiting, diarrhea, high-gastric residuals, or the presence of entero-cutaneous fistulas.[5] Compared to the patients without FI, patients with FI show 10% less EN intake. Moreover, patients with enteral FI have more ventilator days and longer ICU length of stay than those without enteral FI. The daily mortality hazard rate could increase significantly once enteral FI is noted.[4]

Early FI prevention in patients with sepsis has clinical significance. Previous studies have mostly focused on the analysis of susceptibility factors for FI in patients with sepsis, and there are no prediction models for FI in patients with sepsis. Thus, we aimed to develop and validate a predictive model for FI in patients with sepsis, providing a basis for clinical decision-making during the period for early prevention of EN intolerance.

   Patients and Methods Top

Study design and data collection

We conducted a retrospective, dual center clinical study. The study was approved by our hospital ethics committee, and informed consent was not required. A total of 235 consecutive adult patients with sepsis (age >18 years) who were admitted to the ICU of the People's Hospital of Guangxi Zhuang Autonomous Region were enrolled between June 2018 and June 2020. Furthermore, 136 adult patients with sepsis admitted to Yongwu Hospital of Guangxi Zhuang Autonomous Region were enrolled in the same period. The diagnostic criteria for sepsis were the Sepsis 3.0 Consensus criteria.[6] All enrolled patients had no contraindications to EN, whereas patients with severe septic shock, gastrointestinal tumor, chronic diarrhea, gastrointestinal bleeding, gastrointestinal surgery, or length of ICU stay less than 7 days were excluded. The enrolled patients were allocated to the feeding tolerance (FT) group and the FI group, according to whether they had FI during their ICU stay. The patients were fed through a nasoduodenal tube or nasogastric tube and intermittent (q4h, four times a day) or bolus feeding. The nutrition formula was a short-peptide or intact-protein formula. Vomiting, distension, high-gastric residuals (gastric residual volume ≥500 mL/24 h), diarrhea, and high intra-abdominal pressure (intra-abdominal pressure >12 mmHg) were considered to be the outcomes of enteral FI.[5]

Based on previous studies,[4],[5],[7],[8],[9],[10],[11] the collected exposure data included information on demographics (age and sex), medical history (hypertension, diabetes, coronary heart disease, kidney failure, chronic obstructive pulmonary disease, cerebrovascular accident), infection site, clinical presentation, laboratory results on admission (albumin, creatinine, urea nitrogen), disease severity (Acute Physiology and Chronic Health Evaluation II score, Sequential Organ Failure Assessment score), clinical treatment (sedation, analgesia, antibiotics, continuous feeding), EN type (short peptides, intact protein), and patient outcomes. All exposure data were collected from the database of electronic medical records and laboratory test reports.

Statistical analyses

A total of 195 patients were included in the study. Moreover, 124 patients from the People's Hospital of Guangxi Zhuang Autonomous Region were assigned to the training dataset, and 71 patients from Yongwu Hospital of Guangxi Zhuang Autonomous Region were assigned to the testing dataset. The continuous variables are expressed as mean with standard deviation, and categorical variables are presented as frequencies with percentages.

We constructed different models based on the training set through the grid search and five-fold cross-validation. These models included logistic regression, naive Bayesian, random forest, gradient boosting tree, and deep learning (multilayer feedforward artificial neural network [ANN] algorithm). We evaluated the performance of different models on the testing dataset as external validation. The area under the receiver operating characteristic curve (AUC) was used as the evaluation metrics. We selected the model that had the best performance and constructed an online predictive tool. Besides, we also established calibration curves and performed the decision curve analysis. All analyses were performed using R software (version 4.0.1).

   Results Top

Demographics and clinical characteristics

A total of 195 patients were included in the study, with 124 and 71 patients in the training set and the testing set, respectively. Among all the patients, 86 (44.1%) were diagnosed with FI, 49 (39.5%) with FI from the training set, and 37 (52.1%) with FI in the testing set. The majority of the patients in the training set were older men (average age 66.7 ± 17.2 years; male patients 70.9%). The demographic information of the testing set was generally similar to that of the training set (average age 71.5 ± 13.8 years; male patients 70.4%). The most common site of infection was the lower respiratory tract (94.3% patients in the training set and 78.9% patients in the testing set), while the most common underlying disease was hypertension (64.5% patients in the training set and 38.0% patients in the testing set). The patient characteristics in the primary cohorts and validation cohorts are presented in [Table 1].
Table 1: Characteristics of patients in the training and validation cohorts

Click here to view

Development and choice of a prediction model

We constructed five different models, with an AUC ranging between 0.70 and 0.94 [Figure 1]. Subsequently, we evaluated the predictive performance of the five models with external validation. The AUC of these models for the testing set ranged between 0.60 and 0.79 [Figure 2]. The deep learning model, which had the highest AUC (0.82, 95% confidence interval = 0.74–0.90) in the testing set achieved the best performance. Twenty-seven variables were selected for modeling since the model incorporating these variables had the best predictive power. The relative importance of the top 15 variables calculated by the deep learning model is shown in [Figure 3]. The lower respiratory tract infection (infection site of the lung) was the most important contributing factor, followed by peptide EN, and shock.
Figure 1: ROC curve of the training cohort. AUC of each model is 0.70 (95% confidence interval [CI]; 0.60–0.80) in Bayes Net model, 0.73 (95% CI 0.64–0.82) in Logistic regression model, 0.82 (95% CI 0.74–0.90) in Deep Learning model 0.92 (95% CI 0.87–0.97) in Random Forest model, and 0.94 (95% CI 0.90–0.99) in Gradient Boosting model.

Click here to view
Figure 2: ROC curve of the validation cohort. AUC of each model is 0.73 (95% CI 0.60–0.85) in Bayes Net model, 0.69 (95% CI 0.56–0.81) in Logistic regression model, 0.79 (95% CI 0.68–0.89) in Deep Learning model 0.63 (95% CI 0.50–0.76) in Random Forest model, and 0.60 (95% CI 0.47–0.74) in Gradient Boosting model.

Click here to view
Figure 3: The ranking of top 15 predictive variables in the deep learning model. COPD: chronic obstructive pulmonary disease

Click here to view

Discrimination and calibration

The calibration curve for the probability of FI in the overall datasets demonstrated good agreement between prediction and observation [Figure 4]. There was an overprediction of the incidence of FI when the risk was lower than 80%. Additionally, there was an underprediction of the incidence of FI when the risk was higher than 80%.
Figure 4: Calibration curve of deep learning model in overall datasets. The red line represents a perfect prediction by an ideal model. The black line represents the performance of the deep learning model. The calibration curve in overall cohorts shows the agreement between predicted (x-axis) and observed (y-axis) risk of feeding intolerance in ICU patients with sepsis.

Click here to view

Clinical use

The decision curve analysis for the deep learning model is presented in [Figure 5]. The decision curve showed that if the threshold probability of a patient was >15%, the net benefit of the deep learning model was better than the treat-all-patients scheme to predict FI. To enhance clinical usage, we published our deep learning model on
Figure 5: Decision Curve analysis for the deep learning model. The y-axis measures the net benefit. The red line represents the deep learning model. The gray line represents the assumption that all the patients have feeding intolerance. The black line represents the assumption that no patients have feeding intolerance.

Click here to view

[Figure 6] shows the screenshots of the web calculator with a clinical example from the website.
Figure 6: The deep learning-based model showed a risk of 19% for a patient with sepsis aged 69 years with a history of hypertension and diabetes, who had the following features: infection, pneumonia; APACHE II score, 22; albumin, 30.8 g/L; creatinine, 87 μmol/L; urea nitrogen, 8.75 mmol/L; lactic acid, 1.2 mmol/L; sedation and analgesia treatment; treatment with two types of antibiotics; intermittent feeding with short peptides; and intra-abdominal pressure, <12 mmHg.

Click here to view

   Discussion Top

The prevalence of FI in the ICU patients ranges from 2% to 75%,[12] and the prevalence of FI in patients with sepsis is approximately 35%.[13] The incidence of FI in this study was consistent with those reported in the previous studies. FI occurs frequently and is associated with higher mortality and longer ICU stay than FT.[12] There was no difference in the 28-day mortality between the FI and FT groups in this study, as the hospital mortality rate due to sepsis has decreased significantly, which has been attributed to innovations in ICU care.[14] However, numerous patients with sepsis cannot return to a meaningful quality of life owing to ICU-acquired weakness (ICU-AW).[15] Optimal nutrition is an essential intervention to address ICU-AW.[16] To seek optimal nutrition, early prediction of FI is of particular importance among patients with sepsis.

We constructed and validated a deep learning (ANN) model ( for the individualized prediction of FI in ICU patients with sepsis in this study. This model effectively predicts FI risk within 48 h from ICU admission, which can help ICU physicians take timely measures to prevent the occurrence of FI in patients with sepsis. Moreover, FI in septic patients is affected by various factors. This study demonstrated several clinical variables that can significantly predict FI in patients with sepsis, including infection site, nutrition type, shock, continuous feeding, coronary heart disease, antibiotic category, chronic obstructive pulmonary disease, ventilation, hypertension, cerebrovascular accident, intra-abdominal pressure, and analgesia. All these variables are clinically available.

The strongest predictor identified in our study by deep learning was the lower respiratory tract infection, which has never been reported previously. The lower respiratory tract was the leading site of infection in this study (94.35% of the patients in the training set, 78.87% of the patients in the testing set). In most cases, sepsis-associated with FI was secondary to pneumonia [Table 1]. Besides, most patients with abdominal sepsis in our department had abdominal compartment syndrome (ACS), which had contraindications for EN. Thus, the lower respiratory tract was the infection site in most patients enrolled in this study. A previous epidemiological survey has shown that sepsis with pneumonia was associated with a poor outcome.[8] We hypothesize that patients with sepsis secondary to pneumonia have a more intense inflammatory response, which may play a role in FI. Furthermore, most patients with pneumonia received mechanical ventilation, which is an independent risk factor for FI.[9],[17] Accordingly, we recommend early intervention for preventing FI in septic patients with pneumonia.

Additionally, we found that nutrition type is the second crucial predictive variable of FI. Short-peptide EN is predisposed to FI (38 [45.8%] in short-peptide EN, 48 [42.8%] in intact-protein formula) for patients with sepsis, especially diarrhea (28 [33.7%] in short-peptide EN, 16 [14.3%] in intact-protein formula) in our study. The SPIRIT trial by Jakob et al.[18] demonstrated that no differences in the number of diarrhea events were observed between the hydrolyzed protein EN and whole protein feeding. van Zanten et al.[19] combined the meta-analysis data[10] with the SPIRIT trial results (211 patients) and found no benefits with respect to the diarrhea incidence during the ICU stay, favoring peptide-based EN.[19] The high osmotic pressure of short-peptide EN may contribute to diarrhea. More studies are required to investigate whether short-peptide EN is beneficial for patients with sepsis. Moreover, we recommend an intact-protein formula for patients with sepsis in the ICU.

Shock was the third significant predictor, consistent with the previous studies. Mao et al.[11] showed that elevated serum lactate levels were more likely to be associated with FI development in the elderly patients with sepsis treated with vasopressors. The patients with septic shock who receive adequate fluid resuscitation and receive norepinephrine doses of <0.14 μg/kg/min may tolerate early EN.[20] The NUTRIREA-2 trial, the largest randomized controlled trial evaluating EN in septic shock, indicated that the EN group had a higher FI incidence than the parenteral nutrition (PN) group.[21] Societal guidelines offer varying recommendations for the use of EN in patients with septic shock. The recent European Society of Parenteral and Enteral Nutrition (ESPEN) guideline suggests early and progressive EN should be used in sepsis patients after hemodynamic stabilization.[22] The optimal dose and timing of initiation of feeding in patients with septic shock remain unclear.[23] More studies are required to determine the timing and dose of EN in patients with septic shock. Presently, enteral feeding needs to be evaluated according to FT on a case-by-case basis.

To the best of our knowledge, this is the first attempt to use the multilayer feedforward ANN model, a type of deep learning model, as a prediction tool in FI of patients with sepsis. Deep learning refers to the class of machine learning (ML) methods that make use of more abstract representations of the input data to perform a specific task.[24] ANN is an information management model that is similar to the biological nervous system function of the human brain.[25] Our ANN model was constructed by training with stochastic gradient descent using backpropagation. ANN models are flexible and of high predictive accuracy since we can choose different activation functions. In addition, ANN models have other features, including adaptive learning rate, rate annealing, dropout, and L1 or L2 regularization. They can also address the collinearity problem comparing with traditional regression models. We divided the dataset and trained different models using grid search in our study. Therefore, the ANN model outperformed the classical approach (logistic regression model) and other ML algorithms, including random forest, naive Bayesian, and gradient boosting tree. Thus, we selected ANNs as our final predictive model.

Our study has a few limitations. First, since this study was a retrospective, observational study, this study is subject to the disadvantages of a retrospective study. Second, as this was a dual center study, the results may not be generalizable to other populations within China or other countries. We suggest that in the future, multicenter studies should be conducted to validate this model further. Third, the sample size was small because of the strict inclusion and exclusion criteria. Thus, large sample sizes are needed in the future. The performance of a prediction model will be continuously improved when there are large sample sizes, benefiting from the strong self-learning ability of ANNs.

In conclusion, a new prediction model was developed and validated using deep learning, which can effectively predict the occurrence of EN intolerance in ICU patients with sepsis ( Through this predictive model, timely preventive measures can be undertaken by clinicians to reduce the occurrence of FI in patients with sepsis. The lower respiratory tract as the infection site is the strongest predictor for FI. Simple clinical information such as nutrient and septic shock type can be useful in stratifying a septic patient's risk of EN intolerance.

Financial support and sponsorship

This study was supported by grants from the Science and Technology Project Foundation of Nanning, no. 2020032 and Foundation of the Key Laboratory of Guangxi Zhuang Autonomous Region Health Commission, no. ZZH 2020013.

Conflicts of interest

There are no conflicts of interest.

   References Top

Singer M, Deutschman CS, Seymour CW. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016;315:801–10.  Back to cited text no. 1
Englert JA, Rogers AJ. Metabolism, metabolomics, and nutritional support of patients with sepsis. Clin Chest Med 2016;37:321-31.  Back to cited text no. 2
Reintam Blaser A, Starkopf J, Alhazzani W, Berger MM, Casaer MP, Deane AM, et al. Early enteral nutrition in critically ill patients: ESICM clinical practice guidelines. Intensive Care Med 2017;43:380-98.  Back to cited text no. 3
Heyland DK, Ortiz A, Stoppe C, Patel JJ, Yeh DD, Dukes G, et al. Incidence, risk factors, and clinical consequence of enteral feeding intolerance in the mechanically ventilated critically ill: An analysis of a multicenter, multiyear database. Crit Care Med 2021;49:49-59.  Back to cited text no. 4
Reintam Blaser A, Malbrain ML, Starkopf J, Fruhwald S, Jakob SM, De Waele J, et al. Gastrointestinal function in intensive care patients: Terminology, definitions and management. Recommendations of the ESICM working group on abdominal problems. Intensive Care Med 2012;38:384-94.  Back to cited text no. 5
Rhodes A, Evans LE, Alhazzani W, Levy MM, Antonelli M, Ferrer R, et al. Surviving sepsis campaign: International guidelines for management of sepsis and septic shock: 2016. Intensive Care Med 2017;43:304-77.  Back to cited text no. 6
Nasiri M, Farsi Z, Ahangari M, Dadgari F. Comparison of intermittent and bolus enteral feeding methods on enteral feeding intolerance of patients with sepsis: A triple-blind controlled trial in intensive care units. Middle East J Dig Dis 2017;9:218-27.  Back to cited text no. 7
Chou EH, Mann S, Hsu TC, Hsu WT, Liu CC, Bhakta T, et al. Incidence, trends, and outcomes of infection sites among hospitalizations of sepsis: A nationwide study. PLoS One 2020;15:e0227752. doi: 10.1371/journal.pone. 0227752.  Back to cited text no. 8
Huang HH, Chang SJ, Hsu CW, Chang TM, Kang SP, Liu MY. Severity of illness influences the efficacy of enteral feeding route on clinical outcomes in patients with critical illness. J Acad Nutr Diet 2012;112:1138-46.  Back to cited text no. 9
Canadian Clinical Practice Guidelines. 4.3 Strategies for optimizing and minimizing risks of EN: Whole protein vs. peptides. Semi quantitative scoring worksheet ( [Last accessed on 2017 Jun 14].  Back to cited text no. 10
Mao Z, Liu G, Yu Q, Qi S, Lou Y, Liu C, et al. Association between serum lactate levels and enteral feeding intolerance in septic patients treated with vasopressors: A retrospective cohort study. Ann Transl Med 2020;8:1240.  Back to cited text no. 11
Blaser AR, Starkopf J, Kirsimägi Ü, Deane AM. Definition, prevalence, and outcome of feeding intolerance in intensive care: A systematic review and meta-analysis. Acta Anaesthesiol Scand 2014;58:914-22.  Back to cited text no. 12
Lavrentieva A, Kontakiotis T, Bitzani M. Enteral nutrition intolerance in critically ill septic burn patients. J Burn Care Res 2014;35:313-8.  Back to cited text no. 13
Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000–2012. JAMA 2014;311:1308–16.  Back to cited text no. 14
Wischmeyer PE. Nutrition therapy in sepsis. Crit Care Clin 2018;34:107-25.  Back to cited text no. 15
Wischmeyer PE, San-Millan I. Winning the war against ICU-acquired weakness: New innovations in nutrition and exercise physiology. Crit Care 2015;19 Suppl 3(Suppl 3):S6.  Back to cited text no. 16
Zhang SW, Wang H, Su Q, Wang BE, Wang C, Yin CH; MODS Research Group. Clinical epidemiology of 1,087 patients with multiple organ dysfunction syndrome. Chin Crit Care Med 2007;19:2-6.  Back to cited text no. 17
Jakob SM, Bütikofer L, Berger D, Coslovsky M, Takala J. A randomized controlled pilot study to evaluate the effect of an enteral formulation designed to improve gastrointestinal tolerance in the critically ill patient-the SPIRIT trial. Crit Care 2017;21:140.  Back to cited text no. 18
van Zanten ARH, Elke G. Hydrolysed protein enteral nutrition is not superior to polymeric whole protein feeding with regard to gastrointestinal feeding tolerance and feeding adequacy. Crit Care 2017;21:232.  Back to cited text no. 19
Merchan C, Altshuler D, Aberle C, Papadopoulos J, Schwartz D. Tolerability of enteral nutrition in mechanically ventilated patients with septic shock who require vasopressors. J Intensive Care Med 2017;32:540-6.  Back to cited text no. 20
Reignier J, Boisramé-Helms J, Brisard L, Lascarrou JB, Ait Hssain A, Anguel N, et al. Enteral versus parenteral early nutrition in ventilated adults with shock: A randomised, controlled, multicentre, open-label, parallel-group study (NUTRIREA-2). Lancet 2018;391:133-43.  Back to cited text no. 21
Singer P, Blaser AR, Berger MM, Alhazzani W, Calder PC, Casaer MP, et al. ESPEN guideline on clinical nutrition in the intensive care unit. Clin Nutr 2019;38:48-79.  Back to cited text no. 22
Patel JJ, Rice T, Heyland DK. Safety and outcomes of early enteral nutrition in circulatory shock. JPEN J Parenter Enteral Nutr 2020;44:779-84.  Back to cited text no. 23
LeCun Y, Bengio Y, Hinton G. Deep learning. Nature 2015;521:436-44.  Back to cited text no. 24
Wang D, He H, Liu D. Adaptive Critic Nonlinear Robust Control: A Survey. IEEE Trans Cybern. 2017;47:3429-51.  Back to cited text no. 25

Correspondence Address:
Prof. Liao Pinhu
6 Shuangyong Rd, Nanning-530021 Guangxi Province
Login to access the Email id

Source of Support: None, Conflict of Interest: None

DOI: 10.4103/sjg.sjg_286_21

Rights and Permissions


  [Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6]

  [Table 1]


    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Email Alert *
    Add to My List *
* Registration required (free)  

   Patients and Methods
    Article Figures
    Article Tables

 Article Access Statistics
    PDF Downloaded166    
    Comments [Add]    

Recommend this journal