Artificial Neural Networks Versus Multiple Logistic Regression to Predict 30-Day Mortality After Operations For Type A Ascending Aortic Dissection§

INTRODUCTION

Acute aortic dissection (AAD) is a lethal condition requiring emergency surgical therapy, mainly replacing the ascending aorta and/or the aortic arch with or without aortic valve replacement [1]. Short-term mortality of medically treated patients after Type A dissection is high, peaking >50% in the first 48 hours following presentation [2]. In-hospital mortality of operated patients ranges 7-30% in recent series [1-11] and several factors have been considered to contribute to mortality risk [1, 2, 5-7, 11]. Among these, large lists of pre-operative and intra-operative factors have been studied [1-12]. The International Registry of AAD, collecting data from 18 referral centres worldwide identified a preoperative risk stratification scheme and a real average 25.1% figure for surgical mortality in AAD Type A in the current era by investigating 290 clinical variables and their relationship to surgical outcome in 526 of 1032 patients enrolled from 1996 to 2001 [2, 11]. Independent preoperative predictors of operative mortality were history of aortic valve replacement, migrating chest pain, hypotension as sign of AAD Type A, preoperative cardiac tamponade and preoperative limb ischemia [11]. In other investigations, techniques of surgical procedure have been considered among possible risk factors and the impact of retrograde, anterograde or selective cerebral perfusion after circulatory arrest was studied [3, 4, 7-9], although the results were not clear-cut to define the role of these as risk factors for short-term post-operative mortality.

When predictive modelling is used to challenge a very large database where a quantity of patients with a known outcome contributed a long list of variables, the overall predictive accuracy rarely exceeds 75% which has been repeatedly shown in the surgical domain by logistic regression (LR) based outcome predictive models after coronary artery bypass grafts (CABG) [13-15]. More recently, artificial neural networks (NN) were used for risk factor identification and mortality prediction in CABG surgery [16-18], aimed at ameliorating overall predictive accuracy in comparison to LR [18].

We investigated 121 patients consecutively operated during 7 years at a single Cardiac Surgical University Centre and run for the first time a NN model aimed at assessing the predictive potential of 30-day mortality by a large series of previously proposed risk factors, including surgical techniques, also comparing the performance of this model with standard LR results, using a well accepted metric [16, 17]. According to what has recently been reported with outcome predictive models following CABG surgery [18], NN model performance was comparatively better here than LR model performance. Moreover, when NN model was re-applied [19, 20] to data from a second Regional Hospital Centre where 87 patients were operated during a similar time period, there was a further increase in the performance. We therefore propose a list of potential risk factors to predict 30-day mortality after AAD Type A by NN model, tested on two independent series of patients, as methodology dictates in case of NN modelling, which now waits being confirmed also outside Italy for larger widespread use.

METHODS

RESULTS

Table 2 shows univariate differences between survivors and 30-day post-operative deaths following AAD Type A among the grouped 208 patients operated in the 2 Centres of the present study. The comparison with Table 1 illustrates a frequent situation in the Literature whereby out of a series of 22 potential risk factors selected from a larger series of 37 assessed previously (Table 1), based on a significant criterion defined in one Centre, the simple merging of data with those from a second Centre, decreased the number of significant univariate potential predictors from 22 to 9.

Of the 208 overall patients operated in the 2 Centres (respectively Rome and Sant’Anna Hospital) between 2001 and early 2008, the most frequently interventions performed were those on ascending aorta (respectively 56 and 38 patients) followed by those on ascending aorta and aortic valve (respectively 21 and 11 patients), those on ascending aorta and hemiarch (respectively 15 and 7 patients) and those on ascending aorta and arch (respectively 10 and 12 patients). There were relatively few Cabrol (respectively 3 and 1 patients) as compared to Bentall (respectively 16 and 17 patients) surgery. There was a notable difference (see also Table 3 for the coded significant difference) between the 2 Centres in the proportion of anterograde brain perfusion performed (47 of 121 versus 67 of 87) versus the proportion of retrograde brain protection (respectively 73 of 121 versus 14 of 87). However, the duration of extracorporeal circulation was similar (respectively 211±81 versus 216±76 min) in spite of significantly shorter circulatory arrest time in Sant’Anna Hospital (25±23 min) than in Rome (53±34 min, p<0.001). On the other hand, 24-hour postoperative bleeding was only moderately higher in Rome (1022±986 ml) than in Sant’Anna Hospital (768±638 ml, p<0.05), which was unrelated to the type of intervention performed (data not shown). Table 3 also shows other slight differences between data obtained in the 2 Centres.

Tables 3 and 4 present respectively for the Rome and the Sant’Anna Hospital Centres, the forced LR results whereby the 22 potential factors were assessed to see which was a multivariable predictor of 30-day post-operative mortality. Interestingly enough, these tables show that the forced LR solutions did not present in the Rome and Sant’Anna Hospital Centres the same significant covariates to predict 30-day post-operative mortality. Thus in the Rome Centre, previous cardiac surgery (p=0.0246) and type of brain protection (anterograde vs retrograde: p=0.0443) were predictors whereas in the Sant’Anna Hospital Centre only the pre-operative presence of shock was a predictor (p=0.0082), respectively among 120 and 87 patients with complete data. The forced LR solution of Table 3 provided an AUC 87.9±4.1% with 95% CI from 80.7 to 93.2% which was not statistically different from the AUC obtained by the forced LR solution of Table 4 (85.7±5.2% with 95% CI from 78.5 to 91.1%).

By Tiberius software it was then ascertained whether stepwise solutions of LR or NN models might differ. Fig. (1) shows in a compact way that these predictive models do not select the same predictors neither are they ranked similarly. In fact, among significant predictors to be kept in stepwise solutions among the 120 AAD Type A operated patients from the Rome Centre, LR model selected in rank order: immediate post-operative presence of dialysis in continuous, renal complications, chronic renal failure, coded operative brain protection (anterograde better than retrograde perfusion), pre-operative neurological symptoms, age, previous cardiac surgery, the length of extracorporeal circulation, the operative presence of hemopericardium and post-operative enterological complications. On the other hand, NN model selected in rank order: immediate post-operative presence of chronic renal failure, coded operative brain protection (anterograde better than retrograde perfusion), post-operative presence of dialysis in continuous, pre-operative neurological symptoms, post-operative renal complications, the length of extracorporeal circulation, age, the operative presence of hemopericardium, pre-operative presence of intubation, post-operative limb ischemia and enterological complications and the year of surgery. There were 9 predictors selected by both models although with different rank order, whereas previous cardiac surgery was selected among significant stepwise predictors by LR only, and NN model detected the pre-operative presence of intubation, post-operative limb ischemia and the year of surgery as further significant predictors.

Since out of the Rome Centre experience forced and stepwise LR solutions did select similar potential risk factors for 30-day mortality, and post-operative chronic renal failure was defined as a significant predictor by both forced LR and stepwise NN model, it was further ascertained whether the overall accuracies of these latter solutions might differ. There was no significant difference (p=0.553) between the AUC of the forced LR solution of Table 3 and the AUC obtained by the stepwise NN solution shown in Fig. (1) (90.5±3.7% with 95% CI from 83.8 to 95.1%). On the other hand, the Gini’s coefficients obtained from the Rome Centre data with stepwise models were different, the larger being by NN than LR model (0.816 versus 0.712). This means that using the 12 covariates selected by the stepwise NN model enabled a more accurate prediction of 30-day mortality, compared with that seen after the stepwise LR model selection which used 10 covariates, out of 22 covariates measured among 120 patients from the Rome Centre.

Fig. (2) shows the re-application to Sant’Anna Hospital data of the LR and NN models constructed on the Rome Centre data. LR model selected in rank order: immediate post-operative presence of dialysis in continuous, neurological deficits, pre-operative presence of intubation, the quantity of post-operative bleeding in the first 24 h, the presence of pre-operative shock and neurological symptoms. NN model selected in rank order: the presence of pre-operative shock, the length of extracorporeal circulation, post-operative chronic renal failure, the year of surgery, pre-operative presence of intubation, immediate post-operative presence of dialysis in continuous, the quantity of post-operative bleeding in the first 24 h, and the presence of pre-operative neurological symptoms. Interestingly, 5 predictors were selected by both models, when re-applied to Sant’Anna Hospital data, although with different rank order, whereas post-operative neurological deficits was presented among significant stepwise predictors by LR only, and NN model detected the length of extracorporeal circulation, post-operative chronic renal failure and the year of surgery as further significant predictors. Again, the Gini’s coefficients obtained from the Rome Centre model applied to Sant’Anna Hospital data with stepwise models were different, the largest being by NN (0.850 versus 0.761 of the LR re-applied model).

DISCUSSION

The incidence of AAD has been estimated at from 5 to 30 per million people per year in the United States, which is 880 to 147 times less than the incidence of acute myocardial infarction (AMI), but still provides an important clinical problem and sometimes a dilemma for the differentiating difficulties between these presentations [2]. In addition, since AAD and AMI advocate for very distinct therapeutic strategies, risk profiling may be important for the former and might rapidly dictate surgical treatment aimed at offering the chance of decreasing short-term mortality from as high as 58% in medically treated patients to the current average figure of 25.1% (and sometimes less) when surgery is performed [2]. The International Registry of AAD [2, 11] confirmed that patient selection plays an important role in determining surgical outcomes in patients with Type A presentation and that knowledge of significant risk factors for operative mortality may contribute to better management and a more defined risk assessment in affected patients. Indeed, the overall in-hospital mortality was as high as 31.4% in unstable patients categorized in presence of cardiac tamponade, shock, congestive heart failure, cerebro-vascular accident, stroke, coma, acute myocardial and/or mesenteric ischemia and acute renal failure at the time of operation, which contrasts with 16.7% in stable patients.

Nevertheless, it is important to underline that the global overall accuracy with the abovementioned series of covariates, explored by standard LR, while useful to define high risk individuals, was relatively unsatisfactory, similar to what seen in protocols investigating high risk in CABG surgery where ROC AUC around 0.75 or less were in general reported [13-15]. This prompted to look after different methods such as neural networks [16-18], which were however not previously, studied in AAD Type A patients.

We have analyzed a series of 22 demographic, pre-operative, operative and post-operative characteristics, selected from 37 such variables that previous investigations have considered (Table 1) as potential predictors of 30-day mortality after AAD Type A among 208 patients operated in 2 Italian Centres. By forced and stepwise LR model along with by stepwise NN model we defined several covariates which may help predict 30-day mortality among operated patients, depending on the model’s type and method and on the Centre where the patients were treated (Tables 3 and 4; Figs. 1 and 2). When models were produced in one Centre and applied in a second Centre there were predictors which were selected in common. These include (Fig. 2): the presence of pre-operative shock, intubation and neurological symptoms, immediate post-operative presence of dialysis in continuous and the quantity of post-operative bleeding in the first 24 h. By NN model, the length of extracorporeal circulation, post-operative chronic renal failure and the year of surgery were specifically detected. Our data confirm and extend the results of the International Registry of AAD [2, 11] since also operative and immediate post-operative factors were observed as potential predictors of short-term mortality. Furthermore, a quite accurate overall prediction was observed in our study, based on AUC and Gini’s coefficients results, pointing to the applicability of NN model prediction of post-operative mortality after AAD Type A, which extends the results obtained with this method in CABG surgery [16-19].

The predictive power of risk factors with multivariable models such as the multiple logistic function, the proportional hazards life table Cox model, the Poisson model, and the Weibull life table model, all of which are now standard methods [21-25], is an important aspect of risk stratification after surgery [14, 15]. More recently, other methods have arisen to perform multivariable predictions of bins, including multilayer feed-forward networks [26]. Artificial NN have become popular in medical applications [16-20], due to relatively widespread availability of free-, share-, and commercial-ware [see: http://neuralnetworks.ai-depot.com/Software.html] and the recent increase of personal computer power. There has been an effort to cope with the limitations of methods such as logistic regression [24], whose predictive accuracy rarely exceeded 75% when appropriately tested by ROC curves and comparisons [28-32], in the majority of epidemiological or clinical cardiovascular investigations [13-18, 36-41].

When the performance and/or reliability of predictive models is limited, or of low sensitivity and specificity, their capability may be hampered to identify high risk subjects who deserve individualized treatment [27]. The neural network method stems [19, 20] from its potential for improved predictive performance by exploring, hidden layers to find nonlinearities, interactions and nonlinear interactions among predictors. The attraction of neural networks is quite evident from the impressive growth of results published with these methods in the last 18 years [20]. However, there are relatively few comparative reports on the performance and accuracy of neural networks, which was assessed only versus multiple logistic function, to predict events in clinical [18] or epidemiological [36] cardiovascular studies and none has been performed on different populations once the respective predictive models were produced out of an index cohort, which may be seen as a critical deficiency [27].

What we have shown here for the first time is that a NN model was apt at assessing the 30-day mortality predictive potential of a large series of previously proposed risk factors [1,2,5-7,11], including surgical techniques, and this better than by standard LR, as far as the ROC AUC metric is concerned [16,17], similar to outcome reports after CABG surgery [13-18]. When NN model was re-applied to data from a second Regional Hospital Centre, which is a specific imperative of NN methodology [19, 20] that in a different context was not performed previously [18], there was a further increase in the performance. However, the list of potential risk factors to predict 30-day mortality after AAD Type A by NN model, tested on two independent and large series of patients, might be lacking of more sensitive factors, similar to The International Registry of AAD [2,11]. Among these, anatomo-surgical covariates [12] need certainly further study. Therefore, adoption of NN model and the list of predictive covariates selected here need a test outside Italy before larger widespread use.

Some limitations have to be considered here. Although we applied NN methodology of selecting a model in a first group of patients and then applied this into a second independent group of patients from the same nation-based population, both with similar 30-day mortality (Table 2 27.3 and 22.9% in Rome and Catanzaro, respectively), the overall 208 patients included in this investigation may represent a relatively small sample size. This calls for the assessment, among larger samples recruited in Italy or abroad, to confirm the reproducibility of our results. It might be important to extend the censoring from 30 to 90 days since several patients, alive by 30 days, may continue to vegetate in care centres, due to neurological damage. Whereas this may further cooperate to the validity of the predictive models to be then produced, no doubt that longer-term (1- to 5- year or more) survival prediction may also be important. However, it is at present unknown whether similar risk factor may apply as in case of short-term prediction. Finally, there might be different options to consider and take care of covariates, specifically those considered here as coded variables. However, whereas this may further extend the covariates number (which will demand a much larger sample to be studied), it is not clear, based on a preliminary but unreported evidence from these data, whether this is strictly needed for NN model to fit adequately.

In conclusion, an important take-home message for clinicians should be that with neural networks a globally more accurate prediction might be obtained, which deserves further study, although the list of potentially important predictive factors in AAD Type A patients is still elusive and more data from anatomo pathohysiological sources may probably still be needed and demand further investigations.

REFERENCES