A nomogram for individualized estimation of survival among adult patients with adrenocortical carcinoma after surgery: a retrospective analysis and multicenter validation study

Background Clinical outcome of adrenocortical carcinoma (ACC) varies because of its heterogeneous nature and reliable prognostic prediction model for adult ACC patients is limited. The objective of this study was to develop and externally validate a nomogram for overall survival (OS) prediction in adult patients with ACC after surgery. Methods Based on the data from the Surveillance Epidemiology, and End Results (SEER) database, adults patients diagnosed with ACC between January 1988 and December 2015 were identified and classified into a training set, comprised of 404 patients diagnosed between January 2007 and December 2015, and an internal validation set, comprised of 318 patients diagnosed between January 1988 and December 2006. The endpoint of this study was OS. The nomogram was developed using a multivariate Cox proportional hazards regression algorithm in the training set and its performance was evaluated in terms of its discriminative ability, calibration, and clinical usefulness. The nomogram was then validated using the internal SEER validation, also externally validated using the Cancer Genome Atlas set (TCGA, 82 patients diagnosed between 1998 and 2012) and a Chinese multicenter cohort dataset (82 patients diagnosed between December 2002 and May 2018), respectively. Results Age at diagnosis, T stage, N stage, and M stage were identified as independent predictors for OS. A nomogram incorporating these four predictors was constructed using the training set and demonstrated good calibration and discrimination (C-index 95% confidence interval [CI], 0.715 [0.679–0.751]), which was validated in the internal validation set (C-index [95% CI], 0.672 [0.637–0.707]), the TCGA set (C-index [95% CI], 0.810 [0.732–0.888]) and the Chinese multicenter set (C-index [95% CI], 0.726 [0.633–0.819]), respectively. Encouragingly, the nomogram was able to successfully distinguished patients with a high-risk of mortality in all enrolled patients and in the subgroup analyses. Decision curve analysis indicated that the nomogram was clinically useful and applicable. Conclusions The study presents a nomogram that incorporates clinicopathological predictors, which can accurately predict the OS of adult ACC patients after surgery. This model and the corresponding risk classification system have the potential to guide therapy decisions after surgery.


Background
Adrenocortical carcinoma (ACC) is a rare disease in both pediatric and adult patients with an overall incidence of 0.5-2.0 cases per million people per year [1]. Complete surgical resection is considered as the main curative form of treatment for localized ACC [2]. However, ACC is a vicious tumor with a high degree of malignancy and recurrence rate [3][4][5][6]. Its 5-year overall survival (OS) rate is estimated to range between 16 and 60% [7][8][9].
Adjuvant therapy including mitotane has demonstrated the potential to improve the prognosis of ACC patients [10][11][12], although confirmatory randomized, prospective trials on adjuvant therapy are yet to be published. If the prognosis of ACC could be accurately predicted, comprehensive treatment would be timely given to high-risk patients to improve their survival outcome. The American Joint Committee on Cancer (AJCC) TNM staging system is globally recognized and implemented to estimate the survival of ACC patients [13,14], but it is largely constrained by its inability to consider other determining clinicopathological factors, such as age, gender, and tumor size, which may also have considerable impact on the patients' survival [15,16].
Only a few studies have established prediction models for clinicians and researchers to access the prognosis of ACC patients because of its low incidence [17][18][19][20]. However, these studies were partly limited for clinical applicability as in some, the patient's age was used as a categorical variable rather than continuous variable [17,18], in others, the cases with insufficient data (data of radiation therapy, chemotherapy, and histologic grade et al.) were not excluded for analysis or even lacked external validation [17,20]. In addition, another important determining limitation was that these proposed models were developed using the data of ACC patients of all ages and thereby neglected the differences in prognosis predictors between pediatric and adult patients [17][18][19][20]. In fact, adult and pediatric ACC patients are different not only in incidence and clinical presentation but also in some aspects of biological behaviors. ACC in adult patients is more aggressive and is associated with poorer clinical outcomes despite undergoing complete surgical resection as compared to pediatrics ACC. The 5-year survival rate in adult patients was reported of being 37%-39% and in pediatric patients 53%-56% [21,22]. One study analyzing the data from the Surveillance, Epidemiology and End Results (SEER) database found that the overall 5-year survival of ACC patients in adults was mediocrely between 30 and 40% while that of pediatrics was 57% [23]. The genomic characteristics of ACC are also different between pediatric and adult patients. For instance, germline TP53 mutations are less common in adults with ACC, and IGF2 overexpression is a marker of poor prognosis in adult ACC patients, but not in pediatric patients [24,25]. Meanwhile, adult patients seem to have less obvious symptoms of hormonal overproduction, i.e. virilization and precocious puberty, and have clear-cut pathological criteria for malignancy [26] meaning that tumors among adult patients can be adequately classified based on the Weiss or Van Slooten scores. As such, an easy-to-implement model for prognostication of the postoperative survival tailored for adult ACC patients is greatly needed to provide more personalized treatment, especially for high-risk patients.
In the present study, we aimed to develop a nomogram for predicting the survival of post-operative adult ACC patients using the SEER database and to validate it using external validation using the Cancer Genome Atlas (TCGA) database and a multicenter Chinese cohort for wider clinical application.

Patients and data collection
In this multicenter retrospective study, three independent datasets of adult ACC patients were retrieved. The cases recruitment methodology is illustrated in Fig. 1 were used for external validation. For the Chinese cohort, the retrospective analysis of anonymous patient data was approved by the institutional review board at each participating institution. Due to the retrospective nature of this study, informed consent was not required and patients' data were used anonymously.
Demographic and clinicopathological data including age at diagnosis, gender, tumor laterality, tumor size, TNM stage, tumor stage group, survival status, and survival time were retrieved. TNM stage was defined according to the UICC/AJCC TNM Classification. The tumor stage group was defined based on the 7th AJCC staging system and the European Network for the Study of Adrenal Tumors (ENSAT) staging system consistent with the 8th AJCC staging system. The main outcome was OS, defined as the time from the date of diagnosis to the date of death or last follow-up.
Of note, the SEER data were accessed using the SEER*Stat version 8.3.5 software on January 3, 2019, and data from the TCGA set were downloaded from the TCGA-ACC project on January 23, 2019 (https ://porta l.gdc.cance r.gov/). For the Chinese cohort, the data were censored on December 31, 2018.

Development of the nomogram
In the training set, clinicopathological predictors were tested using the univariable Cox proportional hazards regression analyses. Three models for OS prediction using multivariable Cox proportional hazards regression analyses were developed. Model 1 incorporated the TNM stage, while models 2 and 3 incorporated the 7th AJCC stage group and ENSAT stage group, respectively. Backward stepwise selection was applied by using the Akaike's Information Criterion (AIC) as the stopping rule [27] and age at diagnosis was included in all three models. The discrimination accuracy of the models was quantified using the Harrell's concordance index (C-index) [28]. The optimal model was selected by comparing their C-indices and based on which the nomogram was developed.
Performance assessment of the nomogram in the training set C-index was obtained to quantitatively evaluate the discriminative ability of the nomogram. Calibration curves were plotted to assess the calibration of the nomogram. Bootstrapping using 1000 resampling procedures was applied to calculate the C-index that was corrected for potential overfitting.

Validation of the nomogram
The performance of the nomogram was validated using the SEER internal validation dataset and externally validated using the TCGA and Chinese dataset. The multivariate Cox proportional hazards regression formula of the nomogram formed in the training set was applied to the patients in the validation sets, with risk scores calculated for each patient to reflect the risk of cancer mortality. Cox proportional hazards regression analyses were performed using the risk scores in the validation sets. The discrimination and calibration of the nomogram were then assessed based on the regression analyses to validate its performance.

Survival risk classification based on the nomogram
In the training dataset, the optimal risk score for ACC mortality cutoff value was identified using the X-tile plots [29]. Based on the value obtained, all patients were classified into a high-and low-risk group. The Kaplan-Meier method and log-rank test were used to assess and compare the OS of adult ACC patients after surgery in the different risk groups. Stratified analyses were also performed within the various subgroups according to sex and tumor location.

Clinical usefulness of the nomogram
Decision curve analysis (DCA) was performed by calculating the net benefits for a range of threshold probabilities to estimate the clinical usefulness of the nomogram. The DCA algorithm, a validated approach, was utilized for evaluating alternative diagnostic and prognostic strategies [30].

Statistical analysis
The X-tile software version 3.6.1 (Yale University, New Haven, CT, USA) was used to determine the optimal risk score cutoff value. All other computations were conducted using the R software, version 3.5.2 (The R Foundation for Statistical Computing, https ://www.r-proje ct.org/). The Cox proportional hazards regression analyses were performed by the R software "survival" and "MASS" packages. The nomogram and calibration plots were produced using the "rms" package. The DCA was performed using the function "stdca.R". Statistical significance was set at P values less than 0.05 in a two-tailed test.

Patient characteristics
In total, 722 eligible ACC patients from the SEER database were identified and classified as the SEER training set (n = 404) and the SEER internal validation set (n = 318). There were also two external validation sets, namely the TCGA validation set (n = 82) and the Chinese multicenter validation set (n = 82). The patients' characteristics of the training and three validation datasets are shown in  Table 2 shows the findings of univariate and multivariate analyses in the training set. Age at diagnosis, ENSAT stage group, and 7th AJCC T, N, M and TNM stage were found to be significantly associated with OS. As for the multivariate analyses, age at diagnosis was included in all three models. Model 1 incorporated T stage, N stage, and M stage, while models 2 and 3 incorporated the 7th AJCC TNM and ENSAT stage group, respectively. The Cox regression coefficients of each included factors in the three models are displayed in Table 3. The C-indices of the models are listed in  (Fig. 2a). The calibration curves for the 1-, 3-and 5-year OS showed favorable calibration of the nomogram in the training set (Fig. 2b).

Validation of the nomogram
The favorable discrimination ability of the nomogram was validated in the SEER internal validation dataset (C-index [95% CI], 0.672 [0.637-0.707]). In addition, the performance was also confirmed in the TCGA and Chinese multicenter external validation set, with C-indices of 0.810 (95% CI 0.732-0.888) and 0.726 (95% CI 0.633-0.819), respectively. Good consistency was also observed between actual survival data and the nomogram prediction in the three validation datasets (Fig. 2c-e). Therefore, the presented nomogram performed well in both the training and validation sets.

Survival risk classification based on the nomogram
The X-tile plots showed that the optimal mortality risk score cutoff value was 2.96 (Fig. 3), and was used to classify the patients into a high-(risk score ≥ 2.96) and low-risk group. Kaplan-Meier curves for survival outcomes of the different risk subgroups showed significant distinction in survival probability in the training set (Fig. 4a, P < 0.001); which was further confirmed in the three validation datasets (Fig. 4b-

Clinical usefulness of the nomogram
DCA analysis was performed to illustrate the net benefit at 5 years in each cohort. When the threshold probabilities exceeded 21% in the SEER training set, ranged between 34% and 95% in the SEER internal validation set, exceeded 6% in the TCGA validation set and 12% in the Chinese multicenter validation set, the use of the nomogram to predict the prognosis of adult ACC patients provided greater net benefit than the "treat all" or "treat none" strategies, indicating the favorable potential clinical applicability of the nomogram (Fig. 6).

Discussion
In the present study, a nomogram incorporating age at diagnosis, T stage, N stage, and M stage was developed to predict the OS probability for adult ACC patients after surgery and was externally validated using multiethnicity and multicenter datasets. The nomogram showed good discrimination and calibration in both the training and validation datasets. Also, the DCA revealed it had promising clinical applicability. Thus, the constructed nomogram can provide an easy-to-use and individualized tool to help physicians to make more informed treatment decisions for treating adult ACC patients. Concerning the development of ACC, the only curative approach to ACC is complete tumor resection. However, the 5-year survival after surgery for ACC range between 16 and 60% [22,31,32], showing the prognostic heterogeneity associated with this disease. Some studies reported that adjuvant therapy in localized disease may provide survival benefits [10,[33][34][35]. However, the  necessity of adjuvant therapy remains elusive. Therefore, accurate prognostic predication after surgery for adult ACC patients is significant not only for the adjuvant treatment selection but also to inform patients about their long-term prognoses. However, there lacked a clear optimal method in present literature to predict the outcome of ACC patients and stratify them into different risk subgroups. Several, but debatable, factors related to ACC prognosis were identified in previous studies. Some have reported that there were no correlations between age, sex, tumor size to the outcome of ACC [10,36,37], while others showed that age, sex, high tumor grade, and tumor size were significantly associated with prognosis [18,31,38]. Indeed, all of them focused on ACC patients of all ages (adults and pediatrics). The number of pediatric ACC patients was roughly about 12% to 20% of the total investigated cohort from these studies. Therefore, different biological behavior and clinical presentations between adult and pediatric ACC patients might have accounted for these inconsistent results. In contrast, in this present study, only adult patients were investigated and we found that age at diagnosis, T stage, N stage and M stage were independent predictors of OS after surgery. Similar to our findings, there have been other studies reporting that old age was a poorer prognostic factor for OS in adults as compared to the young patients [14,36,39]. OS might be affected by age not only related to the clinical course of the disease, but also for age-related complications [40]. Notably, a recent report has proposed a novel staging system incorporating patients' age and was not based on the patient's tumor size [14], because age at diagnosis may better inform clinicians about proper individualized treatment and prognostication. Also, in this present study, the TNM stage contributed as a main part of the final risk score and demonstrated better prognostic performance when combined age. Our nomogram had superior prognostic ability compared to the AJCC and ENSAT stage group models (Table 4). Also, its discrimination and calibration displayed good performance and was validated using internal and external validation datasets. Thus, it has the potential to be implemented in realworld clinical practice.
Further, the formulated nomogram can be comprehensively used for individualized treatment planification due to its potential to accurately stratify adult ACC patients based on their mortality risk [5,8,41] into two distinct prognostic groups, namely high-and low-risk groups. To the best of our knowledge, this is the first nomogram for   X-tile plots to identify the optimal risk score cutoff based on OS in the SEER training set. a X-tile plot for the training set. The X-tile plot was generated by dividing risk scores into three populations (low, middle and high) or two populations (low and high). Each pixel (point) of the X-tile plot represents the data from a given set of divisions. The X-axis represents all potential risk score cutoff from low to high (left to right) that define a low subset, whereas the Y-axis represents risk score cutoff value from high to low (top to bottom), that define a high subset. The arrows represent the direction in which the low subset (X-axis) and the high subset (Y-axis) increase in size. Data along the hypotenuse represent results from a single cutoff value that divides the data into high or low subsets. The coloration of the plot represents the strength of the association at each division, ranging from low (dark, black) to high (bright, red or green). Inverse associations between the risk score and survival are colored red, whereas direct associations are colored green. b The distributions of the number of patients by risk score. c Kaplan-Meier plots categorized by the low-risk and high-risk groups according to the optimal risk score cutoff. The optimal OS risk score cutoff was determined as 2.96 (χ 2 = 97.7, P < 0.001) Compared with other prognostic models, our model was validated in three independent validation cohorts with promising results. The favorable discriminating ability of the nomogram in all validation sets supports its generalizability for routine clinical use. Some limitations of the present study were as follows. First, this study may be potentially limited due to its retrospective nature and associated with inherent biases. We excluded patients with missing data during data collection as their inclusion would have simultaneously affected the credibility of the results. Second, the multivariable model did not include some potential prognostic predictors, such as the hormone status, Ki-67 index, Weiss score, SF-1, calretinin, and SRC1, because these informations were not uniformly available in the retrieved datasets. A more comprehensive model considering all potential risk factors might be expected to have better prognostic performance. Third, the follow-up time was shorter in the Chinese multicenter validation dataset, and close monitoring and five-year follow-up data are still required for these patients. Kaplan-Meier survival curves categorized into low-risk and high-risk groups in stratified analyses for the entire study cohort. Significance between the OS of the high-risk and low-risk patients was observed in both sex a male and b female, and tumor location, c left-sided ACC, d right-sided ACC. OS overall survival, ACC adrenocortical carcinoma

Conclusions
In conclusion, we have developed a nomogram able to predict the postoperative OS tailored for adult ACC patients. The nomogram demonstrated favorable predictive accuracy and clinical usefulness after validation in datasets comprised of different populations and ethnicity. The proposed nomogram is an easy-to-use tool with promising clinical applicability to provide individualized patient counseling, timely surveillance, and clinical assessments.  SEER internal validation; c TCGA validation; d Chinese multicenter validation set. The horizontal solid black line represents the assumption that no patients will experience the event, and the solid gray line represents the assumption that all patients will relapse. On decision curve analyses, the nomogram showed superior net benefit in all different cohorts across a range of threshold probabilities. DCA decision curve analysis, SEER Surveillance Epidemiology, and End Results database, TCGA the Cancer Genome Atlas set