1. Introduction

Kidney cancer is among the ten most common cancers worldwide ⁽¹⁾, ⁽²⁾; unfortunately, it is hard to detect early through normal clinical means. It is not a single disease; instead, it comprises different histologically and genetically distinct types of cancer, each with its own histologic type, which in turn has its own clinical course and therapy responses ⁽³⁾, ⁽⁴⁾. The Cancer Genome Atlas (TCGA) Research Network has conducted a series of comprehensive molecular characterizations in distinctive histologic types of kidney cancers ⁽⁵⁾.

Kidney cancer is the 6th most frequent cancer in males and the 10th in females, representing 5% and 3% of all new cases, respectively ⁽⁶⁾. Gender disparities in kidney cancer incidence have been reported, with a higher incidence and worse outcome in males ⁽⁷⁾. Over half of all people aged 50 have cysts, which are fluid-filled and are usually benign (noncancerous) and do not need treatment ⁽⁸⁾. Solid tumors of the kidney are rare; however, approximately three-quarters of these tumors are cancerous, with a potential to spread ⁽⁹⁾. According to the Centers for Disease Control (CDC), in the United States in 2014, black men were the most likely to get kidney cancer (24.7 per 100,000), followed by white men (22.0 per 100,000). Among women, African-American women are the most likely to get kidney and renal pelvis cancers (12.4 per 100,000), followed by Hispanic women (11.9 per 100,000) ⁽¹⁰⁾. Studies suggest that the distribution of kidney cancer subtypes differs between racial groups ⁽¹¹⁾, ⁽¹²⁾. Race and ethnicity cause inter-tumoral heterogeneity in cancers, ranging from disease incidence, morbidity, and mortality rates to treatment outcomes ⁽¹³⁾, ⁽¹⁴⁾. Therefore, the identification of population-specific molecular biomarkers is essential ⁽¹⁵⁾.

Identifying genes that contribute to the prognosis of cancer patients is one of the challenges faced while providing appropriate treatment for patients. The critical challenges in bioinformatics are searching for biomarkers that represent the state of patients and predicting the prognosis of cancer patients. The number of gene data is enormous compared with the number of patients, making it challenging to analyze it. To solve these problems, significant genes that represent the state of patients must be extracted. In addition, developing a classification model from the extracted genes may be helpful for early diagnosis and prediction of the prognosis of cancer patients. Cancer is caused by gene variation, damaging genes regulating cell replication in a predetermined order; thus, cells multiply unlimitedly. Therefore, the cells invade adjacent normal tissues and are transferred to the whole body. Because cancer stem cells from mutated genes, they are thought to be a genetic disorder, although only a small number of cancers are genetic. In the case of a mutation in a reproductive cell, the mutation is transferred over generations, and it exists in the whole somatic cell ⁽¹⁶⁾. To predict the state of patients, researchers applied deep learning techniques for analyzing the mutation in a sequence and, studies have accurately predicted major mutations that cause diseases such as spinal muscular atrophy, hereditary non-species colon cancer, and autism ⁽¹⁷⁾.

Kidney cancer is a primary tumor stemming from kidney and renal cell carcinoma; it is a malignant tumor that accounts for over 90% of cases ⁽²⁾, ⁽¹⁸⁾. Because kidney cancer has no symptoms in the early stages, there is often a progressive step at the time of discovery. According to the national cancer registration statistics published in 2020, among the 243,837 cases of cancer in 2018, 5,456 were attributed to kidney cancer, accounting for 2.2% of all cancer cases. By gender, kidney cancer ranked eighth with 3.0% (3,806 cases) of all male cancers ⁽¹⁹⁾. In addition, the symptoms and treatment of kidney cancer decrease patients’ quality of life by increasing the disease burden and medical costs. Risk factors for kidney cancer include environmental habits, living factors, genetic factors, and existing kidney diseases. Among them, smoking, obesity, high blood pressure, and eating habits can be the causes associated with living factors ⁽²⁰⁾. Recently, researchers conducted research to extract features using genetic data from kidney cancer patients and apply classification algorithms through neighborhood component analysis methods ⁽²¹⁾. Furthermore, we used big data from a large cohort (KOTCC database) of kidney cancer patients collected from eight domestic medical institutions to extract variables affecting kidney cancer recurrence. We applied a machine learning algorithm to predict recurrence within five years of surgery ⁽²²⁾.

In this study, we propose a method to extract genes that affect prognosis prediction in kidney cancer patients using a deep learning algorithm and apply a classification algorithm based on the extracted genes to predict the prognosis of cancer patients. We combined gene expression data and clinical data from kidney cancer patients obtained from TCGA portal sites to extract genes that contribute to patient prognosis and applied classification techniques to present their utilization ⁽²³⁾. Next, we selected gender (male, female), sample type (primary cancer, normal), and race (white, black, Asian) as the target variables for analysis. Notably, we extracted genes from kidney cancer patients based on gender, sample type, and race to overcome heterogeneity and extract genetic biomarkers that could allow a more accurate prognosis prediction. After testing the functionality of genes, we presented their applicability and developed the optimal prediction model by comparing and analyzing classification algorithms using extracted genes.

2. Related works

Machine learning and deep learning algorithms are being applied to various analyses of biological data. Some studies predicted the risk of 20 cancers by applying machine learning techniques and artificial intelligence methods to genetic big data analysis ⁽²⁴⁾. The Bayesian classifier has been applied to the problem of classifying proteins that have sequence and structural information, and studies have also used the Bayesian network to combine various details related to proteins and genes with improving the predictive performance for gene function ⁽²⁵⁾. As such, different machine learning technologies are being applied for the analysis of biological data. In a study utilizing the TCGA-KIRC database, they used CT and MRI scan data and clinical data of 227 kidney cancer patients were used to predict the classification accuracy of the cancer stage by applying deep learning ⁽²⁶⁾. In another study, significant gene extraction from kidney cancer data in the TCGA was performed using a deep autoencoder compared to the traditional methods such as least absolute shrinkage and selection (LASSO). The predictive accuracies of classification were compared with the conventional state-of-the-art classification methods and analyzed ⁽²⁷⁾. Researchers integrated data of various cancer patient types, conducted the analysis using AE structures, presented their availability in clinical applications, and suggested ways to efficiently perform posterior inference via stochastic variational inference and learning algorithms in the presence of posterior probability distributions and continuous latent variables for extensive data ⁽²⁸⁾. An AE is a neural network in which the output is set to input x to extract features. By learning how to reconstruct an input, the AE extracts basic or abstract properties that facilitate the accurate prediction of the information. In principle, a linear AE with a single hidden layer in a multi-layer perceptron is the same as principal component analysis (PCA) ⁽²⁹⁾, ⁽³⁰⁾. More generally, nonlinear autoencoders have been studied to extract key properties, including high-level features and Gabor-filter features ⁽³¹⁾, ⁽³²⁾.

The variational autoencoder (VAE) is a model that, given training data as a generative model, produces new data with a sampled value in the same distribution as the actual distribution of the training data. An AE is a model that compresses high-dimensional input data into smaller representations in the stochastic form. Unlike the conventional AE, which maps inputs to latent vectors, VAE maps input data to parameters in the identical probability distributions as the mean and variance of Gaussian distributions. This method produces structured latent spaces and is therefore helpful for image generation ^(33-35).

Supervised autoencoder (SAE) is a neural network that jointly predicts the targets and inputs (reconstruction). For a single hidden layer, this simply means that a classification loss is added to the output layer. The innermost layer has a classification loss added to the layer for a deeper AE, which is usually handed off to the supervised learner after training the AE. The SAE uses unsupervised auxiliary tasks to improve the generalization performance ^(36-38).

The CVAE is a modification of existing VAE structures that enable supervised learning, which considers category information when learning data distributions in the form of added class label y to encoders and decoders. CVAE is a deep conditional generative model for structured output prediction using Gaussian latent variables. The model has efficiently trained in the stochastic gradient variational Bayes framework and allows fast prediction using stochastic feed-forward inference ^(39-41).

We validated the performance of the proposed framework by comparing it with the traditional data mining and classification methods. The proposed framework employs the various AE-based deep learning techniques by taking advantage of pre-training and fine-tuning strategies. The experimental results show that the AE-based deep learning methods show better performances than the combinations of traditional data mining and classification methods.

3. Methodologies

3.1 Architecture

The main challenge faced by general analysis is the characteristic of genetic data because they have more gene expression values than the number of samples. We propose a novel deep learning–based framework by combining the various AE-based techniques for cancer analysis and compared with the existing feature extraction methods—PCA and NMF—and demonstrate its superiority. The following section describes a pre-training method for auto-encoder-based feature extraction. Proper training of neural networks requires a large amount of learning data; however, often, we have a small quantity of labeled learning data and large amounts of unlabeled learning data. In this case, unlabeled learning data are used to pre-train each layer of a neural network called unsupervised pre-training. AE and VAE only have reconstruction loss in pre-training, but SAE and CVAE also include classification loss. Once the parameters for each layer have been determined to some extent, the classification performance can be improved through fine-tuning using labeled learning data.

In particular, feature extraction was first performed to compare traditional classification algorithms with deep learning techniques. We used conventional dimension reduction techniques such as principal component analysis (PCA) and non-negative matrix factorization (NMF) followed by state-of-the-art classification algorithms. And we used deep learning techniques such as autoencoder (AE), variational autoencoder (VAE), conditional autoencoder (CAE), and conditional variational autoencoder (CVAE), followed by a neural network classifier. For significant gene selection using traditional classification algorithms, we used PCA and NMF, solved the data imbalance problem, applied various classification techniques, and compared and analyzed the results. For deep learning–based significant gene selection, we used all the improved algorithms based on the AE algorithm. We compared the extracted genes, and the classification accuracy was analyzed using a multi-layer perceptron (MLP).

3.1.1 Autoencoder

The AE is a deep learning structure for efficiently coding data. Coding refers to compressing data; in other words, dimensionality reduction is the transformation of data from a high-dimensional space into a low-dimensional space to efficiently represent some data. The neural network architecture of AE has the same input and output and can be represented by Fig. 1 as a symmetrically constructed structure. Because dimensionality reduction is the goal in our study, we take some data X and obtain the node value Z of the hidden layer as a combination of the weighted multiplication and sum and the activation function, which we call the encoder.

그림. 1. 오토인코더의 아키텍쳐

Fig. 1. Architecture of autoencoder

AE has the same structure as MLP, except that the input and output layers have the same number of neurons. Because AE reconstructs the input, the output is called reconstruction, and the loss function is calculated using the difference between the input and reconstruction. When learning AE, it follows unsupervised learning, and the loss is used as the maximum likelihood (ML). Once the hidden layer Z parameters have been determined to some extent, we can use the labeled learning data can be used to perform supervised fine-tuning.

3.1.2 Variational autoencoder

A VAE combines the input data X with the mean (μ) and variance (σ2) (two vector outputs) through the encoder to create a normal distribution. That allows the sampling to create latent vector Z to pass through the decoder to produce new data similar to any existing input data. Therefore, the VAE is a generative model developed generate new data using probability distributions. The structure of the VAE is shown in Fig. 2.

We used the ideal sampling function posterior (to sample, allowing generators to learn the input data well. Equation (1) is used to make the value generated by the sampling function equal to the input value. The maximum likelihood estimation that maximizes the value of (1) shows how well the reconstruction restores data like the input data when the Z vector (latent vector) extracted from the ideal sampling function is given.

그림. 2. 변형 오토인코더의 아키텍쳐

Fig. 2. Architecture of a variational autoencoder

(1)

$Eq_{\Phi}(z|x)[\log(p(x|z))]$

Finding an optimal formula that satisfies these conditions results in evidence lower bound formula that fulfills the above conditions when X is given to the network as evidence.

(2)

$Eq_{\Phi}(z|x)[\log(p(x|z))]-KL(q_{\Phi}(z|x)||p(z))+KL(q_{\Phi}(z|x)||p(z|x))$

The first term of (2) is the reconstruction term, indicating how well it is restored from the ideal sampling function. The second term is a regularization term, which makes the perfect sampling function the same as the prior as possible. The conditions are given to sample values like priors among several samples. The third term represents the distance between the two probability distributions, the distance between the ideal sampling function $(q_{\Phi}(z |x))$ and the sample function $p(z |x)$.

3.1.3 Supervised autoencoder

An SAE is an AE with the addition of a classification loss to the representation layer. For a single hidden layer, this means that a classification loss is added to the output layer. For a deeper AE, the innermost layer would have a classification loss added to the layer, usually handed off to the supervised learner after training the AE, which is explained in Fig. 3.

그림. 3. 감독 오토인코더의 아키텍쳐

Fig. 3. Architecture of the supervised autoencoder

3.1.4 Conditional variational autoencoder

The CVAE is a modification of existing VAE structures which enables supervised learning. CVAE adds a class label y to the encoder and decoder, considering the category information when learning data distributions. Thus, in CVAE, a particular condition is given and added to the encoder and decoder if the label information is known. The y-value is given along with x to find the latent vector z in the encoder. Similarly, the decoder can represent the y-value that generates data as follows. Therefore, the loss function is represented by the reconstruction loss and classification loss. The form is shown in Fig. 4.

그림. 4. 조건부 변형 오토인코더의 아키텍쳐

Fig. 4. Architecture of the conditional variational autoencoderㄴ

4. Experiments

4.2 Overall analytical structure

We leveraged the integrated data obtained from TCGA to conduct classification analysis on gene expression data using traditional classification techniques and deep learning–based MLP. Fig. 5 shows the overall framework used by traditional classification algorithms. We calculate the interquartile range for outlier detection, a widely used technique that helps find outliers in continually distributed data. We used IQR in our preprocessing because it is a reasonably robust measure of variability. Besides, it is not affected by outliers since it uses the middle 50% of the distribution for calculation and is computationally cheap. First, we eliminated noise and outliers from the genetic data of kidney cancer and extracted 5,000 genes via chi-square tests. We performed 5-fold cross-validation (train (80%) and test (20%)) on 5,000 data samples and used PCA and NMF as data transformation methods. Subsequently, we utilized SMOTE algorithms to solve the data imbalance problem for gender, race, and sample type variables. Furthermore, the classification accuracy of the said variables (race, gender, and sample type) was compared and analyzed by applying classification algorithms such as KNN, SVM, DT, RF, AB, NB, and MLP.

그림. 5. 신장암 유전자 발현 데이터에 대한 전통적인 분류

Fig. 5. Traditional classification for gene expression of kidney cancer

The classification accuracies of the deep learning techniques for race, gender, and sample type based on the AE are shown in Fig. 6. In AE-based techniques, we eliminated noise and outliers, and 5,000 genes were extracted via chi-square tests. We performed 5-fold cross-validation (80% for training and 20% for testing) on the selected 5,000 features and corresponding data samples followed by AE, VAE, SAE, and CVAE during the pre-training and training phase. Finally, we extracted the 100 latent variables. We also solved the imbalanced data problem by fine-tuning the generative pre-trained encoder for highly imbalanced data. The encoder and MLP were combined as classifiers to predict the classification accuracy for race, gender, and sample type.

Compared to Fig. 5, the experiments consist in assessing two different approaches when training the classification model, allowing fine-tuning of the entire network and embedding the AEs into the classification network, namely by only importing the encoding layers. The unsupervised pre-training on the gene expression data and fine-tuning it on specific tasks affect the classification performance. The experimental results show that autoencoder-based approaches achieved a higher classification performance than the traditional classification approaches as reported in the next sections.

그림. 6. 신장암 유전자 발현 데이터에 대한 오토인코더 기반 분류

Fig. 6. Autoencoder-based classification for gene expression data of kidney cancer

5. Results

This section extensively evaluates our approach and compares it with other unsupervised feature extraction techniques, followed by over-sampling and state-of-the-art classifiers. We also report on an ablation study we conducted to explore the most significant 20 genes for each clinical information.

Tables 2-4 show the performance comparison among all methods according to gender, race, and sample type, respectively. The classification performance of values using the micro-average is superior to that of evaluation metrics using the macro-average. The AE-based methods achieved higher performance than the conventional feature extraction methods. That means that AE-based methods can better extract the complexity of cancer and produce more meaningful features. We used only an MLP classifier for the features extracted by AE-based methods because of its neural network structure, and we did not use any sampling for it. When the data imbalance problem was solved using traditional algorithms, the generative AE-based methods achieved higher performance than when sampling was performed using SMOTE algorithms.

Table 2 presents the classification results for gender. The results show that VAE achieved a macro-F1-score of 0.958, and a micro-F1-score of 0.962, indicating higher classification performance than the other methods. It offers results comparable with other AE-based methods and improves the highest performance results of conventional PCA+SVM with SMOTE over-sampling, by 0.021 macro- and 0.02 micro-F1-score, respectively. A gender disparity exists in the incidence of kidney carcinomas, with more incidence reported in men ⁽⁴⁴⁾. Men are at a higher risk of developing kidney cancer and usually have a more aggressive disease at the time of diagnosis. Females generally show more favorable histological kidney cancer and have better oncological outcomes than males ⁽⁴⁵⁾. Extracting valuable features by VAE or other AE-based methods gives deeper information about gender-related differences in kidney cancer therapy.

Table 3 shows that when the target variable is race, and the label is white, black or African-American, and Asian, the class label imbalance is very severe. Our data included 940 (81.2%), 150 (13.0%), and 17 (1.5%) white, African-American, and Asian samples, respectively. Clearly, race prediction is a more challenging task than other clinical prediction tasks. It shows a much lower macro-averaged performance. The results show that CVAE achieved a macro-F1-score of 0.763, and a micro-F1-score of 0.959, indicating a higher classification performance than the other methods. It offers a micro-F1-score that is comparable with other AE-based methods and improves the highest performance results of conventional PCA+SVM with SMOTE over-sampling by 0.121 macro- and 0.018 micro-F1-score, respectively. It also enhances the highest performance results for the AE method by 0.076. We can conclude that CVAE can extract the complexity of cancer and works well for more complex tasks than other AE-based methods. Surveillance and epidemiology data indicate that kidney cancer incidence and mortality rates are higher among African-American patients compared to white patients ⁽⁴⁶⁾.

White and Asian patients (age 63.9 and 62.6 years, respectively) had a slightly older age of onset than Black and Native American patients (age 60.7 and 60.3 years) ⁽⁴⁷⁾. However, feature extraction for racial information is challenging; we can achieve a higher macro-F1-score (higher than 70%) using the CVAE method.

Table 4 presents the breakdown results for the sample types. The label for the sample type was 1,010 primary tumors (87.3%) and 139 solid tissue normal (12%), and the remaining had missing values. The results show that all AE-based methods achieved comparable results, with macro-F1-score of 0.996 and micro F1-score of 0.998, and a higher classification performance than the other methods. All AE-based methods improve the

highest performance results of conventional PCA+KNN, PCA+MLP without any over-sampling, and PCA+MLP with SMOTE over-sampling, by 0.002 macro- and 0.001 micro-F1-score, respectively. Survival in patients with kidney cancer can be correlated with the expression of various genes based solely on the expression profile in the primary kidney tumor ⁽⁴⁸⁾. Compared to other tasks, distinguishing extracted features is more straightforward, and predicting sample type is much easier. For sample types, classifiers based on both traditional techniques and deep learning performed well. Using the AE-based pre-training algorithm is slightly better, and overall, than the other compared methods. The are several methods to predict cancer subtypes or sample types using deep learning techniques on gene expression data ^(49-51). To the best of our knowledge, methods identifying kidney cancer biomarkers by combining AE-based methods and model interpretation techniques are still lacking.

In general, unsupervised learning algorithms applied to gene expression data extract biological and technical signals present in input samples. It is best to compress gene expression data using several algorithms and many different latent space dimensionalities. These compressed gene expression features represent important biological signals, including gender, race, and presence of tumor. We showed, through several experiments tracking lower dimensional gene expression representations, and supervised learning performance, that optimal biological features are learned using a variety of latent space dimensionalities and different compression algorithms.

6. Conclusions

We combined kidney cancer clinical data and gene data collected through the TCGA database to extract significant gender, race, and sample type genes. We conducted a classification analysis based on these data. Based on deep learning algorithms, we compared and analyzed datasets using traditional classification techniques and pre-training processes, such as AE, VAE, SAE, and CVAE. For feature extraction of significant genes for the classification analysis using traditional techniques, PCA and NMF techniques were employed, while in our proposed deep learning–based techniques, important genes were extracted through pre-training processes such as AE, VAE, SAE, CVAE, and fine-tuning. As a result, deep learning–based effective gene extraction methods performed better.

There are several methods to predict cancer subtypes or sample types using deep learning techniques on gene expression data. To the best of our knowledge, there is a lack methods for identifying kidney cancer biomarkers that combine AE-based methods and model interpretation techniques. As shown in Tables 2-4, extracting race-related features is the most challenging task, and sample type feature extraction is much easier than other tasks. For the challenging tasks, CVAE outperforms the other methods.

Furthermore, we compared micro and macro measures according to the number of class labels of the target variables. The micro-measure exhibited better performance. In the future, the extracted genes will be able to confirm the gene’s function through verification and help predict the prognosis of kidney cancer patients. In further work, we will consider the other data samples such as clinical, RNA, DNA methylation, etc.