Advances on Tumor Image Segmentation Based on Artificial Neural Network ()
1. Introduction
Image technology is applied more and more in disease diagnosis which mainly includes X-ray, Computed Tomography (CT), Ultrasound, Magnetic Resonance Imaging (MRI) et al. Image technology can help doctors to observe directly and identify normal and pathological tissues intuitively, and to provide basis for the clinic diagnosis and making of operation plan. To make correct diagnosis, the clinician should deal with a lot of images using computers. Image segmentation is one of the most important technologies.
Image segmentation is to divide an image into some subareas first and then identify/extract the concerned ones. Conventional image segmentation methods include edge detection method, threshold method, regional growth method et al. The edge detection method distinguishes the edge according to the change rate of the gray value. The image is divided by the detected edges. It is easy to divide the area excessively by this method. Threshold method divides the image by the given threshold. It is easy to operate. But the threshold is difficult to determine. Regional growth method chooses a seed point first and then merges the surrounding points with similar characteristics to form a subarea. These conventional methods require many human-computer interactions and are easy to be disturbed by many factors such as noises.
Up to now, image segmentation technology is still not mature. Studies are looking for new methods. Recently, artificial neural network methods such as BP network, Convolutional Neural Network, are applied in the image segmentation with the development of artificial intelligence [1].
On the viewpoints above, the progress on segmentation of tumor image by using artificial network is summarized. The application of CNN in segmentation of tumor image is mainly analyzed. The trend of tumor image segmentation by using artificial network is discussed.
2. BP Network and Application in Image Segmentation
The BP (back propagation) network model is a well-known multi-layer training model. This model is effectively applied in many fields such as image treatment, classification etc. A BP network model consists of input layer, middle layers and output layer. The dependent variables are input in the input layer. The data outcome from the output layer are the independent variables. The layer locates between the input and output layers are called the middle/hidden layers. The factors of the model are built by learning from the history data. The relations between dependent variables and independent variables may be determined at one time and the results are influenced few even if some dependent or independent data are lost.
The BP neural network is characterized by gaining possession of hidden layers. A BP neural network without a hidden layer is adequate to describe the system of the linearized relationship between inputs and outputs. Generally, a BP network with one hidden layer is enough for most applications. Therefore, BP network with three layers: input layer, middle layer and output layer is often used in practice.
During segmentation of image by using BP network, all pixels in the image are divided into target and non-target pixels first. After that, the non-target pixels are removed and the retained pixels form the target image. In other words, this process includes two main steps: 1) to collect well-trained or having been artificial divided sample set of images to build BP network. The network is trained by the sample set to determine the best weight values and connection conditions. 2) To determine the best threshold for segmentation of the concerned images by using the trained network and then finish the segmentation of image.
BP network has some problems during applied in image segmentation such as slow learning speed, falling easily into local optimum though there are many advantages such as good operability and flexible algorithm. Therefore, BP network is often combined with other methods, such as particle swarm optimization, wavelet analysis etc., to improve the accuracy of image segmentation. Meanwhile, other types of artificial network are also been presented such as convolutional neural network (CNN), recursive neural network, more network models are used in the segmentation of images [2].
3. CNN Model and Application in Image Segmentation
CNN is a kind of network with convolutional structure to reduce the usage of memory and cost of computation. It is developed based on the BP network. CNN has the characteristics of local sensing, multiple structure, weight sharing and pooling layer. It is more stable than BP network when applied in medical image segmentation. A CNN consists of input layer, convolutional layer and pooling layer, output layer. Output layer is the last layer and is a classifier which can be realized by using logistic regression, Softmax regression etc. The middle layers consisted of alternate convolutional and pooling layers and are the core module of CNN for extracting of characteristics [3].
Local receptive field means each neuron needs to percept local characteristics. The information of all local neurons is synthesized at the high layer to obtain the global information. In this way, the local characteristics are retained as well as the number of connections decreases. Weight sharing means the image is operated by the same convolutional core. In other words, all neurons of the first hidden layer can detect the same characteristics of different positions in an image, which leads the detected results to have perfect translation invariance. Pooling can make the translation, deformation and scale transformation have stronger robustness and fewer dimensions.
CNN has many advantages relative to general neural network, such as 1) CNN can better adapt the structure of image; 2) Extract and classification operation can be done in the same time, which makes the characteristics extract be benefit for the classifying; 3) Weight sharing can reduce the training parameters of the network, which makes the structure of network simple and the adaption strong.
Because the image segmentation means to treat a large amount of data, CNN with high efficiency is needed in practice. Supervised learning CNN, the early used in segmentation of image, is replaced gradually by semi-supervised learning CNN and unsupervised CNN.
3.1. Supervised Learning CNN
Conventional CNN operates in a type of sliding window. Only one small part of the image is used for computation. In other words, a convolutional core is used to slide the whole image. The efficiency of this method is very low because of the small sliding distance and weakening of the overall features.
A supervised CNN needs a large amount of artificial labeled data for training. So some improved networks are built such as U-Net network which was presented by Ronneberger et al. [4]. It is the first fully convolutional neural network used in medical image segmentation. This network can extract the spatial information of the whole image one time. Meantime, it can avoid the large area overlapping of the adjacent data that exist in the sliding-window type CNN. These advantages make this type of network can improve the correction of segmentation and needs not large amount of data.
3.2. Unsupervised Learning CNN
To save the time cost on the artificial labeling and segmentation of images, researchers try to build an unsupervised learning network, which can increases the efficiency greatly and avoid the human interference. For example, Moriya et al. [5] built a CNN with 3 convolutional layers and 1 pooling layer and 2 fully connected layers to use in the segmentation of 3D lung tumor images. It is shown that this method is better than the threshold method. Bao et al. [6] presented a kind of CNN with multi-specification structures to improve the segmentation accuracy of brain tumor. Each layer of this model has multiple convolutional cores with different scale. This model can make the segmentation smoother as well as improve the accuracy and efficiency.
3.3. Semi-Supervised Learning CNN
To reduce the learning time, some researchers presented a kind of method that combines the unsupervised pre-training and the supervised training with fine adjustment. This type of method is called semi-supervised learning. In the stage of pre-training, the parameters of the network are assigned initial values. Then the supervised training is processed. In this way, the updating of parameters is small or only the last layers are finely adjusted, such as self-encoder is often used in the pre-training to improve the efficiency. Kallenberg et al. [7], Sun et al. [8] presented a network (Convolutional Sparse Auto-Encoder) in the segmentation of breast image. The studies show that the semi-supervised learning network can carry out segmentation with high accuracy in the condition of only a few samples.
3.4. Segmentation Process by Neural Network
The process of image segmentation by CNN generally includes the following steps: pretreatment, building and marking of the data set for training, detecting and segmentation of tumor image. Each step is introduced simply in the following [9].
1) pretreatment
At present, most of the deep learning algorithm is evaluated based on the public data set, such as data set of brain tumor segmentation (BRATS). However, the public data set has only limited data and so is difficult to train a network with strong generalization ability. Therefore, data enhancement (rotation, translation and deformation etc.) is adopted to increase data. To overcome the inhomogeneity of the tumor image and noise, the gray scale of the image should be normalized. The noise should be removed.
2) building and marking of the data set for training
The medical images are often not enough. Generally, the data are divided into many parts. One is for testing and the others are for training. The data for training should be marked. Manual marking of image takes a lot of human resources, thus some methods for marking by computer are presented, such as CNN is used for marking of tumor image.
3) Building and optimization of the network
An optimized neural network can save many human resources and cost of computing. Some optimization measures are often adopted in the building of network for segmentation of image. Shallow network needs to choose suitable middle layer number and neurons in each layer. Deep network needs to determine the structure and the data set for training first. The number of hidden layers and pooling layer and convolutional core are also needed to optimize.
4) Detecting and segmentation of tumor image
After finishing the above three steps, the segmentation of image can be processed. It is the key step of tumor image treatment by using neural network. The aim of this step is to look for the form and scale of the focus. CNN can solve the problems such as differencing the noise and feature of the same type of focuses by establishing large sample set, even if the focuses are different persons. R-CNN is the one used more in medical image treatment which extracts the suspected focus by using selecting-search-method, then extracts the features of the focus. When the real focus is found, the image segmentation is used for doctors to improve the accuracy and speed of diagnosis.
It is very important to carry out early detection of mammary tumor by using medical image technology in improving the survival rate. Convolutional neural network combined with other methods such as CNN-Structured Support Vector Machine, DPN (Deep polynomial network)-MKL (Multiple Kernel Learning) can enhance the correction and efficiency of image segmentation.
4. Application of Artificial Neural Network in Tumor Segmentation
4.1. Brain Tumor Segmentation
The brain tumor often occurs in the white matter area. The image appears to be nodular, lumpy or infiltrative. There is no obvious envelope and obvious boundary. In most cases, the edema cannot be observed.
The quantitative analysis of brain tumor includes the measurement of the biomarkers in an image such as the max diameter, volume to quantify the disease. The detection of the biomarker depends on the accurate segmentation of the tumor. For segmentation of brain tumor, researchers have presented different frameworks and methods based on CNN [10]. Havaei et al. [11] presented a framework to divide the image of brain tumor fully automatically. Pereira et al. [12] presented a smaller convolutional core and deeper network structure relative to Havaei’s framework. Nie et al. [13] presented a fully convolutional neural network which can train the network by end to end to reduce the learning time of the network. Cao et al. [14] presented a 3D CNN with multi-pooling layers to solve the problem that the extract of differences between different modes with 2D CNN. The accuracy of image segmentation by modified CNN is improved obviously (Figure 1) though it is not quite the same as the manual one.
4.2. Segmentation of Mammary Tumor
Mammary tumor is one of the most common tumors of women. It is very important to detect the mammary tumor by using image technology to improve the survival rate.
The characteristics of the mammary tumor and image are different for different kinds of tumors. For example, some have fuzzy edges, some have obvious burrs. The density of mammary tumor at two sides is not symmetrical.
Dhungel et al. [15] divided the image of mammary tumor by combining the structured support vector machine and CNN. It is shown that this method can improve the correction and effectivity obviously of the segmentation. Liu et al. [16] found that the combination of deep polynomial network and multiple kernel learning is better than common CNN in mammary tumor classification based on ultrasonic image.
CNN is better than the common segmentation methods in segmentation of pulmonary nodule, brain tumor and mammary tumor. CNN can also decrease the evaluation errors due to manual intervention and has high accuracy in tumor image segmentation [17] [18].
![]()
(a) ![]()
(b)
Figure 1. The comparison of segmentation by CNN and human (After [14]). (a) Segmentation by human; (b) Segmentation by CNN.
CNN has strong power in spatial recognition. It can extract higher order spatial features and decreases the number of parameters greatly by sharing the convolutional core. Part of the network adopts the method of unsupervised learning to realize the semantic segmentation of an image to some extent, which can largely reduce the burden of manual image reading.
5. Conclusions
The advances of image segmentation by using artificial neural network are summarized. The application of BP network and CNN in tumor image segmentation is mainly discussed.
Many kinds of artificial neural networks have been used in the segmentation of medical image. The application of different artificial neural networks shows their wide prospect in the segmentation of tumor images.
CNN is a kind of very promising method in the segmentation of tumor image. Various CNN models have been developed and used in the segmentation of tumor image, such as supervised learning CNN, unsupervised learning CNN and semi-supervised learning CNN. CNN is also often combined with other methods to analyze the tumor image.
However, some deficiencies of image segmentation by using artificial neural network should be overcome. Massive operation of CNN needs hardware with high performance. Otherwise, the image resolution has to decrease, which will reduce the correction of segmentation. The accuracy of image segmentation by using neural network is smaller than that by using artificial operation at present. More practical methods are required to be built.
The training of network needs a lot of samples with segmentation markers. To build the marked data set will cost large human and financial resources. It is necessary to look for new methods which can automatically generate markers in the building of image segmentation data set.