The thesis is devoted to development of the system that makes it possible to determine the cancer tissue types on optical images. The learning method of the new combined neural network was developed. The idea of the method is to split the process of optimal values computing into two stages. At the first stage, the nonlinear parameters of RB functions are optimized. At the second stage, the weights of the neural network are optimized. This helps to reduce the optimization problem complexity and to speed up the work of the learning algorithm. To study the effectiveness of the developed method experiments and comparative analysis of different neural network classes have been performed. Such neural networks were used in this work as Back Propagation neural network, Cascade Neural network, NEFCLASS neural network, RBF neural network, and improved cascade RBF neural network. The use of the combined algorithm helped to reduce the error of tissue type classification by 18% in comparison with other neural networks that were used in this research. It was determined that by using the proposed method, the epithelium state was correctly identified in 87.5% cases, while type 1 and type 2 errors were 11.25% and 1.25% correspondingly, which was acceptable for the practical usage of this method for the given problem. The system is based on a hypothesis, that alike optical properties have alike types of tissues. The difference between different tissues is more significant than the tissue difference for different patients. On the basis of the above mentioned hypothesis, the system was developed which determines the tissue type from the image. Functioning of the system consists of the following stages. First, it is necessary to form initial data. For this purpose, in a place in which a biopsy was made, an area is selected measuring 20x20 pixels, thus, tissue texture is taken into account from a place, where a biopsy was made. It results in 6400 input variables for each case. Due to a large number of variables their number is reduced to 14 using principal component analysis (PCA). Transformed data are fed into inputs of neural networks. Then, the segmentation of the images is implemented using SOM Kohonen. After the segmentation, an area is selected measuring 20x20 pixels in each segment and the data are transformed using PCA. The transformed data are fed to the inputs of the previously trained neural networks and, for each segment, the percentage of tissue content is computed. The developed methods of images classification can be used in medical diagnostics as an auxiliary tool, and the experimental studies of the different methods effectiveness can help to choose the best method for the use. The developed computer intelligent system is used for the medical diagnostics in the Institute of Pediatrics, Obstetrics and Gynecology, in the Kyiv regional hospital №2 and in a medical center "Syrets", which is confirmed by the corresponding documents. The practical use of the system appeared to be effective during preventive examinations. The developed system is also used in the educational process of the mathematical methods of the system analysis department in the educational-scientific complex "Institute for the applied system analysis" of National Technical University of Ukraine "Kyiv Polytechnic Institute" in the course "Theory of information and pattern recognition".
