The thesis maintaining the doctor degree of engineering science on speciality 05.13.23 – “Systems and means of artificial intelligence.” – National technical university of Ukraine "Kyiv Polytechnic Institute named after Igor Sikorsky," Kyiv, 2019.
The necessity of the development of integrated, hybrid systems based on deep learning is substantiated. Such systems consist of various elements (components), united in the interests of achieving objectives set.
In the thesis the actual scientific-applied problem, which has the important scientific and practical importance, is solved and it consists in the development of methods and algorithms for solving the problem of structural-parametric synthesis of deep learning hybrid neural networks (HNN).
It is shown that the main problems of synthesis of HNN, at present, are:
– absence of formal methods for choosing the type of neural networks (NN), adequate for the class of tasks to be solved;
– insufficient work on the issues of automatic formation of the topology of NN, which does not allow to create NN of high accuracy and minimum complexity (minimum computational costs);
– insufficient grounds for choosing optimization methods in the training procedure of NN, which leads to significant errors.
In the course of the thesis work the methodology of structural-parametric synthesis of HNN; method of structural-parametric synthesis of modules of HNN; algorithm of structural-parametric synthesis of ensemble of modules of HNN; algorithm of structural-parametric synthesis of HNN of deep learning; methods of prediction based on the use of HNN of deep learning is developed.
A new methodology for the synthesis of HNN is developed, which is differed by the fact that in the first stage the optimal base neural network is selected; in the second stage, as a result of solving the multicriteria optimization problem, it is modified; in the third stage, the problem of structural-parametric synthesis of modules is considered at the fourth stage the problem of structural and parametric synthesis of the ensemble is solved, which allows to improve the accuracy of the systems operation in their minimal complexity. The problem of optimal choice of basic neural network (BNN) topology is solved by using the method of selection. Numerous examples of optimal choice of BNN are given in the work.
Based on the analysis carried out, in this work, it is proposed to synthesize the hybrid topology in the form of a parallel ensemble of NN modules with a layer of association. As a procedure for building an ensemble, the use of begging, which has advantages over others, is substantiated. To optimize the size of the ensemble, an algorithm for simplification was developed with the help of the complementary value method, which also takes into account the interaction of the classifiers with each other. The weigh coefficients of the association of modules in the ensemble were determined on the basis of the use of the method of dynamic averaging.
A new hybrid algorithm of deep learning neural network topology formation has been developed, which in opposite to the known ones the parameters of the main network are determined by the sequential execution of each search iteration sequentially with each of the basic algorithms (swarm particles and genetic), the comparison of the found results and the use of the best found solutions of each algorithm, that allows to increase the accuracy and speed of network work under minimal complexity.
The problems that arise when solving prediction problems of time series with a large number of input variables are determined. The hybrid method of solving forecasting problems is developed, which is distinguished by the fact that it implements deep learning based on the use of a single-layer network with neurons of the type sigm_piecewise, constructed using the group method of data handling method, with the subsequent learning of the entire network as a whole by the method of reverse error propagation in order to find the global extremum, which increases the accuracy of prediction.
The regularization prediction method is developed which in opposite to the known ones, it can be used in the case of heterogeneity of data and is based on the use of a soft clustering algorithm, in which as a surface model, separating clusters, it is used single layer NN with sigm_piecewise neurons and local NN, one for each cluster whose training are carried out only on examples from one cluster, that increases the accuracy of prediction.
Keywords: hybrid neural networks, artificial intelligence systems, image processing, systems of medical and technical diagnostics, automated traffic control systems, information fire systems.
