Popov S. Artificial neural networks with specialized architectures in time series processing tasks

Українська версія

Thesis for the degree of Doctor of Science (DSc)

State registration number

0510U000045

Applicant for

Specialization

  • 05.13.23 - Системи та засоби штучного інтелекту

28-12-2009

Specialized Academic Board

Д 64.052.01

Kharkiv National University Of Radio Electronics

Essay

Research object - processes of different physical nature, described by time series. Research target is the synthesis of specialized architectures of artificial neural networks to solve the problems of time series processing, taking into account a priori information about the properties of the modeled signals and the influencing factors, and allow extracting new information on these properties from the data. Methods of research - the theory of artificial neural networks (ANN), which allowed the synthesis of new types of neurons and ANN architectures, theories of optimization and linear algebra lead to better methods of ANN training, theory of fuzzy logic that allowed taking into account the influence of factors given in ordinal and nominal scales of measurement, mathematical analysis, which allowed to identify the properties of the analyzed functions, simulation confirmed the validity of the theoretical results, mathematical statistics, which allowed to analyze the results of experiments. Theoretical and practical results - a general approach to the problem of taking into account a priori information about the properties of the processed time series and influencing factors in the synthesis of specialized architectures of neuro-fuzzy network is developed based on fuzzification of input variables using membership functions of different types, which allowed to process the data given in different scales of measurement (quantitative, ordinal, nominal) and include information on the form of influence of different factors; top-down approach to solving the problem of nonlinear decomposition of real functions is developed that do not require a priori information about the properties of the function that allows identifying new information about the properties of the modeled signals and the influencing factors; methods of combining of neural networks ensembles and neural networks learning are improved, which take into account the particular problems of time series processing. Scientific novelty - a specialized architecture of feedforward neuro-fuzzy network and specialized architectures of recurrent neuro-fuzzy networks, which are modifications of recurrent Elman and Madhavan networks and recurrent echo-state network, are proposed for the first time, characterized by the presence of the first hidden layer of delay and fuzzification blocks with membership functions of different types, which makes it possible to take into account a priori information about the properties of the processed time series and the influencing factors; architecture of the neuro-fuzzy unit is proposed for the first time, which differs from the well-known neo-fuzzy neuron by the presence of an additional nonlinearity in the form of "squashing" activation function, thereby limiting the range of variation of its output signal; specialized neuro-fuzzy network based neuro-fuzzy units and dynamic finite impulse response neurons is proposed for the first time, which differs by not-fully-connected hybrid architecture, which increased the flexibility of the specialized neuro-fuzzy network and reduce the number of adjustable parameters; a hybrid neuron-like unit architecture, distinguished by its ability to use linear and nonlinear synapses and synapse-filters with finite and infinite impulse responses is proposed for the first time, which makes it possible to build specialized architectures of the neuro-fuzzy networks with desired properties; a specialized neuro-fuzzy network architecture based on neuron-like units is proposed for the first time that differs from well-known architectures by the independent choice of the type and parameters of individual synapses in accordance with the available a priori information about the properties of input variables and the predicted time series, which reduced the number of adjustable parameters and improved the generalizing properties of the network; specialized architecture of the neural network for nonlinear decomposition of real functions is proposed for the first time, characterized by the presence of several subnetworks that approximate the independent influence of individual input signals, which makes it possible to identify new information about the properties of the processed time series and influencing factors; learning methods of neuro-fuzzy unit is improved by the use of robust learning criteria thereby reducing errors while processing data containing outliers; method for combining an ensemble of neural networks that process multi-dimensional data is improved, which is characterized by nonnegative weights limitation, which makes it possible to assess "the contribution" of each network within the ensemble in a combined estimate; neural networks learning method is improved, distinguished by taking into account the symmetry of the parameters space related to the possibility of interchanging processing elements in hidden layers and a simultaneous change of signs of groups of parameters, thus significantly reducing the volume of the optimization problem search space; neural networks learning method is improved, which unlike well-known methods does not require missing values preprocessing, which makes it possible to directly process incomplete data sets and obtain estimates of missing values based on the representation of data sets in the reduced dimension space. Degree of implementation - the research results are used in Khartep, ltd., Kharkiv, which is confirmed by the act of 30.07.2009 and NEC "Ukrenergo", Kiev, which is confirmed by the act of 03.09.2009; scientific results, conclusions and recommendations contained in the thesis are used in the preparation of courses "Neural network methods of computational intelligence" and "Intelligent control systems and diagnostics", which are taught to students of the specialty "Intelligent Decision Support Systems" of Kharkiv National University of Radio Electronics, which is confirmed by the act of 10.09.2009, and in research projects of Kharkiv National University of Radio Electronics, which is confirmed by the act of 14.09.2009. The scope of use - in organizations that deal with similar problems of developing intelligent systems and methods for time series processing, in the areas of information technology, finance, medicine, biology, ecology, energy, transportation, and in the educational process in the preparation of specialists in the areas of intellectual information processing.

Files

Similar theses