Horbatiuk V. Information technology for non–stationary time series forecasting based on neural networks

Українська версія

Thesis for the degree of Candidate of Sciences (CSc)

State registration number

0421U101042

Applicant for

Specialization

  • 05.13.06 - Інформаційні технології

16-04-2021

Specialized Academic Board

Д 26.002.29

National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute"

Essay

The thesis is devoted to the development of information technology together with new and improved methods and models for the relevant scientific and applied problem of time series forecasting. Classical approaches to the forecasting problem are based on the use of probability theory and mathematical statistics assuming that the forecasted object’s model (linear, polynomial, exponential and others) are precisely known. They also typically assume the predicted object to be stationary or expect a specific type of “nonstationarity”. It is expedient and relevant to develop new methods, models and information technology for non–stationary time series forecasting with the use of neural networks, which will improve the quality of the forecast by having more flexible assumptions about forecasted object and by reducing the influence of overfitting when amount of available data points is limited. A new class of non-stationary time series is described. Due to focusing on a specific class of time series, its properties can be used when developing new methods, models and algorithms for building forecasting models for the selected time series class. Existing methods of non-stationary time series forecasting have been analyzed, together with their main advantages and disadvantages when applying to the selected class of time series. A new generic approach for the forecasting models building for time series from the selected class has been developed. The approach is based on the analysis of the class properties, along with the advantages and disadvantages of existing methods. The approach consists of two high level steps: first, building the model that determines the currently active conditional probability distribution for the given input vector and the second step – building multiple local models based on the previously estimated information of active conditional probability distributions for each sample vector. The multi–row GMDH algorithm has been improved by using the backpropagation and dropout, which allowed to fine–tune the forecasting model initial parameters thus improving the forecasting accuracy. A new model of an artificial neuron – Sigmoid Piecewise (SP) neuron has been developed, which is more efficient for time series from the selected class description. It consists of three weighted sum blocks and a new activation function of three variables, which allows to obtain a model that approximates a simple piecewise linear function and to independently adjust parameters, that define the piecewise function’s separating hyperplane, and parameters, that define linear functions for the inputs from different sides of the hyperplane. Due to these differences, the new model can better approximate certain functions and is more suitable for time series from the selected class. Experimental tests were carried out on real world time series, in which it was enough to use 2 times fewer parameters in the SP network than in the ReLU network to achieve the same error value. In another test, the error of SP network on the test set was the smallest among all other methods being compared, and 8% less than the error of the second most accurate method (ARIMA). The soft clustering method has been improved by using the separating hypersurfaces model, which allowed solving the clustering problem for a certain class of criteria as a differentiable function’s optimization problem. In this case, the number of parameters that need to be tuned does not depend on the number of samples, making it possible to limit the number of parameters if necessary. A new method for non-stationary time series forecasting has been developed. The method is based on clustering implementation followed by a construction of multiple local forecasting models. The method uses improved soft clustering method based on separating hypersurfaces model and the special regularization. The added special regularization reduces the influence of the dataset size on the quality of the obtained forecasting model and increases resilience of local models to the overfitting. New and improved methods have been compared with existing ones on the artificial and real datasets. Based on the created methods and models, the information technology for non–stationary time series forecasting has been developed to forecast the demand for a line of relay protection MRZS devices at the state enterprise «PA Kyivprylad». As a result, the forecasting model had 11% lower mean absolute error than the best corresponding error among tested existing methods. It became possible to more accurately determine the demand for components and consumables, and thereby reduce costs.

Files

Similar theses