Melnychenko A. Methods and software tools for improving the performance of pattern recognition models based on machine learning

This dissertation is devoted to the analysis of neural network optimization methods and the development of software tools to increase the performance of neural networks during training and execution. In today's high-tech world, neural networks have come to the forefront as a key technology. This variation of mathematical models has demonstrated high performance in many tasks ranging from computer vision to natural language understanding, thereby becoming an integral part of everyday life. However, the deployment of neural networks in real-world scenarios is often hampered by their computational complexity and resource intensity. The large amount of power consumption required to train and use large neural models also has a negative impact on the environment. Computational complexity often manifests itself in the form of a large number of parameters and deep architectures that require a significant amount of computing power both for training and for further use on end devices. This complexity is particularly problematic in applications of neural networks on Internet of Things (IoT) devices, where computing resources are often limited. Resource-intensive characteristics include computing power and memory usage. This issue is particularly relevant in mobile and embedded devices where memory is a limited resource. Moreover, the latency caused by a lack of resources is often unacceptable in a number of tasks, including autonomous control systems, where even a small delay in decision-making can have serious consequences. Optimization of neural networks is an urgent task in the technology industry, which is emphasized by empirical data. The amount of computing resources required to train state-of-the-art neural networks has doubled approximately every 3 months since 2012. This exponential growth in computational requirements is not sustainable in the long run, especially considering the energy consumption and environmental impact associated with data centers. The purpose of this thesis is to increase the efficiency of neural network models, namely, to reduce the loss of accuracy while increasing performance, after applying methods to optimize deep learning models created to solve computer vision problems. The scientific novelty of the results is as follows. An improved model of the RetinaFace neural network for face detection is proposed, which, unlike the existing ones, uses the SNIP pruning method for optimization, which allows the use of sparse matrices for storing and executing the network for further improvement and performance. An improved SNIP pinning method for the RetinaFace face detection model is proposed, which, unlike the existing ones, provides for the possibility of excluding contextual modules from the pinning process. The improved method allows achieving higher accuracy with the same number of excluded parameters. For the first time, a pre-training tuning method for transformer architecture models has been developed, which, unlike the existing ones, takes into account the importance of the "attention" mechanism. The use of the developed method allows to significantly increase the accuracy of classification of the final model compared to the SNIP method. For the first time, a software architecture for modelling and studying pre-training methods for neural networks has been developed, which, unlike existing ones, allows to reduce the matrix of network weights to a sparse format using the proposed mechanism for assessing the importance of weights. The optimized RetinaFace network contains 68% fewer parameters than the original network, with a loss of accuracy of only 1.4%. The improved method reduced the accuracy loss from 1.4% to 0.7% compared to the SNIP method when compared to the uncropped model, with a 68% reduction in parameters. Implementation of the pruning method for the transformer architecture allowed to train the network with an accuracy improvement of up to 37% compared to the SNIP method when compared to the uncut model, while reducing the number of parameters by 90%. The results of determining the criteria for the importance of weights obtained by the developed algorithm can be used to increase the performance of neural networks from 20% to 65% by using sparse matrices of 2:4 format, depending on the GPU. The study established that additional outputs for Siamese neural networks designed to establish the similarity of two images do not increase the speed of convergence and model accuracy.