PhD thesis in the field of knowledge 12 Information technologies in a specialty 121 Software engineering. – National Technical University of Ukraine “Igor Sikorsky Kyiv Polytechnic Institute”, Kyiv, 2025.
Artificial intelligence is one of the most prominent fields of software development in the modern world of information technologies. Significant progress has been achieved in computer vision tasks, particularly object detection, over the past decade due to the advancements in deep learning methods and the growth of computational capabilities.
For the successful development and implementation of AI-based software tools, it is necessary to collect and annotate large volumes of data, which requires considerable human resources and time. Low-quality data annotation can lead to inaccurate and erroneous AI methods, consequently resulting in errors in software computations. Current data annotation tools do not always meet the needs of software developers working with AI, especially in the context of large-scale video data annotation, which increases the labor intensity of creating high-quality datasets.
The outlined problems establish a pressing scientific task of improving the efficiency of video data annotation processes for computer vision tasks, which is addressed in this dissertation.
The aim of the dissertation is to enhance the speed of video data annotation in object detection tasks by improving the methods and software tools designed for video data annotation.
To achieve this aim, the study investigates neural network training methods that improve object detection accuracy without modifying models or increasing number of parameters, as well as approaches to reduce the number of frames processed by computer vision methods. Conducted study on visual-language models to improve accuracy. A dual-architecture system for automated data annotation and its supporting software have been developed. Experimental research demonstrates the effectiveness of the proposed solutions.
One of the main challenges in achieving the aim is ensuring high-quality automated data annotation, which minimizes the percentage of errors requiring manual correction. To address this, various techniques and methods were employed, including the novel dual-architecture approach, a data prioritization method, an iterative keyframe selection method, and multimodal neural networks.
The main result of this work is the creation of a dual-architecture software system for data annotation automation and the implementation of automatic video annotation methods. These methods ensure high annotation accuracy and reduce the time needed for post-annotation refinement by annotators after automated annotation. The developed methods were tested on real-world tasks to demonstrate their efficiency and advantages.
In the dissertation, a number of new scientific results were obtained:
For the first time, a dual software architecture for automated data annotation has been proposed. Utilizing the method of adaptively-aggregated neural network training, this architecture accelerates the annotation process and, unlike existing counterparts, enables the effective application of zero-shot and active neural network learning for data annotation. It also allows for more flexible software utilization across various computer vision tasks.
For the first time, a novel method for prioritizing difficult samples for neural network training is introduced, improving dataset quality without prior video annotation and enhancing object detection accuracy. Unlike existing approaches, this approach relies solely on automatically generated data representations.
For the first time, an iterative method for selecting keyframes in long videos is proposed, enabling accurate identification of keyframes and segments while accounting for dynamic content changes, improving segmentation accuracy and reducing video data for processing.
For the first time, a method for aggregating knowledge between the textual and visual components in a visual-language model (VLM) has been proposed to model complex multimodal interactions, providing higher accuracy in recognizing complex scenes in videos and their descriptions compared to existing counterparts..
The main results of the dissertation were published in 6 scientific papers, in particular, in 4 scientific articles, which is indexed in the Scopus database, 1 article published in a scientific journal included in the list of scientific professional editions of Ukraine (category «B»), as well as 1 publication in materials of scientific and technical conferences.
