Rvach D. Algorithms and Software for Digital Twin Technology of Mulsemedia Objects. – Qualifying scientific work, manuscript.
PhD thesis in the field of knowledge 12 Information technologies in speciality 121 Software Engineering. – National Technical University of Ukraine "Igor Sikorsky Kyiv Polytechnic Institute", Kyiv, 2024.
Digital twins are a technology that is purposed for creation a digital copy of a physical object for further analysis and simulation of its behavior. The technology of digital twins offers a new approach to the presentation and processing of a dynamic digital model of a physical object or process, its past, present and future states and behavior. A digital twin is a collection of virtual information structures that fully describe a future or actually existing physical object from the micro level (single-element level) to the macro level (general appearance, general properties of the object as a whole). Digital twin technology connects a specific physical system with a computer model that reflects the architecture, dynamics, and actual state of that specific system. Sensors that enable continuous monitoring of the object can be used as sources of information to create such individualized dynamic models.
Analysis of existing software systems for working with digital twins shows that they are narrowly focused on creating digital twins of specific objects for a specific industry and are not able to follow a more universal approach to creating a digital copy of a real object. The problem of lack of unification and standardization of data exchange formats between the physical and digital copy of the object was also identified to a large extent.
Solving these tasks is possible by creating a unified, flexible and standardized software system that allows creating digital copies of physical objects with different parameters and adjusting work with them according to the necessary requirements.
Digital twin technology is commonly used in manufacturing and engineering. On the other hand, there are practically no examples of the application of this technology in both education and healthcare. Nevertheless, the application of the technology of digital twins in these industries has great potential, as it is possible to achieve a new quality of education and healthcare services due to its use. However, to adapt the technology of digital twins to the requirements of these industries, the feature of which is the direct participation of a human in processes, procedures and services, it is reasonable to take into account the latest technical capabilities aimed at digital processing of data on the physical properties of real-world objects that a human is able to perceive. Therefore, an urgent task is to expand the technology of digital twins by applying the concept of mulsemedia, which makes it possible to create new interfaces for human-machine interaction.
The purpose of the dissertation is to increase the efficiency of mulsemedia objects state data processing in software systems based on the concept of a digital twin.
In the first section of the dissertation, methods, approaches, and software for processing data of digital twins of mulsemedia objects were analyzed. The technology of digital twins and the concept of mulsemedia were studied. The current state of existing software systems based on the technology of digital twins was studied. Insufficient universality of approaches to creating a digital twin in the specified software systems was revealed. As a result, functional and non-functional requirements for this class of software were formed.
In the second section, algorithmic and software solutions for the consolidation of mulsemedia data based on the concept of multi-image have been developed. A method of mulsemedia data consolidation is proposed, which involves processing mulsemedia data using an algebraic system of aggregates. An extension of the JSON format to create a time-synchronized mulsemedia data structure named TJSON is proposed. Discrete wavelet transforms and steganography algorithms were used to combine data from different modalities into a single digital object. It is proposed to use parallel calculations to increase the effectiveness of the implementation of the proposed method.
In the third section, available approaches to data processing of mulsemedia objects are considered, one of which is based on the application of the ASAMPL programming language. Ways to improve the ASAMPL programming language are proposed, namely: the syntax has been updated, which made it possible to improve code metrics when programming in the ASAMPL language; new software designs were introduced, which expanded the capabilities of the language, in particular, made it possible to use parallel calculations. The specified changes are included in the new version of the programming language named ASAMPL 2.0.
