The thesis is devoted to developing the mathematical methods of cryptanalysis of new asymmetric ciphers with special properties and creating the mathematical methods of increasing the performance of the components of the given ciphers. In accordance with the terminology used in this study, such ciphers are the asymmetric encryption schemes that possess useful properties generally not inherent in this class of ciphers and thus have wider application area in comparison with the one arising from the definition of an asymmetric encryption scheme. Homomorphity, post-quantumness and noise immunity are the examples of these properties. The aim of the thesis is to determine the properties and cryptographic strength of the matrix-based knapsack cipher, as well as to increase the performance of the components of the fountain QC-MDPC McEliece cipher. This aim has been achieved by means of solving the following tasks: determine the quantitative properties of the matrix-based knapsack cipher, which include the time complexities of encryption, decryption and key pair generation, the sizes of the public and private keys, the ciphertext expansion factor and the level of security against a brute-force attack; determine for the matrix-based knapsack cipher the presence of the property of additive homomorphity; develop such a method of cryptanalysis of the matrix-based knapsack cipher that is more efficient than a brute-force attack and determine the computational complexity of the method developed; develop such a method for generating a robust soliton distribution that has better performance than the standard method; implement research results. These tasks have been solved using the methods of abstract algebra (the theory of finite fields, the group theory), linear algebra, probability theory, mathematical statistics and the theory of computational complexity. The research has yielded the following scientific results. For the first time, the quantitive properties of the matrix-based knapsack cipher have been determined. The properties, which have been considered, include the time complexities of encryption, decryption and key pair generation, the sizes of the public and private keys, the ciphertext expansion factor and the level of security against a brute-force attack. This result is required to substantiate decisions about expediency of using the given cipher. For the first time, the property of additive homomorphity for the matrix-based knapsack cipher has been proved. The given result indicates the possibility of using this cipher to build a protocol of secret e-voting. For the first time, polynomial-time methods of cryptanalysis of the matrix-based knapsack cipher have been proposed. These attacks permit recovering the plaintext from the ciphertext of the given cryptosystem in the absence of the secret key. The aforesaid result substantiates the conclusion about inexpediency of using this cipher as a privacy tool and thus permits eliminating the information security risks, which arise from the use of the given cryptoscheme. A list of these risks can be made using the first two results. For the first time, the majorant-superposition method for generating a robust soliton distribution has been proposed. This method differs from the standard one in time complexity, which is O(1) provided that long arithmetics is not used. Applying the proposed method permits increasing several times the performance of generating a robust soliton distribution by the encoder of the LT error-correction codes, which are used in the fountain QC-MDPC McEliece cipher. First of all, the given result is of practical significance for simulation modeling. The matrix-based knapsack cipher and the proposed attack on it have been software implemented as the dynamic link library MBKCLib and applications, which permit using and timing its facilities. The proposed and standard methods for generating a robust soliton distribution have been software implemented as the dynamic link library RSDLib and application RSDTest, which has been designed for verification of correctness of the given methods by means of the Pearson's goodness-of-fit test and for investigation of their performance. The obtained results have been implemented in the international project «TEMPUS SEREIN: Modernization of Postgraduate Studies on Security and Resilience for Human and Industry Related Domains», two state scientific-research works, as well as in the education process of the Department of Computer Systems, Networks and Cybersecurity of National Aerospace University «Kharkiv Aviation Institute». The thesis materials have been included as components in three scientific-research reports and the textbook developed within the framework of the aforementioned project. The thesis materials have been published in scientific periodicals as 6 articles, three of which are Scopus publications, as well as in proceedings of scientific conferences as 2 papers indexed in Scopus.
