Ph.D thesis is devoted to solving actual scientific problem of development the method of constructing NTRU-like encryption schemes that are provable secure against chosen plaintext attacks. In recent years, numerous researches has been carried out in the field of quantum technologies and quantum computers, which use quantum mechanical phenomena to solve computational problems that are practically unsolvable with the help of conventional computers. Due to the fact that the appearance of a quantum computer is only a matter of time, there is a threat to the current state of security of special information and communication systems. This necessitates the creation of new cryptosystems that are security to quantum attacks. Today, NTRU-like cryptosystems form one of the most promising classes of post-quantum cryptosystems. Almost a third of all cryptosystems and protocols submitted to the NIST post-quantum competition (PQC) are lattice or NTRU-like. In addition, the latest post-quantum public-key encryption algorithm standardized in Ukraine (DSTU 8961:2019 “Skelya”) is also NTRU-like. Progress in lattice cryptography stimulates the creation of symmetric post-quantum encryption schemes, the security of which is based on the complexity of solving only one particular problem. Along with that, the only symmetric NTRU-like encryption scheme known today (NTRUCipher) is vulnerable to certain attacks. For the first time, analytical relations for estimating the probability of reversibility of random polynomials used in NTRU-like encryption schemes were obtained. In contrast to the known relation for the probability of reversibility of a random equiprobable element of a truncated polynomials ring, the obtained relations are valid for a more general scheme of random polynomials formation. They are based on the application of the Fourier transformation of probability distributions on a finite field and make it possible to estimate (and in some practically important cases to calculate) the probability value of reversibility of random polynomials used as components of secret keys of NTRU-like encryption schemes. Analytical relations for estimating decryption failure probability in NTRU-like encryption schemes were improved. Unlike the previously known ones, the obtained relations are valid for all types of modern NTRU-like encryption schemes (both asymmetric and symmetric one). In addition, they make it possible to estimate the decryption failure probability of messages in NTRU-like encryption schemes for a fixed key, thus providing more adequate information about the frequency of decryption failure. The method of estimating the security of symmetric encryption schemes NTRUCipher and NTRUCipher+ due to the research of three additional attacks on these encryption schemes was further developed. Analytical estimates of complexity for the mentioned attacks were obtained and it was shown that at least one of them can be implemented in real time (although it does not allow to recover the keys of encryption schemes but only to distinguish the sequences of their encrypted messages from a purely random sequence). For the first time, the method of constructing provable secure NTRU-like encryption schemes was proposed. It is shown that, in contrast to known symmetric NTRU-like encryption schemes, the proposed encryption schemes have provable security against chosen plaintext attacks, which is based on the complexity of reference computational Decision-Ring-LWE problem. The practical significance of the obtained results consist in developing the software implementations that allow in real time to calculate the parameters values for constructing proposed provable secure NTRU-like encryption schemes, to calculate the probability of reversibility of random polynomials and the decryption failure probability in arbitrary NTRU-like encryption schemes. In addition, the results obtained in this thesis allow: - reduce the probability of irreversibility of a random polynomial in the ring Rn.q (from 0,5 to 1,5∙10^-2) due to proper selection of parameters q and n of NTRU-like encryption scheme; - choose the parameters of NTRU-like encryption schemes that provide an appropriate (small) value of decryption failure probability of messages for a fixed secret key; - establish that complexity of BKW-attack on the NTRUCipher+ encryption scheme is in 2^15-2^69 times higher compared to the complexity of a similar attack on the NTRUCipher encryption scheme; - prove that the NTRUCipher+ encryption scheme is quite vulnerable to a distinguishing attack that can be implemented in real time (at the same time, the largest value of the material’s amount required to implement the attack is t=2^19); - choose parameters of NTRU-like encryption schemes that guarantee their security at a predetermined level λ (in particular n=631, q=2693, d=56 at λ=2^128, n=883, q=8089, d=168 at λ=2^256).
