Nasirov E. Parallelization of non-negative huge sparse linguistic matrix and tensors factorization

The paper describes algorithms and methods for parallelizing non-negative factorization of sparse matrices and tensors - popular technology in artificial intelligence in general, and in computational linguistics in particular. Two methods of parallelization of the algorithm for factorization of non-negative matrices are proposed: a local algorithm using a hard disk and computations on GPUs and a distributed algorithm using a network of computational nodes and GPUs. The paper also proposes a block-diagonal approach to factorization of inherent sparse linguistic matrices and tensors, which can be reduced to a block-diagonal form. The proposed method also allows the model to be supplemented with new data without no need to perform the nonnegative factorization of the entire super-large tensor from the very beginning. It is also proposed to use the latent Dirichlet distribution to reduce matrices and tensors to the block-diagonal form by constructing thematic diagonal blocks.