Thematic texts generation issues based on recurrent neural networks and word2vec

Abstract

Urgency of the research. The problem of natural language generation is becoming more actual in recent days due to the growing demand for automated generation of object descriptions, article excerpts, news summaries, passages in microblogging services, response messages used by chat bots, etc. Thus, the problem is to generate a text given the context. This paper deals with the problem of generating text specifically in Russian since each language group requires an individual approach.

Target setting. There is no method to generate thematic texts automatically, especially in Russian language, that gives well-interpreted and suitable results.

Actual scientific researches and issues analysis. In the past few years, more articles have been devoted to the topic of generating thematic texts, due to the emergence of new methods for sequences generation using recurrent neural networks. However, approaches related specifically to thematic texts generations, in Russian are insufficiently explored.

Uninvestigated parts of general matters defining. This article focuses on a study and analysis of the proposed approach for generating Russian-language thematic texts. It is specialized in one language group and specific approach in terms of model selection.

The research objective. Create model trained on a group of short passages that identifies a context of a text and as output generates a well-interpreted natural text in Russian.

The statement of basic materials. The analysis of the joint use of the RNN and word2vec models is conducted. Approaches for the transformation of the input text, analysis of sentences structure, prediction of subsequent parts of speech, prediction of following words and the general model structures are proposed. The results of the models are appeared to be well interpreted and meaningful.

Conclusions. The iinterpretability, structure and parameters of the models that showed the best results for the generation were analyzed. The approach proved to be good for generating thematic texts. The results and analysis of the subsequent steps are given.