NAMED ENTITIES DISTRIBUTION IN NEWSPAPER ARTICLES

This paper is focused on the distribution of named entities inside texts for e-learning. An important topic in natural language processing is represented by Named Entities Recognition (NER), which is essential in order to be able to automatically understand the text. In this article we experimented our approach with newspaper articles, but the same technique can be applied to texts written by students, documentation or different other sources of knowledge used in e-learning. In order to perform this analysis we considered a number of 19043 Reuters newspaper articles. The following types of named entities were considered: person names, locations and organizations. For the extraction of the named entities we used Stanford NER software. Among the statistics that we are computing are the average number of sentences per named entity, which is defined as the number of distinct named entities from an article divided by the number of sentences and the average position in the text. We are also computing for each type of named entity the distribution in the sentence and we are representing graphically the result by using cubic spline interpolation. Using the position of named entities in the text we will infer a function based on Poisson’s distribution that models the distribution of named entities. These statistics can be used afterwards in different areas of NLP. For example, if we know that usually named entities are found more often in certain areas of the text, a statistical NER can give a higher priority to those. Another example of usage for such a statistic is the case of automatic text summarization. Some summarization approaches consider always the first sentence of the text. Due to the fact that named entities are an essential factor for automatic summarization, if we would know the typical distribution of named entities we could consider in the summary the sentences that are more probably containing these special tokens. This is important for e-learning because it permits us to identify more easily topics or important sentences within learning materials and thus users are able to acquire knowledge more easily or to determine which knowledge source is relevant for them.