A novel extractive text summarization system with self-organizing map clustering and entity recognition

Extractive text summarization yields the sensitive parts of the document by neglecting the irrelevant and redundant information. In this paper, we propose a new strategy for extractive single-document summarization in Malayalam. Initially, entity recognition is done, followed by relevance analysis is made based on some context-aware features. The scored sentences are then clustered using self-organizing maps (SOM) and from these clusters, relevant sentences are extracted out based on the proposed algorithm. Both theoretical and practical evaluations are done to analyze the implemented system. In theoretical evaluation, gradient calculations of relevance equations are used to know that which of these sentence scoring features are contributing more. The relevance equation is optimized with the help of Lagrange’s multiplier. The complexity analysis of the proposed algorithms is also performed. In practical evaluation, the system compared with online and offline summarizers upon metrics like precision, recall, and F-measure. The system is tested through a non-clustering approach also in order to analyze the impact of clustering used in our work. Some existing strategies like question game evaluation, sentence rank evaluation, and keyword association are also done to evaluate the different parameters like the relevance of sentences, important entity words, etc.