Tackling redundancy in text summarization through different levels of language analysis

One of the main challenges to be addressed in text summarization concerns the detection of redundant information. This paper presents a detailed analysis of three methods for achieving such goal. The proposed methods rely on different levels of language analysis: lexical, syntactic and semantic. Moreover, they are also analyzed for detecting relevance in texts. The results show that semantic-based methods are able to detect up to 90% of redundancy, compared to only the 19% of lexical-based ones. This is also reflected in the quality of the generated summaries, obtaining better summaries when employing syntactic- or semantic-based approaches to remove redundancy.

This research has been funded by the Spanish Government under the project TEXT-MESS 2.0 (TIN2009-13391-C04-01). Moreover, it has been also supported by Conselleria d'Educació —Generalitat Valenciana (grant no. PROMETEO/2009/119 and ACOMP/2010/286).