Automatic Text Summarization in Natural Language Processing

Text summarization is one of the challenging and engrossing problems in the field of Natural Language Processing. The goal of Text Summarization is to read, understand and extract meaningful information from the unstructured data. The idea behind summarization is to get concise and precise summary from the original document. We used Sentence Ranking method based on Word Frequency to determine the important sentences in original text. The Summarizer uses three features to determine important Sentence in a document. The Term Frequency, similarity with the Title and Sentence score are the features considered. Values are assigned to each word and a sentence in a document and create a score for each word and a sentence. Normalize the scores to avoid raw scores that skew the results. The Scores of top 'N' sentences are taken for summarization. Text cleaning, Sentence tokenization, Word tokenization, Normalized Word frequency table generation, and Summarization are steps used in the Sentence Ranking Method. The method is implemented on WikiHow dataset. The results of proposed model are evaluated with ROUGE toolkit which has three evaluation metrics- Precision, Recall and F-Score.