Dynamic Optimization of Minimum Document Frequency for Extractive Text Summarization Using GreedSum Algorithm

2024

Closed access Metadata (CSL-JSON)
DOI 10.1109/opcs63516.2024.10720390
Data confidence 70 medium
How this score is calculated

It is built from checkable properties of the record itself: whether the article has a DOI, how many independent sources confirmed its data, whether the required fields are filled in, and how recent the latest confirmation is. Disagreement between sources and a missing identifier lower it. A value of 80 or above means the record was published without further checking; 60 to 79 means it is published at reduced confidence, with a manual check advisable. A record verified by an editor by hand is never pushed below 90 by a recalculation. This scores our data about the article, not the article itself.

Abstract

This research presents a significant advancement in the GreedSum algorithm, a previously introduced extractive text summarization method. The core innovation lies in the dynamic adjustment of the minimum document frequency (mindf) parameter within the Term Frequency-Inverse Document Frequency (TF-IDF) framework. By integrating a clustering approach, we explored the hypothesis that optimizing the min-df for individual documents can enhance the quality of summaries, as measured by ROUGE scores. The methodology involved grouping similar documents into clusters and fine-tuning the min-df parameter based on cluster characteristics. Extensive experimentation with a large dataset confirmed that this dynamic optimization yields superior results compared to static parameter settings. The findings underscore a novel strategy for improving the accuracy and relevance of automated summarization in scientific literature, offering a tailored approach that adapts to the unique features of each document cluster. This research not only refines the summarization process but also opens new avenues for adaptive natural language processing systems in information extraction and text analysis. Future work will focus on refining dynamic parameter adjustment techniques and exploring additional evaluation metrics to enhance the robustness and applicability of summarization tools across various domains.