Onah, Daniel FO;
Pang, Elaine LL;
El-Haj, Mahmoud;
(2022)
A Data-driven Latent Semantic Analysis for Automatic Text Summarization using LDA Topic Modelling.
ArXiv
Preview |
Text
preprint_manuscript.pdf - Submitted Version Download (2MB) | Preview |
Abstract
With the advent and popularity of big data mining and huge text analysis in modern times, automated text summarization became prominent for extracting and retrieving important information from documents. This research investigates aspects of automatic text summarization from the perspectives of single and multiple documents. Summarization is a task of condensing huge text articles into short, summarized versions. The text is reduced in size for summarization purpose but preserving key vital information and retaining the meaning of the original document. This study presents the Latent Dirichlet Allocation (LDA) approach used to perform topic modelling from summarised medical science journal articles with topics related to genes and diseases. In this study, PyLDAvis web-based interactive visualization tool was used to visualise the selected topics. The visualisation provides an overarching view of the main topics while allowing and attributing deep meaning to the prevalence individual topic. This study presents a novel approach to summarization of single and multiple documents. The results suggest the terms ranked purely by considering their probability of the topic prevalence within the processed document using extractive summarization technique. PyLDAvis visualization describes the flexibility of exploring the terms of the topics' association to the fitted LDA model. The topic modelling result shows prevalence within topics 1 and 2. This association reveals that there is similarity between the terms in topic 1 and 2 in this study. The efficacy of the LDA and the extractive summarization methods were measured using Latent Semantic Analysis (LSA) and Recall-Oriented Understudy for Gisting Evaluation (ROUGE) metrics to evaluate the reliability and validity of the model.
| Type: | Working / discussion paper |
|---|---|
| Title: | A Data-driven Latent Semantic Analysis for Automatic Text Summarization using LDA Topic Modelling |
| Open access status: | An open access version is available from UCL Discovery |
| DOI: | 10.48550/arXiv.2207.14687 |
| Publisher version: | https://doi.org/10.48550/arXiv.2207.14687 |
| Language: | English |
| Additional information: | This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. |
| Keywords: | cs.IR, cs.IR, cs.LG |
| UCL classification: | UCL > Provost and Vice Provost Offices > UCL SLASH > Faculty of Arts and Humanities UCL > Provost and Vice Provost Offices > UCL SLASH > Faculty of Arts and Humanities > Dept of Information Studies UCL > Provost and Vice Provost Offices > UCL SLASH UCL |
| URI: | https://discovery.ucl.ac.uk/id/eprint/10154663 |
Archive Staff Only
 |
View Item |
