Survey on Automatic Text Summarization and Transformer Models Applicability

Wang, Guan; Smetannikov, Ivan; Tianxing, Man

doi:10.1145/3437802.3437832

Cited by 22 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Upon reviewing the literature, it is evident that many studies typically focus on various aspects of transformer models, including their architecture, efficiency, computational power, memory efficiency, and the development of fast and lightweight variants [52]. On the other hand, in other studies, various NLP applications have been explored, including visualization of transformers for NLP [53], examination of pre-training methods used in transformer models [54], usage of transformers for text summarization tasks [55], application of transformer models for detecting different sentiment levels from text-based data [56], and using transformers for extracting useful information from large datasets [57]. In our study, however, unlike the existing research, the aim was to detect the general perception in society regarding newly established/green energy systems and to assist decision-makers in policymaking.…”

Section: Deep Learning and Transformersmentioning

confidence: 99%

Identification of Perceived Challenges in the Green Energy Transition by Turkish Society through Sentiment Analysis

Bilgin,

Soner Kara

2024

Sustainability

View full text Add to dashboard Cite

Green energy refers to energy derived from renewable sources such as solar, wind, hydro, and biomass, which are environmentally sustainable. It aims to reduce reliance on fossil fuels and mitigate environmental impacts. In the Turkish context, alongside positive sentiments regarding the establishment of energy plants, there are also prevalent negative perspectives. Societal responses to the transition towards green energy can be effectively gauged through the analysis of individual comments. However, manually examining thousands of comments is both time-consuming and impractical. To address this challenge, this study proposes the integration of the Transformer method, a Natural Language Processing (NLP) technique. This study presents a defined NLP procedure that utilizes a multi-labeled NLP model, with a particular emphasis on the analysis of comments on social media classified as “dirty text”. The primary objective of this investigation is to ascertain the evolving perception of Turkish society regarding the transition to green energy over time and to conduct a comprehensive analysis utilizing NLP. The study utilizes a dataset that is multi-labeled, wherein emotions are not equally represented and each dataset may contain multiple emotions. Consequently, the measured accuracy rates for the risk, environment, and cost labels are, respectively, 0.950, 0.924, and 0.913, whereas the ROC AUC scores are 0.896, 0.902, and 0.923. The obtained results indicate that the developed model yielded successful outcomes. This study aims to develop a forecasting model tailored to green energy to analyze the current situation and monitor societal behavior dynamically. The central focus is on determining the reactions of Turkish society during the transition to green energy. The insights derived from the study aim to guide decision-makers in formulating policies for the transition. The research concludes with policy recommendations based on the model outputs, providing valuable insights for decision-makers in the context of the green energy transition.

show abstract

Section: Deep Learning and Transformersmentioning

confidence: 99%

Identification of Perceived Challenges in the Green Energy Transition by Turkish Society through Sentiment Analysis

Bilgin,

Soner Kara

2024

Sustainability

View full text Add to dashboard Cite

show abstract

“…Seq2seq models are the basis of natural language processing (NLP) systems that are frequently used to translate from one language to another [10], summarize text [11] and even map images to textual summaries of them [12]. Transformer-based models [8] are now considered the state-of-the-art in NLP applications.…”

Section: Related Workmentioning

confidence: 99%

SYMBA: symbolic computation of squared amplitudes in high energy physics with machine learning

Abdulhakim

Gleyzer

Prosper

2023

Mach. Learn.: Sci. Technol.

View full text Add to dashboard Cite

The cross section is one of the most important physical quantities in high-energy physics and the most time consuming to compute. While machine learning has proven to be highly successful in numerical calculations in high-energy physics, analytical calculations using machine learning are still in their infancy. In this work, we use a sequence-to-sequence model, specifically, a transformer, to compute a key element of the cross section calculation, namely, the squared amplitude of an interaction. We show that a transformer model is able to predict correctly 97.6% and 99% of squared amplitudes of QCD and QED processes, respectively, at a speed that is up to orders of magnitude faster than current symbolic computation frameworks. We discuss the performance of the current model, its limitations and possible future directions for this work.

show abstract

“…Generative AI refers to models and techniques that have the ability to generate new and original content, and within this domain, LLMs specialize in generating text. An LLM such as OpenAI's GPT (Generative Pre-trained Transformer) is basically trained to generate text, or rather to answer questions with paragraphs of text (Guan et al, 2020). Once trained, it can generate complete sentences and paragraphs that are coherent and, in many cases, indistinguishable from those written by humans, simply from an initial stimulus or prompt (Madotto et al, 2021).…”

Section: What Is Generative Ai?mentioning

confidence: 99%

Generative Artificial Intelligence: Fundamentals

Corchado,

López F.,

Núñez V.

et al. 2023

ADCAIJ

View full text Add to dashboard Cite

Generative language models have witnessed substantial traction, notably with the introduction of refined models aimed at more coherent user-AI interactions—principally conversational models. The epitome of this public attention has arguably been the refinement of the GPT-3 model into ChatGPT and its subsequent integration with auxiliary capabilities such as search features in Microsoft Bing. Despite voluminous prior research devoted to its developmental trajectory, the model’s performance, and applicability to a myriad of quotidian tasks remained nebulous and task specific. In terms of technological implementation, the advent of models such as LLMv2 and ChatGPT-4 has elevated the discourse beyond mere textual coherence to nuanced contextual understanding and real-world task completion. Concurrently, emerging architectures that focus on interpreting latent spaces have offered more granular control over text generation, thereby amplifying the model’s applicability across various verticals. Within the purview of cyber defense, especially in the Swiss operational ecosystem, these models pose both unprecedented opportunities and challenges. Their capabilities in data analytics, intrusion detection, and even misinformation combatting is laudable; yet the ethical and security implications concerning data privacy, surveillance, and potential misuse warrant judicious scrutiny.

show abstract

Survey on Automatic Text Summarization and Transformer Models Applicability

Cited by 22 publications

References 16 publications

Identification of Perceived Challenges in the Green Energy Transition by Turkish Society through Sentiment Analysis

Identification of Perceived Challenges in the Green Energy Transition by Turkish Society through Sentiment Analysis

SYMBA: symbolic computation of squared amplitudes in high energy physics with machine learning

Generative Artificial Intelligence: Fundamentals

Contact Info

Product

Resources

About