Enhancing Transformer-based language models with commonsense representations for knowledge-driven machine comprehension

Li, Ronghan; Jiang, Zejun; Wang, Lifang; Lu, Xinyu; Zeng, Meng; Chen, Daqing

doi:10.1016/j.knosys.2021.106936

Cited by 16 publications

(6 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…During a memorable encounter with Mr. Nagao Zhen at an academic conference in Singapore in the summer of 1996, he expressed that machine translation was beginning to turn a profit, an announcement that visibly brought him relief [2]. This sentiment resonates deeply with those who spend a lifetime conducting research that initially seems to yield no return, continuously convincing others of the value in such an investment.…”

Section: Literature Reviewmentioning

confidence: 99%

Improving machine translation and post-editing for Chinese tourism texts using transformer-based models

Feng,

Hu,

Yang

2024

ACE

View full text Add to dashboard Cite

As the digital age and globalization continue to evolve, the demand for accurate machine translation of tourism texts has increased substantially. This paper investigates how to improve the quality of machine translation (MT) and machine translation post-editing (MTPE) of Chinese tourism texts for non-native speakers. A review of the machine translation literature reveals a significant progression in translation methods from rule-based to corpus-based, statistical, and finally to the current neural machine translation (NMT) models. Despite its advanced capabilities, NMT requires large amounts of parallel data for training, which often presents challenges. This study proposes the use of Transformer-based models for MT and MTPE to improve translation quality. A dataset was curated from online sources, mainly Chinese tourism websites. The methodology involved pre-processing the data, performing machine translation using the Transformer model, and post-editing the results. The experiment demonstrated an increase in the BLEU score, suggesting an improvement in translation quality. However, challenges such as the handling of synonyms and geographical nouns were encountered, indicating the need for further research and model optimization.

show abstract

Section: Literature Reviewmentioning

confidence: 99%

Improving machine translation and post-editing for Chinese tourism texts using transformer-based models

Feng,

Hu,

Yang

2024

ACE

View full text Add to dashboard Cite

show abstract

“…LLMs have attracted much attention for understanding the commonsense and reasoning patterns in their latent space ( Zhou et al, 2020 ; Li et al, 2021 ; Bian et al, 2023 ). It has been shown that some abilities in logical and mathematical reasoning seem to emerge when LLMs are prompted appropriately ( Wei et al, 2022b ; a ).…”

Section: State Of the Artmentioning

confidence: 99%

Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning

et al. 2023

View full text Add to dashboard Cite

Long-horizon task planning is essential for the development of intelligent assistive and service robots. In this work, we investigate the applicability of a smaller class of large language models (LLMs), specifically GPT-2, in robotic task planning by learning to decompose tasks into subgoal specifications for a planner to execute sequentially. Our method grounds the input of the LLM on the domain that is represented as a scene graph, enabling it to translate human requests into executable robot plans, thereby learning to reason over long-horizon tasks, as encountered in the ALFRED benchmark. We compare our approach with classical planning and baseline methods to examine the applicability and generalizability of LLM-based planners. Our findings suggest that the knowledge stored in an LLM can be effectively grounded to perform long-horizon task planning, demonstrating the promising potential for the future application of neuro-symbolic planning methods in robotics.

show abstract

“…While working with different browsers, we need to specify the region of actual web content and ignore the desktop background, address input, or tabs. The Accessibility Framework 12 incorporates an extensive array of tools and functionalities aimed at enhancing the accessibility of Apple devices. For developers, this framework offers the opportunity to delve into the accessibility graph that contains all UI elements and their relations, providing an additional source of information regarding the active application's state, layout, and controls.…”

Section: Macos Applicationmentioning

confidence: 99%

“…State-ofthe-art approaches, such as PhishIntention [14], Phishpedia [13], and VisualPhishNet [1], primarily focus on automatically identifying phishing websites on the internet using web crawlers rather than relying on local inference. Solutions, such as DynaPhish [16] and KnowPhish Detector [12], further improve accuracy and extend the applicability of reference-based approaches but still are cloud-first. Our work addresses the performance aspect of such approaches, bringing them locally to the user devices without compromising on phishing detection accuracy.…”

Section: Related Workmentioning

confidence: 99%

Standardization of Implementation of the Mandatory Privately Managed Pillar of the Pension System of Ukraine

Petrukha¹,

Petrukha²,

SVIRIDOVSKA³

2021

EFMR

View full text Add to dashboard Cite

According to the current legislation, the modern Ukrainian pension system is not yet fully formed. In Ukraine, PFU contributions are currently a source of pension benefits for today's retirees. Pay-as-you-go pension system is essentially failing, because as in the rest of the world, our country is experiencing a trend in which the nation is aging: the share of retirees is growing, and the share of able-bodied people is declining. The search of new ways to save for old age is moving toward the creation of mandatory accumulation under the supervision of the state, in particular, this is also caused by the signed by Ukraine Memorandum of Cooperation with the IMF. However, despite the long-term pension reform in Ukraine (in started back in 2004), today, in fact and in law the second level of the pension system doesn't function and doesn't existthe mandatory accumulative one; as well as very underdeveloped, almost non-functioning and completely unpopular among the population is non-state pension system, which forms the third level of the national pension system. Still, against the background of active cooperation with the IMF a number of reforms continue in Ukraine, so in 2019 work on new legislation on the defined contribution pension system began. It is this level of the pension system that can technically increase the level of pensions and their differentiation. Yet the experience of countries that have already carried out such reforms shows that the real effect can be expected for at least 10-15 years, or even 25 years, and in some cases such an effect will not be reached at all. The article is devoted to the issues of legislative implementation of defined contribution pensions. According to the results of an expanded analysis of 27 bills on the referred issue in the period from 2016 to 2021, it can be stated that there is no unifying concept for the reform in the legislative field, so most bills are either withdrawn or sent for revision. In general, the bills of 2016-2020 drew attention to the scattering of changes, which mostly concern certain issues of pension legislation, mainly for vulnerable groups. The new projects, which were sent for consideration in 2019, mainly deal with defined contribution level of pension system. However, even they are constantly rejected, refined, returned for consideration and go through the whole procedure again from the beginning.

show abstract

Enhancing Transformer-based language models with commonsense representations for knowledge-driven machine comprehension

Cited by 16 publications

References 21 publications

Improving machine translation and post-editing for Chinese tourism texts using transformer-based models

Improving machine translation and post-editing for Chinese tourism texts using transformer-based models

Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning

Standardization of Implementation of the Mandatory Privately Managed Pillar of the Pension System of Ukraine

Contact Info

Product

Resources

About