Large language model applications for evaluation: Opportunities and ethical implications

Abstract

Large language models (LLMs) are a type of generative artificial intelligence (AI) designed to produce text‐based content. LLMs use deep learning techniques and massively large data sets to understand, summarize, generate, and predict new text. LLMs caught the public eye in early 2023 when ChatGPT (the first consumer facing LLM) was released. LLM technologies are driven by recent advances in deep‐learning AI techniques, where language models are trained on extremely large text data from the internet and then re‐used for downstream tasks with limited fine‐tuning required. They offer exciting opportunities for evaluators to automate and accelerate time‐consuming tasks involving text analytics and text generation. We estimate that over two‐thirds of evaluation tasks will be affected by LLMs in the next 5 years. Use‐case examples include summarizing text data, extracting key information from text, analyzing and classifying text content, writing text, and translation. Despite the advances, the technologies pose significant challenges and risks. Because LLM technologies are generally trained on text from the internet, they tend to perpetuate biases (racism, sexism, ethnocentrism, and more) and exclusion of non‐majority languages. Current tools like ChatGPT have not been specifically developed for monitoring, evaluation, research, and learning (MERL) purposes, possibly limiting their accuracy and usefulness for evaluation. In addition, technical limitations and challenges with bias can lead to real world harm. To overcome these technical challenges and ethical risks, the evaluation community will need to work collaboratively with the data science community to co‐develop tools and processes and to ensure the application of quality and ethical standards.

Will Artificial Intelligence Affect How Cultural Heritage Will Be Managed in the Future? Responses Generated by Four genAI Models

“…The capacity of genAI models to provide complex responses is not necessarily proportional to the size and diversity of the resources included in the training dataset [8]. The selection of the sources included in a reference dataset results in biased responses, irrespective of whether the reference dataset has been curated by people (as in closed genAI models) (e.g., political bias [9][10][11][12], ethnocentricity [13][14][15], and sexism [16][17][18]), or whether genAI models are also enabled to extract of sources from the internet in general (as the latter overrepresents hegemonic viewpoints at the expense of those of marginalized communities [8,19]). This is not the place to discuss the numerous ethical issues that underlie the selection and use of original sources, such as alleged copyright violations [20,21] or whether primary or secondary sources underpin its 'knowledge' base [5,22].…”

“…Its updated version, ChatGPT (GPT-3.5), was made available to the public in November 2022 in a free research preview to stimulate experimentation [25] which was able to communicate in 95 languages [26]. The design reputedly included a safe answer mode to ensure that ChatGPT responded within ethical human values and did not provide harmful content (e.g., planning attacks, hate speech, and advice on suicide), although this is not assured [19,27,28]. Moreover, such filters can be intentionally subverted [29]).…”

Benchmarking Llama 3 for Chinese News Summation: Accuracy, Cultural Nuance, and Societal Value Alignment

“…However, those traditional metrics may not fully capture the cultural and societal dimensions of text quality, which are important for the Chinese news context [59]. Advanced evaluation approaches have been proposed to address these limitations, incorporating cultural and societal value assessments into the evaluation process [60], [61]. Hybrid metrics that combine linguistic accuracy with cultural relevance have been developed to provide a more comprehensive evaluation of summarization models [62], [63].…”

Generative AI, Research Ethics, and Higher Education Research: Insights from a Scientometric Analysis

“…The existing literature suggests that using large language models for writing research papers raises ethical issues, including technological readiness, privacy, equality, the potential for discrimination and misinformation, intellectual property rights violations, and labor injustices, and the need for updated ethical frameworks to address these concerns [6,27,28].…”

“…Another primary concern is the potential for LLMs to create misinformation and discrimination. LLMs trained on biased datasets have the potential to propagate biased language and harmful beliefs [27]. The manipulation of LLMs to generate fraudulent research papers or change the public's opinion of scientific matters is called into doubt by this.…”