Parallel Context Windows for Large Language Models

Ratner, Nir; Levine, Yoav; Belinkov, Yonatan; Ram, Ori; Inbal, Magar,; Abend, Omri; Karpas, Ehud; Shashua, Amnon; Leyton‐Brown, Kevin; Shoham, Yoav

doi:10.18653/v1/2023.acl-long.352

Cited by 10 publications

(3 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While some formal techniques such as Explicit instruction (providing a clear direction to the LLM to do something) [187], System Specific Instruction (asking a question from the LLM to answer), Formatting with an example (providing a sample question and its answer and asking the LLM to provide an answer in the same manner), Control tokens (use special keywords in the prompt to help the LLM provide an answer while considering special provided criteria) [188] and Interaction and iteration/chaining (interact with model iteratively to reach to a good answer by fine-tuning on each reply) have been presented [79].…”

Section: B Generative Ai Design Cyclementioning

confidence: 99%

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Hadi,

tashi,

Qureshi

et al. 2023

Preprint

View full text Add to dashboard Cite

<p>Within the vast expanse of computerized language processing, a revolutionary entity known as Large Language Models (LLMs) has emerged, wielding immense power in its capacity to comprehend intricate linguistic patterns and conjure coherent and contextually fitting responses. Large language models (LLMs) are a type of artificial intelligence (AI) that have emerged as powerful tools for a wide range of tasks, including natural language processing (NLP), machine translation, and question-answering. This survey paper provides a comprehensive overview of LLMs, including their history, architecture, training methods, applications, and challenges. The paper begins by discussing the fundamental concepts of generative AI and the architecture of generative pre- trained transformers (GPT). It then provides an overview of the history of LLMs, their evolution over time, and the different training methods that have been used to train them. The paper then discusses the wide range of applications of LLMs, including medical, education, finance, and engineering. It also discusses how LLMs are shaping the future of AI and how they can be used to solve real-world problems. The paper then discusses the challenges associated with deploying LLMs in real-world scenarios, including ethical considerations, model biases, interpretability, and computational resource requirements. It also highlights techniques for enhancing the robustness and controllability of LLMs, and addressing bias, fairness, and generation quality issues. Finally, the paper concludes by highlighting the future of LLM research and the challenges that need to be addressed in order to make LLMs more reliable and useful. This survey paper is intended to provide researchers, practitioners, and enthusiasts with a comprehensive understanding of LLMs, their evolution, applications, and challenges. By consolidating the state-of-the-art knowledge in the field, this survey serves as a valuable resource for further advancements in the development and utilization of LLMs for a wide range of real-world applications. The GitHub repo for this project is available at https://github.com/anas-zafar/LLM-Survey</p>

show abstract

Section: B Generative Ai Design Cyclementioning

confidence: 99%

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Hadi,

tashi,

Qureshi

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…However, the study also found that GPT-4 is less proficient in tasks that require complex reasoning or specific domain knowledge, highlighting the limitations of these models [24]. Recent research has addressed various limitations of large language models, including the hand-crafting of task-specific demonstrations [25], the evaluation of code synthesis [26], the cost barrier associated with large models [27], the evaluation protocol for conversational recommendation systems [28], and the context window restriction for off-the-shelf LLMs [29].…”

Section: Foundation Models and Artificial General Intelligence (Agi)mentioning

confidence: 99%

Artificial Intelligence in the 21st Century

Gong

2023

Res Intell Manuf Assem

View full text Add to dashboard Cite

Artificial intelligence (AI) is the most important and interesting technology in the 21st Century due to its vast application. This review focuses on the evolution of AI techniques and their applications in recent decades. Deep learning algorithms/models, represented by Large Language Models (LLMs) have resulted in groundbreaking advancements, indicating that AI is evolving to improve its capacity to interact with and help people in various fields such as finance, medicine, and science research. The potential for research in AI is immense, and there is a need for scientific principles behind AI. Future perspectives on how machines can be developed to work with humans and to be compatible with human values and preferences are also discussed.

show abstract

“…The typical transformer neural network architecture (see Box 1 for a glossary of key terminology) creates meaningful embeddings using the attention mechanism. This architecture consists of encoders, which process input data into a context vector, and decoders, which translate the context vector into the desired output 2–8 . Decoder‐only LLMs, such as OpenAI's ChatGPT, are autoregressive models, indicating that the text‐generation process predicts the next word using all preceding words, and the final outcome of the model is in a form that humans can readily recognize 9,10 .…”

Section: Introductionmentioning

confidence: 99%

Systematic review: The use of large language models as medical chatbots in digestive diseases

Giuffrè,

Kresevic,

You

et al. 2024

Aliment Pharmacol Ther

View full text Add to dashboard Cite

SummaryBackgroundInterest in large language models (LLMs), such as OpenAI's ChatGPT, across multiple specialties has grown as a source of patient‐facing medical advice and provider‐facing clinical decision support. The accuracy of LLM responses for gastroenterology and hepatology‐related questions is unknown.AimsTo evaluate the accuracy and potential safety implications for LLMs for the diagnosis, management and treatment of questions related to gastroenterology and hepatology.MethodsWe conducted a systematic literature search including Cochrane Library, Google Scholar, Ovid Embase, Ovid MEDLINE, PubMed, Scopus and the Web of Science Core Collection to identify relevant articles published from inception until January 28, 2024, using a combination of keywords and controlled vocabulary for LLMs and gastroenterology or hepatology. Accuracy was defined as the percentage of entirely correct answers.ResultsAmong the 1671 reports screened, we identified 33 full‐text articles on using LLMs in gastroenterology and hepatology and included 18 in the final analysis. The accuracy of question‐responding varied across different model versions. For example, accuracy ranged from 6.4% to 45.5% with ChatGPT‐3.5 and was between 40% and 91.4% with ChatGPT‐4. In addition, the absence of standardised methodology and reporting metrics for studies involving LLMs places all the studies at a high risk of bias and does not allow for the generalisation of single‐study results.ConclusionsCurrent general‐purpose LLMs have unacceptably low accuracy on clinical gastroenterology and hepatology tasks, which may lead to adverse patient safety events through incorrect information or triage recommendations, which might overburden healthcare systems or delay necessary care.

show abstract

Parallel Context Windows for Large Language Models

Cited by 10 publications

References 0 publications

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects

Artificial Intelligence in the 21st Century

Systematic review: The use of large language models as medical chatbots in digestive diseases

Contact Info

Product

Resources

About