Assessing Fine-Tuning Efficacy in LLMs: A Case Study with Learning Guidance Chatbots

Abstract

Training and accurately evaluating task- specific chatbots is an important research area for Large Language Models (LLMs). These models can be developed for general purposes with the ability to handle multiple tasks, or fine-tuned for specific applications such as education or customer support. In this study, Mistral 7B, Llama-2 and Phi-2 models are utilized which have proven success on various benchmarks, including question answering. The models were fine-tuned using QLoRa with limited information gathered from course catalogs. The fine-tuned models were evaluated using various metrics, with the responses from GPT-4 taken as the ground truth. The experiments revealed that Phi-2 slightly outperformed Mistral 7B, achieving scores of 0.012 BLEU, 0.184 METEOR, and 0.873 BERT. Considering the evaluation metrics obtained, the strengths and weaknesses of known LLM models, the amount of data required for fine-tuning, and the effect of the fine-tuning method on model performance are discussed.