This individual assessment for the Research Methods module focuses on the application of Large Language Models (LLMs) to a practical data science problem. The assignment is worth 25% of the module and is designed to develop students’ knowledge and understanding of research methods, investigative planning, data analysis, model evaluation and effective technical communication. Students are required to complete both a coding component and a concise written report demonstrating how an LLM has been selected, trained or fine-tuned, applied to a suitable task and evaluated against an appropriate baseline. The assessment begins with familiarisation with relevant literature. Students are expected to investigate the history and development of their chosen problem and examine the methods that have previously been used to address it. The brief provides key papers on common types of LLMs as a starting point for the literature review. Students then select a task that can be addressed through fine-tuning an LLM, with examples including sentiment analysis, fake news detection and topic classification. A publicly available dataset suitable for the selected text-classification problem must also be identified. Suggested sources include Kaggle and Hugging Face Datasets. The data must be appropriately preprocessed, including tokenisation using BERT's tokenizer and division into training and testing sets. Students then fine-tune a pre-trained BERT or BERT-style model using suitable tools such as the Hugging Face Transformers library and PyTorch. Possible model choices include BERT, RoBERTa and T5. The selected model should be appropriate for the specific problem, recognising that different language models may perform differently across tasks. Students are expected to implement a suitable training process using an appropriate optimiser and loss function. Model performance must be evaluated using relevant classification metrics, including accuracy, precision, recall and F1-score. The performance of the selected LLM should also be compared with a baseline model, such as Logistic Regression, Naive Bayes or a pre-trained BERT model. The analysis should explain the results and consider their relevance to the chosen problem. Two main submission components are required. The first is a code notebook, such as a Jupyter or Google Colab notebook, containing annotations explaining the purpose and operation of the relevant code so that another person can understand and reproduce the work. The second is a report of no more than three pages, including appropriate figures, tables and references. The report should cover the motivation and dataset, methodology, model training and evaluation, results and discussion, limitations, conclusion and possible future improvements. The assessment rubric places particular emphasis on coding quality and implementation, model architecture, analysis and interpretation, and report presentation. Strong work should demonstrate well-structured and reusable code, clear explanation of the model architecture and configuration, appropriate evaluation metrics and visualisations, meaningful comparison with relevant literature or baseline models, and critical evaluation of the model's success and possible improvements.
Large Language Models · LLMs · BERT · RoBERTa · T5 · Natural Language Processing · Text Classification · Fine-Tuning · Transfer Learning · Transformers · Hugging Face · PyTorch
Megaminds has supported academic requirements in research methods, research methods and related disciplines.