Big Data Analytics / Machine Learning
3,000 words
Machine Learning on Big Data Using PySpark: Large-Scale Data Analysis and Predictive Modelling
This group-based Machine Learning on Big Data project requires students to apply machine learning techniques to a large real-world dataset using PySpark DataFrames and Spark machine-learning libraries. Students select a substantial dataset, ideally between approximately 300 MB and 1 GB, from sources such as Kaggle, workplace data or other valid repositories, and develop an end-to-end big-data analytics workflow. CN7030 CRWK 26T1 The project begins with data loading and preprocessing using PySpark. Students are expected to handle missing values, perform data normalisation and feature engineering, identify class imbalance and propose appropriate mitigation strategies. Where text datasets are selected, additional preprocessing may include stemming, lemmatization and TF-IDF representation. CN7030 CRWK 26T1 The modelling stage requires implementation of an appropriate machine-learning approach using PySpark MLlib or Spark ML. The brief expects a multiclass rather than binary classification problem and allows techniques including multiclass classification, ensemble learning, clustering and text mining. Students must justify their model choice and consider model robustness, bias and variance when attempting to improve predictive performance. CN7030 CRWK 26T1 Students then perform hyperparameter tuning using techniques such as grid search or random search and evaluate the resulting model with appropriate measures. Relevant evaluation outputs may include accuracy, F1-score, precision, recall and a confusion matrix. Results should also be visualised or clearly presented and interpreted to identify meaningful patterns and performance characteristics. CN7030 CRWK 26T1 The project additionally requires consideration of Legal, Social, Ethical and Professional (LSEP) issues. Students discuss potential ethical concerns associated with their dataset, including bias and privacy risks, and propose suitable mitigation strategies. The final work is consolidated into a single user-friendly HTML analytics report that clearly presents the group's preprocessing, modelling, optimisation, evaluation and interpretation. CN7030 CRWK 26T1 CN7030 CRWK 26T1 Overview word count: approximately 335 words. If you are also uploading the presentation separately to the Reference Library, that should be a second entry under “Presentations and Academic Posters”, because the presentation forms a distinct 40% component and assesses understanding of Spark, preprocessing, modelling, optimisation, evaluation and responses to examiner questions. CN7030 CRWK 26T1
Read Model Answer →
Machine Learning and Big Data
2,000 words
Machine Learning for Big Data
This assignment for the Machine Learning and Big Data module requires students to produce a 2,000-word individual report demonstrating their understanding and practical application of machine learning techniques to big data. The assessment is worth 15 credits and is structured around five interconnected areas: data in big data, machine learning architecture, model deployment, model evaluation, and the machine learning lifecycle. The first part focuses on identifying and evaluating suitable datasets for a selected big data topic and determining whether the datasets are appropriate for the intended machine learning application. Students are expected to examine the characteristics of their data and apply appropriate pre-processing approaches, including consideration of attribute selection and data preparation. The second part addresses machine learning modelling architecture. Students must develop an appropriate architecture for their big data application and may compare alternative architectural approaches. The report should explain the selected machine learning techniques and demonstrate how they operate as part of the proposed system. Practical considerations such as performance, scalability, fault tolerance, technology usage and reliability should also be considered. The third part requires students to implement and deploy the proposed data and machine learning model. This includes testing, visualising and evaluating the resulting outcomes. The fourth part requires critical evaluation of the dataset selection, modelling design, implementation and application, including assessment of whether the selected machine learning techniques are appropriate for the intended purpose. The final part focuses on the complete project lifecycle. Students are expected to critically reflect on the work undertaken, identify what they have learned, evaluate the development process and explain how the machine learning application could be improved in a future implementation. The assessment develops five learning outcomes covering big data sources and applications, machine learning techniques, practical application of machine learning tools, critical evaluation of techniques and tools, and the ability to follow a complete big data analysis lifecycle. The marking criteria allocate 20% to each of these five areas. The assignment is submitted as an individual written report. The brief states that Microsoft Word should be used rather than PDF and requires students to acknowledge sources and any AI tools used in accordance with the stated AI policy.
Read Model Answer →