Linear Regression and Stability of the Moore–Penrose Pseudoinverse Using Python
This machine learning practical and assessment activity develops an understanding of linear regression, Ordinary Least Squares and the Moore–Penrose pseudoinverse using Python. The work progresses from generating synthetic regression datasets to implementing regression algorithms manually, applying established machine learning libraries, analysing real datasets and evaluating the stability of estimated regression coefficients. The laboratory component begins with the generation of synthetic linear regression data using NumPy, including explanatory variables, random noise and an outcome variable. Students then implement simple linear regression without relying on machine learning libraries, using the least-squares solution to estimate the intercept and slope. The resulting observations and fitted regression line are visualised using Matplotlib. The work is subsequently extended to multiple linear regression, where several independent variables are used and coefficients are first calculated manually before the same problem is solved using Scikit-learn. The laboratory also introduces application of regression to the Scikit-learn Diabetes dataset, including feature and target standardisation, model fitting, prediction, correlation analysis and interpretation of regression coefficients. It also highlights the importance of residual analysis when assessing whether a linear model is appropriate. The associated weekly challenge focuses on the stability of linear regression solutions estimated using the Moore–Penrose pseudoinverse. Using a house-price dataset containing variables such as property size, number of bedrooms, distance from the city centre and property age, students construct the design matrix, standardise features and the response variable, and calculate regression coefficients using the pseudoinverse. Students then investigate model robustness by repeatedly fitting the regression model to random subsamples of different sizes and analysing the mean and standard deviation of each coefficient. Tables, boxplots or error-bar visualisations can be used to compare coefficient variability. The final discussion considers which variables are most influential, which coefficients are most stable, how sample size affects stability and whether coefficient interpretation remains reliable across different samples. The final work is submitted as a single PDF exported from Jupyter Notebook or Google Colab, combining documented Python code, experimental results, plots and written interpretation in a professionally organised notebook. Overview word count: approximately 360 wor
Read Model Answer →