Cloud and Big Data Technologies
2,500 words
Cloud and Big Data Technologies – Summative Assessment: Cloud and Big Data Security Application
This summative assessment for the CST4067 Cloud and Big Data Technologies module requires students to design, implement and evaluate an individual cloud and big data security application. The assessment focuses on applying techniques for the complex transformation and processing of data within distributed and cloud-based environments, while considering security, privacy and access-control requirements. Students are expected to develop a practical application and document its design, implementation and evaluation through a technical report and a short demonstration video. Students are given flexibility to select their own project idea, provided that the proposed application is appropriately scoped for the available development period and demonstrates relevant cloud, big data and security technologies. Suggested project areas include data science applications using SQL, MongoDB and cloud storage, privacy-preserving data processing such as Differential Privacy, multi-party authentication, group-based security and access control, Multi-Level Security, Attribute-Based Encryption, and distributed or cloud-based applications incorporating security protocols such as SSH, SSL or IPSEC and access-control mechanisms such as RBAC. The Design and Implementation Document should be no longer than 2,500 words and should explain the major design and implementation aspects of the project. Expected content includes an introduction covering the aims, objectives, project concept, security concepts and cloud technologies used; a requirements specification addressing programme behaviour and security requirements; analysis and design covering protocols, access control and interaction, sequence diagrams or process specifications; implementation details explaining what was achieved and how it was developed; and an evaluation and conclusion discussing successful and unsuccessful aspects, problems encountered and lessons learned. Relevant references, including tutorials, books and academic articles, should also be provided using Harvard or IEEE referencing. The assessment also requires students to submit the implemented Cloud and Big Data Security application together with a highlight demonstration video. The video must be no longer than seven minutes and should demonstrate the main features of the application, including relevant interactions, implementation highlights, security features and, where appropriate, attack scenarios. Assessment is based on the Design and Implementation Document, originality, advanced features, and the effort and quality demonstrated in the application. The assessment specification places particular importance on original development, clear documentation of any tutorials or existing resources used, and evidence that the student understands the technologies implemented. Suggested technologies and project ideas include Google Cloud, Hadoop, Spark, cloud storage, data pipelines, security protocols, access control and privacy-preserving techniques.
Read Model Answer →
Data Warehousing and Big Data
Data Warehousing and Big Data – Inventory Management Data Warehouse
This individual coursework for CS7079 Data Warehousing and Big Data requires students to design, implement and test a data warehouse based on a business case scenario and then export and migrate data to a Big Data platform for further processing. The assessment focuses on the inventory management business process of ABC Consumer Electronics Outlet Ltd, a multi-channel consumer electronics retailer operating from six stores around London and conducting online business across the UK and Europe. The company manages more than 10,000 products across approximately ten categories and more than 200 brands. The company already uses cloud-based and business applications including Vend, Linnworks and Xerox, which generate large volumes of transactional data. However, these datasets are stored separately by the individual applications, making it difficult for managers to produce integrated reports and perform analysis to support business decision-making. A business analyst has therefore recommended the development of a data warehouse capable of meeting the company's reporting and analytical requirements. For this coursework, the solution is focused specifically on the inventory management business process. The inventory management case involves three main business activities: sending purchase orders to suppliers when stock reaches minimum levels, receiving purchase orders and storing stock in appropriate locations, and controlling and maintaining stock levels including adding new products and adjusting existing stock. The required analysis includes daily stock levels for the previous month, weekly identification of products at minimum stock levels, analysis of stock levels by brand, product type and supplier, daily and weekly sent and received stock orders, and analysis of received stock orders by supplier and month. Students must analyse and design a dimensional data model that represents these business activities. This includes defining the grain of three central fact tables, identifying appropriate dimensions, defining dimension attributes and fact measures, and producing simple star schemas showing the relationships between fact tables and dimensions. The design must be justified according to the available data sources and the reporting and analysis requirements. The implementation stage requires students to create the relational database using Microsoft SQL Server Management Studio. Students must create the database, dimension tables and fact tables, establish appropriate primary-key and foreign-key constraints, and demonstrate implementation and testing through SQL commands and their results. Test data must then be populated into the data warehouse. The Big Data component requires migration of test data from the data warehouse to an Apache Hadoop platform using the Hortonworks Data Platform. Students must export the data warehouse data to an external data file, migrate the file into Apache HDFS, create a suitable data structure for loading the data into Hive, and demonstrate Apache Pig for manipulating the loaded data. Implementation and testing of the Big Data storage environment must also be demonstrated through commands and results. The coursework concludes with a personal reflective section in which students discuss what they have learned throughout the overall coursework and the challenges encountered during the process. The final submission is expected to be a well-written, structured and well-presented report combining data warehouse design, database implementation, testing, Big Data migration and reflective learning.
Read Model Answer →