本次澳洲代写是B+大数据Map Reduce编程的一个assignment

## Overview

Write advanced MapReduce programs which give your chance to develop in-depth understanding of

principles when solving complex problems on Hadoop execution platform, and analyze solutions by applying

the knowledge learned in this course to achieve the optimal outcome.

## Learning Outcomes

The key course learning outcomes are:

CLO 1: model and implement efficient big data solutions for various application areas using appropriately

selected algorithms and data structures.

CLO 2: analyse methods and algorithms, to compare and evaluate them with respect to time and space

requirements and make appropriate design choices when solving real-world problems.

CLO 3: motivate and explain trade-offs in big data processing technique design and analysis in written and

oral form.

CLO 4: explain the Big Data Fundamentals, including the evolution of Big Data, the characteristics of Big

Data and the challenges introduced.

CLO 6: apply the novel architectures and platforms introduced for Big data, i.e. Hadoop, MapReduce and

Spark.

## Assessment details

### Task 1 – Count word co-occurrence frequency (10 marks)

Write a MapReduce program that uses pairs approach and outputs the frequency of word pairs.

– Given “(a, b)” and word pair “(b, a)”, they are considered as different word pairs,

– Do not output count the pair of same words, e.g., “(a, a)”,

– The words are considered co-occurred if they are in the same line and the number of words between them <=3.

### Task 2 – Count word pair relative frequency (10 marks)

Write a MapReduce program that uses pairs approach and outputs the relative frequency of word pairs.

– Given “(a, b)” and word pair “(b, a)”, they are considered as different word pairs,

– Do not output count the pair of same words, e.g., “(a, a)”,

– The words are considered co-occurred if they are in the same line and the number of words between them <=3.

### Task 3 – Implement PAM algorithm with a MapReduce Program (20 marks)

The most common realization of k-medoid clustering is the Partitioning Around Medoids (PAM) algorithm

which is described below:

(a) Your program must correct implement PAM. In your code, provide detailed comments to specify

where each step is implemented. For example

//Step 2 start.

…

Block of code;

…

//Step 2 end.

Run your PAM MapReduce program to cluster a point dataset NYTaxiLC1000

(with 1000 points in

longitude and latitude from line 1 to line 1000) where 1 ≤ 𝑘𝑘 ≤ 6. Note the initial medoids are always points

at line 100, 200, 300, 400, 500 and 600 (i.e., 𝑘𝑘 = 1, the initial medoid is point at line 100; 𝑘𝑘 = 2, the initial

medoids are points at line 100, 200; and so on for k=3, 4, 5 and 6).

(b) Visualize the clustering results. The points belonging to the same cluster are with the same color.

The medoid of each cluster is highlighted.

(c) Analyse what is the best setting of 𝑘𝑘 (3 ≤ 𝑘𝑘 ≤ 6) and explain why.

step 1. Initialize: randomly select 𝑘𝑘 of the 𝑛𝑛 data points as the medoids

step 2. Assignment step: Associate each data point to the closest medoid.

step 3. Update step: For each medoid m and each data point 𝑜𝑜 associated to 𝑚𝑚 , swap 𝑚𝑚 and 𝑜𝑜, and compute the

total cost of the configuration (that is, the average dissimilarity of 𝑜𝑜 to all the data points associated

to 𝑚𝑚). Select the medoid o with the lowest cost of the configuration.

step 4. Repeat alternating steps 2 and 3 until there is no change in the assignments.

**程序代写代做C/C++/JAVA/安卓/PYTHON/留学生/PHP/APP开发/MATLAB**

本网站支持淘宝 支付宝 微信支付 paypal等等交易。如果不放心可以用淘宝交易！

**E-mail:** itcsdx@outlook.com **微信:**itcsdx

如果您使用手机请先保存二维码，微信识别。如果用电脑，直接掏出手机果断扫描。