# 人工智能代写｜CSCI 5512: Artificial Intelligence II Homework 4

## Instructions

Follow the rules strictly. If we cannot run your code, you will not get any credit.

Things to submit

1.hw4 sol.pdf: A document which contains solutions to all problems. This document must be in PDF format (doc, jpg, etc. formats are not accepted). If you submit a scanned copy of a hand-written document, make sure the copy is clearly readable, otherwise no credit may be given.

2.Python code for Problems 1,2, 3, and extra credit problem 5.

3.README.txt: README fifile that contains your name, student ID, email, assumptions you are making, other students you discussed the homework with, and other necessary details.

4.Any other fifiles, except the data, which are necessary for your code (such as package dependencies like a requirements.txt or yml fifile).

1.(40points)The perceptron learning rule learns a linear model for classiﬁcation problems by iteratively updating a weight vector via:

wt+1 = wt + αt(yit  – ✶(wt(工)xit   > 0))xit

where t indexes the iteration, it  is the data point used in iteration t (i.e., for N total data points it  e {1, . . . , N}), yit    e {0, 1} is the label, xit    e Rd  is the feature vector, αt  e R is the learning rate, and  ✶(x) is the indicator function which returns 1 if the condition x is true and 0 otherwise. Implement the perceptron algorithm (from scratch) with αt = 1/,t in ﬁle hw4 q1.py.  Randomly iterate through all data points for 100,000 iterations.  Using the scikit-learn wine dataset1  with 80% of the data as training data and the remaining 20% as test data and only considering classes 0 and 1, apply your code to the dataset and compute the test set classiﬁcation error.  Do this for 10 repetitions, each time starting the algorithm from scratch and computing the classiﬁcation error.  Report the average classiﬁcation error and standard deviation across the 10 repetitions on the test set. Your code must print these numbers out via the terminal; also report them in the hw4 writeup pdf. An example of how to call your code is: python  hw4 q1.py.

2.(40 points) The k-nearest neighbor (knn) algorithm predicts the class of a test data point by computing the distance to all neighbors in the training set, selecting the k neighbors with smallest distance (i.e., the nearest neighbors), and predicting the class via majority vote among the neighbors’ class labels. Implement the knn algorithm (from scratch) in fifile hw4 q2.py and use 5-fold cross validation to select the optimal value of k = {23, 51, 101}. Use the Euclidean distance to compute the distance between data points

(i.e., |xa – xb|2  = Apply your code to the scikit-learn wine dataset (same as in problem 1). Report the average classifification error on the test folds for each value of k. (In other words, for a given k value, compute the classifification error on each of the 5 test folds, then average these values. Repeat for each value of k.) Your code must print these numbers out via the terminal; also report them in the hw4 writeup pdf. Explain which value of k you would choose to build your fifinal model. An example of how to call your code is: python hw4 q2.py. You can use the cross validation2 function numpy.model selection.KFold() and Euclidean distance3 function numpy.linalg.norm() to compute the cross validation splits and distances.

3.  (30points)The least squares linear regression algorithm learns a linear function f (x) = wx which minimizes the squared error on the training data. Often we want to also minimize the complexity of the linear function (e.g., the magnitude of w) using L2 regularization to avoid overﬁtting. The closed-form solution for this regularized linear regression problem is where w is the weight vector, y is the target value vector of length N (for N data ponts),X is the design matrix of size N × d (recall, we stack the N data points, each which has length d, into a matrix to construct X), I is the identity matrix of size d × d (i.e., a matrix with d rows and columns where the values on the diagonal are all 1s and the non diagonal values are 0s), and λ e R is the regularization parameter. Implement the least squares linear regression algorithm (from scratch) with L2 regularization in ﬁle hw4 q3.py using the solution above.  Apply your code to the scikit-learn diabetes dataset4  and compute the average root mean squared error (RMSE) on the test folds for each value of λ.  For a test fold with Ntest datapoints, the RMSE is computed via Use 5-fold cross validation to select the optimal value of λ  e {0.1, 1, 10, 100} and explain which value you would choose to build your ﬁnal model.   Your code must print out the RMSE for each value of λ via the terminal; also report them in the hw4 writeup pdf.  An example of how to call your code is:  python  hw4 q3.py.  You may want to use mathemat- ical numpy functions such as numpy.dot(),  numpy.transpose(),  numpy.linalg.inv(), numpy.eye(),  numpy.linalg.norm(), etc.

Extra Credit. In order for extra credit solutions to be considered,  you must provide  rea- sonable solutions to all parts  above.  If youskip any part of a problem or do not provide  a reasonable solution  (i.e., make a real eﬀort), we will not count any extra credit towards your grade.

4.ExtraCredit  (5 points) In problem 3, we considered the least squares linear regression problem with L2  regularization and the solution for the optimal w vector was given. Derive this optimal w vector by hand.  For full credit, you must show all steps of your derivation. Hint, start with the optimization problem

w* = argminweRd  |y – Xw|2(2) + λ|w|2(2) .

5.Extra Credit (15 points) In this problem, we will consider classifying the wine dataset (same as above) using the scikit-learn5 machine learning library. (You do not need to imple-ment the algorithms from scratch, use scikit-learn instead). Using the following algorithms,learn a model to classify the wine dataset:

• Logistic regression (set max iters = 3000),
• Linear support vector machine (SVM) (set kernel=‘linear’),
• Random forest (RF) (set criterion=‘entropy’, max depth=1),
• For each algorithm, use 5-fold cross validation (you may use the scikit-learn CV function6 for
• this problem) to tune the following hyperparameters:
• Logistic regression7: C [1e 5, 1e 4, 1e 3, 1e 2, 0.1, 1, 10, 100, 1000],
• SVM8: C [1e 5, 1e 4, 1e 3, 1e 2, 0.1, 1, 10, 100, 1000],
• RF9: n estimators [1, 10, 20, 30, 40, 50, 100, 200],
• Adaboost10: n estimators [1, 10, 20, 30, 40, 50, 100, 200].

For each algorithm and hyperparameter, plot the mean classifification error rate and standard deviation (as error bars) across the 5 test folds (you may need to use a log scale for the x-axis). For each algorithm, choose the ‘best’ hyperparameter and explain your choice. Submit a single python fifile named hw4 ec.py which takes no arguments and runs and saves the plots for each algorithm in the current working directory. Make sure to also include these plots in your hw4 writeup pdf. E-mail: itcsdx@outlook.com  微信:itcsdx 