机器学习代写｜Fundamentals of Machine Learning Exercise 6
This exercise is dedicated to regularized regression.
Please hand in your solution as a Jupyter notebook regularized-regression.ipynb, accompanied
with exported regularized-regression.html. Zip all les into a single archive ex06.zip and
upload this le to your assigned tutor on MaMPF before the given deadline.
Note: Each team creates only a single upload, and all team members must join it as described in
the MaMPF documentation at https://mampf.blog/zettelabgaben-fur-studierende/.
Important: Make sure that your MaMPF name is the same as your name on Muesli.
We now identify submissions purely from the MaMPF name. If we are unable to
identify your submission you will not receive points for the exercise!
1 Bias and variance of ridge regression (8 points)
Ridge regression solves the regularized least squares problem
with regularization parameter τ ≥ 0. Regularization introduces some bias into the solution in order
to achieve a potentially large gain in variance. Assume that the true model is y = Xβ∗+ with zero
mean Gaussian noise ∼ N(0; σ2) and centered features N1 Pi Xi = 0 (note that these assumptions
imply that y is also centered in expectation). Prove that expectation and covariance matrix of the
regularized solution (both taken over all possible training sets of size N) are then given by
where S and Sτ are the ordinary and regularized scatter matrices:
Notice that expectation and covariance reduce to the corresponding expressions of ordinary least
squares (as derived in the lecture) when τ = 0:
Since Sτ is greater than S (in any norm), regularization has a shrinking eect on both expectation
2 Denoising of a CT image (11 points)
In the last task of exercise 5, we diminished the radiation dose for our patient by using fewer
projection angles. However, this introduced noise in the resulting tomogram. Ridge regression oers
a simple possibility to reduce the noise level1.
To apply regularization to tomographic reconstruction, the D × D diagonal matrix pτ ID must be
appended at the bottom of the weight matrix X from exercise 5:
Correspondingly, vector y must be extended with D zeros.
Add a new parameter to the function construct_X() to activate regularization
X = construct_X(M, alphas, Np = None, tau=0)
For tau=0, the function shall behave exactly like the original version from exercise 5. If tau>0,
it shall return the augmented matrix X0. Reconstruct the tomogram for 64 angles with τ =
0; 1; 10; 100; 1000; 10000 and display the resulting CT images. Find the value of τ with the best
compromise of image sharpness and noise.
Compare the ridge regression results with Gaussian ltering, another popular noise reduction techni
que, which replaces each pixel with a weighted average of its neighbors (see https://en.wikipedia.
org/wiki/Gaussian_filter for more details). A suitable implementation is provided by function
gaussian_filter in module scipy.ndimage.filters. Apply gaussian_filter with lter sizes
sigma = 1,2,3,5,7 to the CT image with τ = 0, display the results and comment on the simila
rities and dierences between ridge regression and Gaussian ltering.