CS 475 Machine Learning: Homework 5
Deep Learning: Programming
In this assignment you will learn to use PyTorch to train classifiers to recognize images of simple drawings using a subset of the Google QuickDraw
Your grade will be based on the performance on held-out test data of 3 models which
you will implement. Additionally, the 3 models will also be used in the Lab section of
Requirements This assignment uses Python 3.7. The Python libraries that you will
need (and are allowed to use) are given to you in the requirements.txt file. Install
them using pip3: pip3 install -r requirements.txt.
The next step is to create a convolutional neural network for multiclass classification,
again using cross-entropy loss. As a reminder, 2-D convolutional neural networks operate
on images rather than vectors. (This means that your model will need to reshape the input
into a 2 dimensional image!) Their key property is that hidden units are formed using local
spatial regions of the input image, with weights that are shared over spatial regions. To see
an animation of this behavior.
In particular, pay attention to the first animation (no padding, no strides) and to the 9th
animation (no padding, with strides). Here, a 3 × 3 convolutional filter is run over the
image, with the image shown in blue and the output map (which is just a collection of
the hidden units) shown in green. You can think of each convolutional filter as a small,
translation-invariant pattern detector.
We’ve provided the variables you should use for this model. The first CNN layer
is a k × k convolution that takes in an image with one channel and outputs an image with c channels (where k is the value of –cnn-n1-kernel and c is the value of
–cnn-n1-channels. The second conv layer is a k2 × k2 convolution that takes in an
image with c channels and outputs an image with 8 channels (where k2 is the value of
–cnn-n2-kernel). The output image has approximately half the height and half the
width because of the stride of 2.
You can see that this model has 3 hyperparameters (k, k2, c) in addition to the learning
rate, optimizer, batch size, and number of training iterations. Use the Lab assignment to
help select a good number of channels for our first CNN layer.
1. Complete the SimpleConvNN model in models.py, by implementing the forward
function. Remember, the output of this model should be logits! (See Figure 2)
You should apply a ReLU activation to the output of the 2 convolutional layers.
Implement this model to feed the output of the first convolution into the second
convolution. Then, you should have an output which is of size [batch size, 8, 8,
8]. To convert this into logits, you should use a max-pooling layer (for example, the
over the channels. This will give us a vector of 8 items, each the max value for it’s
respective channel in the previous output.
2. This model will take a lot more steps to train than the previous model. To know
whether you have trained long enough I recommend to look at the graph of the dev
loss over training steps. Does it seem like it could still be trending upward? That
means you need to train longer! Try to find a setting that at least matches your ff
model on dev data.
3. Save your best CNN model as cnn.torch and the test predictions it makes as
cnn-predictions.txt. Submit both the model file and the predictions file.
1.7 Best Model (20 points)
In this section your goal is to maximize performance. Do whatever you’d like to your
network configurations to maximize validation accuracy. Feel free to vary the optimizer,
mini-batch sizes, the number of layers and/or the number of hidden units / number of
filters per layer; include dropout if you’d like do; etc. You can even go the extra mile with
techniques such as data augmentation, where input images may be randomly cropped
and/or translate and/or blurred and/or rotated etc. We’ve added the scikit-image
package to the requirements.txt file, if you want to
take this approach.
However, there are a couple limitations: You may not add additional training
data, and you must be able to store your model in a .torch file no bigger than 1MB.
Hints. You may want to focus on convolutional networks, since they are especially well
suited for processing images. You may often find yourself in a situation where training is
just too slow (for example where validation accuracy fails to climb after a 5 minute period).
It is up to you to cut experiments off early: if training in a reasonable amount of time
isn’t viable, then you can try to change your network or hyperparameters to help speed
things up. In addition, earlier we repeatedly asked that you vary other parameters as
necessary to maximize performance. There is obviously an enormous number of possible
configurations, involving optimizers, learning rates, mini-batch sizes, etc. Our advice is
to find settings that consistently work well across simple architectures, and to only adjust
these settings if insights based on training and validation curves suggest that you should
do so. In particular, remember that Adam helps you avoid manual learning-rate tuning,
and remember that very small minibatches and very large minibatches will both lead to
slow performance, so striking a balance is important.
1. Complete the BestNN model in models.py. You need to define the features for this
model, as well as the forward function.
2. Train this network. Remember, it’s up to you to ensure that you can train your
model in a reasonable amount of time!
3. We care about your ideas and work on designing this model. Create a README.md
file listing what hyperparameters, network architecture and other methods you tried,
what seems to not work, and what seems to work in general.
4. Save your best model as best.torch and it’s test predictions as best-predictions.txt.
Submit the model file, the predictions file and the README.md
To receive full credit, you need to achieve a test accuracy which indicates that you
have put a sufficient amount of effort into building a good classifier. This should improve
over the best model in previous sections by a reasonable amount.
本网站支持淘宝 支付宝 微信支付 paypal等等交易。如果不放心可以用淘宝交易！
E-mail: [email protected] 微信:itcsdx