2023年11月25日

计算机视觉代写｜EECS 442 Problem Set 10: Optical Flow and Epipolar Geometry

本次美国代写是一个计算机视觉的Problem Set

The starter code can be found at:

https://colab.research.google.com/drive/1IxO7g27c31rZqtwG8HuWvHU_3goho6ZY?usp=
sharing

We recommend editing and running your code in Google Colab, although you are welcome to
use your local machine instead.

Problem 10.1 Optical flow
Algorithm 1 PWC-Net(I1; I2)

1: Build 7-level feature pyramids c1; c2 from I1; I2.
2: Compute a cost volume using cl 1, cl 2 and then estimate the optical flow wl at the coarsest
pyramid level l = 6.
3: for level l = 5,4,3,2, do
4: Warp the feature map cl 2 to obtain cl w, using upsampled coarse flow from wl+1.
5: Compute a cost volume using cl 1 and cl w, according to Equation (2).
6: Estimate the optical flow wl.
7: end for
8: Refine the final optical flow w2 using a context network.
9: return w2

In this problem, you will complete an implementation of PWC-Net [1], a neural network that
estimates the optical flow estimation between a pair of images I1 and I2. Your solution will
only require approximately one line of code! There are three key components in the PWC-Net,
namely, Pyramid, Warping and Cost-volume, which will be described in more detail below.
Fig. 1 shows an overview of the network structure. You should refer to the PWCNet in the
provided notebook for further implementation details.

Computing optical flow with PWC-Net. The way that PWC-Net computes optical
flow closely resembles the \classic” multi-scale optical flow pipeline we discussed in class.

However, instead of matching images by directly comparing pixel intensities, it uses learned
CNN features (Algorithm 1).

PWC-Net matches the images in a multi-level, coarse-to-fine manner. It extracts multi-level
feature pyramids from both images using fully convolutional networks and then computes
a cost volume at each level. This is a volume C(x; y; ∆u; ∆v) that measures the similarity
between the features at position (x; y) in I1 and the feature (x + ∆u; y + ∆v) in I2, and is
computed by taking the dot product between the features in the two feature maps. We then
process the cost volume using multiple convolution layers to estimate the flow at that level.

We’ll repeat this procedure at multiple levels. After estimating a flow wl+1 = (ul+1; vl+1)
at level l + 1, when we repeat our search in subsequent levels, we’ll recenter our search on
these new flow locations, i.e. we’ll search for ∆u and ∆v only within a small window around
(x + ul+1; vl+1). For simplicity and speed, we do this by warping the features in the second
image, moving the matched features at position (x + ul+1; y + vl+1) to (x; y). This allows us
to simply write a function that simply performs the match in a small window around each
feature.

As in classic flow methods, we’ll perform this iterative computation in conjunction with a
pyramid. We’ll compute optical flow at coarse levels of the pyramid first, and use our flow
estimates to initialize finer levels of the pyramid. We’ll use different layers of a CNN as our
feature pyramid representation.

PWC-Net components. While we’ve given you a large system to implement, you’ll only
need to implement two pieces of it: setting up the flow with right scale for warping at each
level, and computing the cost volume. For reference, we provide more information about the
important components of PWC-Net below: