Python OpenCV代写 | 图像处理 | SIFT | CSE327 Homework2

本次python opencv代写要求学生使用opencv进行场景匹配和对象识别,主要使用SIFT进行匹配和识别。主要包括使用SIFT特征匹配变换后的图像、使用SIFT功能进行场景拼接、使用HOG功能进行对象识别这三个task。

Description


In this homework you will experiment with SIFT features for scene matching and object recognition. You will work with the SIFT tutorial and code from the University of Toronto. In the compressed homework file (hw2_files.zip), you will find the tutorial document (tutSIFT04.pdf) and a paper from the International Journal of Computer Vision (ijcv04.pdf) describing SIFT and object recognition. Although the tutorial document assumes matlab implemention, you should still be able to follow the technical details in it. In addition, you are STRONGLY encouraged to read this paper unless you’re already quite familiar with matching and recognition using SIFT.

There are 3 problems in this homework with a total of 100 points. Two bonus questions with extra 5 and 15 points are provided under problem 1 and 2 respectively. The maximum points you may earn from this homework is 100 + 20 = 120 points. Be sure to read Submission Guidelines below. They are important.

Using SIFT in OpenCV 3.x.x


Feature descriptors like SIFT and SURF are no longer included in OpenCV since version 3. This section provides instructions on how to use SIFT for those who use OpenCV 3.x.x. If you are using OpenCV 2.x.x then you are all set, please skip this section. Read this if you are curious about why SIFT is removed https://www.pyimagesearch.com/2015/07/16/where-did-sift-and-surf-go-in-opencv-3/.

One simple way to use the OpenCV in-built function SIFT is to switch back to version 2.x.x, but if you want to keep using OpenCV 3.x.x, do the following:

  1. uninstall your original OpenCV package
  2. install opencv-contrib-python using pip (pip is a Python tool for installing packages written in Python), please find detailed instructions at https://pypi.python.org/pypi/opencv-contrib-python

After you have your OpenCV set up, you should be able to use cv2.xfeatures2d.SIFT_create() to create a SIFT object, whose functions are listed at http://docs.opencv.org/3.0-beta/modules/xfeatures2d/doc/nonfree_features.html

Some Resources


In addition to the tutorial document, the following resources can definitely help you in this homework:

# import packages here
import cv2
import math
import numpy as np
import matplotlib.pyplot as plt

Problem 1: Match transformed images using SIFT features

{40 points + bonus 5} You will transform a given image, and match it back to the original image using SIFT keypoints.

  • Step 1 (5pt). Use the function from SIFT class to detect keypoints from the given image. Plot the image with keypoints scale and orientation overlaid.
  • Step 2 (10pt). Rotate your image counter clockwise by 80 degrees with the cv2.warpAffine function. Extract SIFT keypoints for this rotated image and plot the rotated picture with keypoints scale and orientation overlaid just as in step 1.
  • Step 3 (15pt). Match the SIFT keypoints of the original image and the rotated imag using the knnMatch function in the cv2.BFMatcher class. Discard bad matches using the ratio test proposed by D.Lowe in the SIFT paper. Use 0.1 as the ratio in this homework. Note that this is for display purpose only. Draw the filtered good keypoint matches on the image and display it. The image you draw should have two images side by side with matching lines across them.
  • Step 4 (10pt). Use the RANSAC algorithm to find the affine transformation from the rotated image to the original image. You are not required to implement the RANSAC algorithm yourself, instead you could use the cv2.findHomography function (set the 3rd parameter method to cv2.RANSAC) to compute the transformation matrix. Transform the rotated image back using this matrix and the cv2.warpPerspective function. Display the recovered image.
  • Bonus (5pt). You might have noticed that the rotated image from step 2 is cropped. Rotate the image without any cropping and you will be awarded an extra 5 points.
def drawMatches(img1, kp1, img2, kp2, matches):
    """
    My own implementation of cv2.drawMatches as OpenCV 2.4.9
    does not have this function available but it's supported in
    OpenCV 3.0.0

    This function takes in two images with their associated 
    keypoints, as well as a list of DMatch data structure (matches) 
    that contains which keypoints matched in which images.

    An image will be produced where a montage is shown with
    the first image followed by the second image beside it.

    Keypoints are delineated with circles, while lines are connected
    between matching keypoints.

    img1,img2 - Grayscale images
    kp1,kp2 - Detected list of keypoints through any of the OpenCV keypoint 
              detection algorithms
    matches - A list of matches of corresponding keypoints through any
              OpenCV keypoint matching algorithm
    """

    # Create a new output image that concatenates the two images together
    # (a.k.a) a montage
    rows1 = img1.shape[0]
    cols1 = img1.shape[1]
    rows2 = img2.shape[0]
    cols2 = img2.shape[1]

    # Create the output image
    # The rows of the output are the largest between the two images
    # and the columns are simply the sum of the two together
    # The intent is to make this a colour image, so make this 3 channels
    out = np.zeros((max([rows1,rows2]),cols1+cols2,3), dtype='uint8')

    # Place the first image to the left
    out[:rows1,:cols1] = np.dstack([img1, img1, img1])

    # Place the next image to the right of it
    out[:rows2,cols1:] = np.dstack([img2, img2, img2])

    # For each pair of points we have between both images
    # draw circles, then connect a line between them
    for mat in matches:

        # Get the matching keypoints for each of the images
        img1_idx = mat.queryIdx
        img2_idx = mat.trainIdx

        # x - columns
        # y - rows
        (x1,y1) = kp1[img1_idx].pt
        (x2,y2) = kp2[img2_idx].pt

        # Draw a small circle at both co-ordinates
        # radius 4
        # colour blue
        # thickness = 1
        cv2.circle(out, (int(x1),int(y1)), 4, (255, 0, 0), 1)   
        cv2.circle(out, (int(x2)+cols1,int(y2)), 4, (255, 0, 0), 1)

        # Draw a line in between the two points
        # thickness = 1
        # colour blue
        cv2.line(out, (int(x1),int(y1)), (int(x2)+cols1,int(y2)), (0,255,0), 2)

    # Also return the image if you'd like a copy
    return out

# Read image
img_input = cv2.imread('SourceImages/sift_input.jpg', 0)

# initiate SIFT detector

# find the keypoints and descriptors with SIFT

# Darw keypoints on the image
# ===== This is your first output =====
res1 = 

# rotate image

# find the keypoints and descriptors on the rotated image

# Darw keypoints on the rotated image
# ===== This is your second output =====
res2 = 

# ====== Plot functions, DO NOT CHANGE =====
# Plot result images
plt.figure(figsize=(12,8))
plt.subplot(1, 2, 1)
plt.imshow(res1, 'gray')
plt.title('original img')
plt.axis('off')

plt.subplot(1, 2, 2)
plt.imshow(res2, 'gray')
plt.title('rotated img')
plt.axis('off')
# ==========================================

# compute feature matching

# Apply ratio test
good_matches = [] # Append filtered matches to this list
        
# draw matching results with the given drawMatches function
# ===== This is your third output =====
res3 = 

# ====== Plot functions, DO NOT CHANGE =====
plt.figure(figsize=(12,8))
plt.imshow(res3)
plt.title('matching')
plt.axis('off')
# ==========================================

# estimate similarity transform
if len(good_matches) > 4:
    
    # find perspective transform matrix using RANSAC
    rot, mask = 
    print "Transformation Matrix = \n", rot
    
    # mapping rotataed image back with the calculated rotation matrix
    # ===== This is your fourth output =====
    res4 = 
else:
    print "Not enough matches are found - %d/%d" % (len(good_matches),4)

# ====== Plot functions, DO NOT CHANGE =====
# plot result images
plt.figure(figsize=(12,8))
plt.subplot(1, 2, 1)
plt.imshow(img_input, 'gray')
plt.title('original img')
plt.axis('off')
    
plt.subplot(1, 2, 2)
plt.imshow(res4, 'gray')
plt.title('recovered img')  
plt.axis('off')
# ==========================================

Problem 2: Scene stitching with SIFT features

{30 points + 15 bonus} You will match and align between different views of a scene with SIFT features.

Use cv2.copyMakeBorder function to pad the center image with zeros into a larger size. Hint: the final output image should be of size 1608 × 1312. Extract SIFT features for all images and go through the same procedures as you did in problem 1. Your goal is to find the affine transformation between the two images and then align one of your images to the other using cv2.warpPerspective. Use the cv2.addWeighted function to blend the aligned images and show the stitched result. Examples can be found at http://docs.opencv.org/trunk/d0/d86/tutorial_py_image_arithmetics.html. Use parameters 0.5 and 0.5 for alpha blending.

  • Step 1 (15pt). Compute the transformation from the right image to the center image. Warp the right image with the computed transformation. Stitch the center and right images with alpha blending. Display the SIFT feature matching between the center and right images like you did in problem 1. Display the stitched result (center and right image).
  • Step 2 (15pt) Compute the transformation from the left image to the stitched image from step 1. Warp the left image with the computed transformation. Stich the left and result images from step 1 with alpha blending. Display the SIFT feature matching between the result image from step 1 and the left image like what you did in problem 1. Display the final stitched result (all three images).
  • Bonus (15pt). Instead of using cv2.addWeighted to do the blending, implement Laplacian Pyramids to blend the two aligned images. Tutorials can be found at http://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_pyramids/py_pyramids.html. Display the stitched result (center and right image) and the final stitched result (all three images) with laplacian blending instead of alpha blending.
imgCenter = cv2.imread('SourceImages/stitch_m.png', 0)
imgRight  = cv2.imread('SourceImages/stitch_r.png', 0)
imgLeft   = cv2.imread('SourceImages/stitch_l.png', 0)

# initalize the stitched image as the center image
imgCenter = cv2.copyMakeBorder(imgCenter,200,200,500,500,cv2.BORDER_CONSTANT)

# blend two images
def alpha_blend(img, warped):
    
    # Implement alpha_blending
    
    return blended

def Laplacian_Blending(A, B, mask, num_levels=6):
    # assume mask is float32 [0,1]

    # generate Gaussian pyramid for A,B and mask

    # generate Laplacian Pyramids for A,B and masks

    # Now blend images according to mask in each level

    # now reconstruct

    return blended

def getTransform(img1, img2):
    # compute sift descriptors

    # find all mactches

    # Apply ratio test

    # draw matches
    img_match = # call given drawMatches function   

    # estimate transform matrix using RANSAC

    # find perspective transform matrix using RANSAC
    H, mask = # call cv2.findHomography
    
    return H, img_match

def perspective_warping(imgCenter, imgLeft, imgRight):
    
    # Get homography from right to center
    # ===== img_match1 is your first output =====
    T_R2C, img_match1 = # call getTransform to get the transformation from the right to the center image
    
    # Blend center and right
    # ===== stitched_cr is your second output =====
    stitched_cr = # call alpha_blend
    
    # Get homography from left to stitched center_right
    # ===== img_match2 is your third output =====
    T_L2CR, img_match2 = # call getTransform to get the transformation from the left to stitched_cr
    
    # Blend left and center_right
    # ===== stitched_res is your fourth output =====
    stitched_res = # call alpha_blend
    
    return stitched_res, stitched_cr, img_match1, img_match2

def perspective_warping_laplacian_blending(imgCenter, imgLeft, imgRight):
    
    # Get homography from right to center
    
    # Blend center and right
    # ===== This is your first bonus output =====
    stitched_cr = # call Laplacian_Blending to stitch the center and right image
    
    # Get homography from left to stitched center_right
    
    # Blend left and center_right
    # ===== This is your second bonus output =====
    stitched_res = # call Laplacian_Blending to stitch the stitched_cr and left image
    
    return stitched_res, stitched_cr


# ====== Plot functions, DO NOT CHANGE =====
stitched_res, stitched_cr, img_match1, img_match2 = perspective_warping(imgCenter, imgLeft, imgRight)
stitched_res_lap, stitched_cr_lap = perspective_warping_laplacian_blending(imgCenter, imgLeft, imgRight)
        
plt.figure(figsize=(25,50))
plt.subplot(4, 1, 1)
plt.imshow(img_match1)
plt.title("center and right matches")
plt.axis('off')
plt.subplot(4, 1, 2)
plt.imshow(stitched_cr)
plt.title("center, right: stitched result")
plt.axis('off')
plt.subplot(4, 1, 3)
plt.imshow(img_match2)
plt.title("left and center_right matches")
plt.axis('off')
plt.subplot(4, 1, 4)
plt.imshow(stitched_res)
plt.title("left, center, right: stitched result")
plt.axis('off')
plt.show()

plt.figure(figsize=(25,50))
plt.subplot(2, 1, 1)
plt.imshow(stitched_cr_lap)
plt.title("Bonus, center, right: stitched result")
plt.axis('off')
plt.subplot(2, 1, 2)
plt.imshow(stitched_res_lap)
plt.title("Bonus, left, center, right: stitched result")
plt.axis('off')
# =============================================

Problem 3: Object Recognition with HOG features

{30 points} You will use the histogram of oriented gradients (HOG) to extract features from objects and recognize them.

HOG decomposes an image into multiple cells, computes the direction of the gradients for all pixels in each cell, and creates a histogram of gradient orientation for that cell. Object recognition with HOG is usually done by extracting HOG features from a training set of images, learning a support vector machine (SVM) from those features, and then testing a new image with the SVM to determine the existence of an object.

You can use cv2.HOGDescriptor to extract the HoG feature and cv2.ml.SVM_create for SVMs (and a lot of other algorithms). You can also use Python machine learning packages for SVM, e.g.scikit-learn and for HoG computation, e.g. scikit-image. Please find the OpenCV SVM tutorial at https://www.learnopencv.com/handwritten-digits-classification-an-opencv-c-python-tutorial/.

An image set located under SourceImages/human_vs_cars is provided containing 20 images. You will first train an SVM with the HoG features and then predict the class of an image with the trained SVM. For simplicity, we will be dealing with a binary classification problem with two classes, namely, cars and humans. There are 10 images for each class.

Some of the function names and arguments are provided, you may change them as you see fit.

  • Step 1 (5pts). Load in the images and create a vector of corresponding labels (0 for car and 1 for human). An example label vector should be something like [1,1,1,1,1,0,0,0,0,0]. Shuffle the images randomly and display them in a 2 x 10 grid with figsize = (18, 15).
  • Step 2 (10pts). Extract HoG features from all images. You can use the OpenCV function cv2.HOGDescriptor or hog routine from scikit-image. Display the HoG features for all images in a 2 x 10 grid with figsize = (18, 15).
  • Step 3. Use the first 16 examples from the shuffled dataset as training data on which to train an SVM. The rest 4 are used as test data. Reshape the HoG feature matrix as necessary to feed into the SVM. Train the classifier. DO NOT train with test data. No output is expected from this part.
  • Step 4 (15pts). Perform predictions with your trained SVM on the test data. Output a vector of predictions, a vector of ground truth labels, and prediction accuracy.
import skimage.exposure
from skimage.feature import hog
from sklearn.svm import LinearSVC

# load data
def loadData(file):

    # Implement your loadData(file) here

loadData('SourceImages/human_vs_cars/car_')
loadData('SourceImages/human_vs_cars/human_')

# ===== Display your first graph here =====

# create a vector of labels
# assume labels: car = 0, human = 1

 


程序代写代做C/C++/JAVA/安卓/PYTHON/留学生/PHP/APP开发/MATLAB


本网站支持淘宝 支付宝 微信支付  paypal等等交易。如果不放心可以用淘宝交易!

E-mail: [email protected]  微信:dmxyzl003


如果您使用手机请先保存二维码,微信识别。如果用电脑,直接掏出手机果断扫描。

发表评论