这是一篇来自美国的关于AR和图像镶嵌技术的计算机视觉代写

ASSIGNMENT DESCRIPTION

Description

Problem Set 3 introduces basic concepts behind Augmented Reality, using the contents that you will learn in modules 3A-3D and 4A-4C: Projective geometry, Corner detection, Perspective imaging, and Homographies, respectively. Additionally, you will also learn how to insert images within images and stitch multiple images together.

Learning Objectives

• Find markers using corner detection and / or pattern recognition.
• Learn how projective geometry can be used to transform a sample image from one plane to another.

• Address the marker recognition problem when there is noise in the scene.
• Implement backwards (reverse) warping.
• Implement Harris corner detection to identify correspondence points for an image with multiple views.

• Address the presence of distortion / noise in an image.

Problem Overview

Methods to be used

In this assignment you are to use methods that work with Feature Correspondence and Corner detection. You will also apply methods that are part of Projective Geometry and Image Warping, however you will have to do these manually using linear algebra concepts.

Rules

You may use image processing functions to find color channels, load images, find edges (such as with Canny). Don’t forget that those have a variety of parameters and you may need to experiment with them. There are certain functions that may not be allowed and are specified in the assignment’s autograder Ed. post. Refer to this problem set’s autograder post for a list of banned function calls.

Refer to this problem set’s autograder post for a list of banned function calls.

Please do not use absolute paths in your submission code. All paths should be relative to the submission directory. Any submissions with absolute paths are in danger of receiving a penalty!

INSTRUCTIONS

Obtaining the Starter Files:

Obtain the starter code from the PS3 github repo.

Programming Instructions

Your main programming task is to complete the api described in the file ps3.py. The driver program experiment.py helps to illustrate the intended use and will output the files needed for the writeup.

Write-up Instructions

Create ps3_report.pdf – a PDF file that shows all your output for the problem set, including images labeled appropriately (by filename, e.g. ps3-1-a-1.png) so it is clear which section they are for and the small number of written responses necessary to answer some of the questions (as indicated). For a guide as to how to showcase your results, please refer to the Latex template for PS3.

How to Submit

Two assignments have been created on Gradescope: one for the report – PS3_report, and the other for the code – PS3_code.

• Report: the report (PDF only) must be submitted to the PS3_report assignment.
• Code: all files must be submitted to the PS3_code assignment. DO NOT upload zipped folders or any sub-folders, please upload each file individually. Drag and drop all files into Gradescope.

Notes

• You can only submit to the autograder 10 times in an hour. You’ll receive a message like “You have exceeded the number of submissions in the last hour. Please wait for 36.0 mins before you submit again.” when you exceed those 10 submissions. You’ll also receive a message “You can submit 8 times in the next 53.0 mins” with each submission so that you may keep track of your submissions.

• If you wish to modify the autograder functions, create a copy of those functions and DO NOT mess with the original function call.

YOU MUST SUBMIT your report and code separately, i.e., two submissions for the code and the report, respectively. Only your last submission before the deadline will be counted for each of the code and the report.

Write-up Instructions

The assignment will be graded out of 100 points. The last submission before the time limit will only be considered. The code portion (autograder) represents 60% of the grade and the report the remaining 40%.

The images included in your report must be generated using experiment.py. This file should be set to be run as is to verify your results. Your report grade will be affected if we cannot reproduce your output images.

The report grade breakdown is shown in the question heading. As for the code grade, you will be able to see it in the console message you receive when submitting.

Assignment Overview

A glass/windshield manufacturer wants to develop an interactive screen that can be used in cars and eyeglasses. They have partnered with a billboard manufacturer in order to render certain marketing products according to each customer’s preferences.

Their goal is to detect four points (markers) currently present in the screen’s field-of-view and insert an image or video in the scene. To help with this task, the advertising company is installing blank billboards with four distinct markers, which determine the area’s intended four corners. The advertising company plans to insert a target image / video into this space.

They have hired you to produce the necessary software to make this happen. They have set up their sensors so that you will receive an image / video feed and a target image / video. They expect an altered image / video that contains the target content rendered in the scene, visible in the screen.

1 MARKER DETECTION IN A SIMULATED SCENE [40]

The first task is to identify the markers for this Augmented Reality exercise. In real practice,markers can be used (in the form of unique pictures) that stand out from the background of an image. Below is an image with four markers.

Notice that they contain a cross section bounded by a circle. The cross-section is useful in that it forms a distinguished corner. In this section you will create a function/set of functions that can the detect these markers, as shown above. You will use the images provided to detect the (x, y) center coordinates of each of these markers in the image. The position should be represented by the center of the marker (where the cross-section is). To approach this problem you should consider using techniques like detecting circles in the image, detecting corners and/or detecting a template.

Code: Complete find_markers(image)

You will use the function mark_location(image, pt) in experiment.py to create a resulting image that highlights the center of each marker and overlays the marker coordinates in the image.

Each marker should present their location similar to this: