2023年11月25日

Python代写 | CMPT 456 Assignment 4

本次Python代写是完成Kappa统计信息

CMPT 456 Assignment 4
Question 1 (25 points)
This question is about Kappa statistic. Consider the following judgements by two judges A and B.
A
Yes No
B Yes 320 480
No 550 150
Please calculate the Kappa statistic and explain your calculation. Are the two judges in agreement?
Question 2 (25 points)
This question is about NDCG. Suppose a 5-point scale is used in relevance judgements, and a
ranking of answers to a query is: 4, 3, 1, 0, 3, 2, 1, 4, 0, 2. What is the value of NDCG @ 10?
Question 3 (25 points)
Consider an information need for which there are 4 relevant documents in the collection.
Contrast two systems run on this collection. Their top 10 results are judged for relevance as
follows (the leftmost item is the top ranked search result):
System 1 R N R N N N N N R R
System 2 N R N N R R R N N N
1. What is the MAP of each system? Which has a higher MAP?
2. What is the R-precision of each system? (Does it rank the systems the same as MAP?)
Question 4 (25 points)
This question is about break-even point. Recall that the break-even point is the point where the
precision and the recall on the top-n returned answers are identical. What is the relationship
between the value of F1 and the break-even point? Must there always be a break-even point
between precision and recall? Either mathematically prove that there must be or give a
counterexample.