Administration
Assessment of Learning
ITEM ANALYSIS, DISCRIMINATION, AND POINT BISERIAL
Robert B. Supernaw 2001
ITEM ANALYSIS
Item difficulty = pi = 0 to 1.0 (higher the easier, greater percentage get the item correct).
Objective: Upon completion of the lecture, the student will be able to identify the two principal opioid receptors of clinical significance.
Question #1: The two principal opioid receptors of clinical significance are the _________ and the _________ receptors.
- Question #2: Which of the following pairs are the two principal opioid receptors of clinical significance?
-
- alpha and beta
- delta and kappa
- kappa and mu
- mu and gamma
- alpha and delta
What does a pi = 0.60 for Question #1 mean? (60% of the students got it right, but less than half actually knew the correct answer; see below as to why).
What does a pi = 0.60 for Question #2 mean?
| Number of choices | Percentage who know answer | Percentage who guess answer | pi | Lord's pi |
|---|---|---|---|---|
| 5 | 0.5 | 0.5 / 5 | 0.5 + (0.5 / 5) = 0.60 | 0.72 |
| 4 | 0.5 | 0.5 / 4 | 0.5 + (0.5 / 4) = 0.62 | 0.74 |
| 3 | 0.5 | 0.5 / 3 | 0.5 + (0.5 / 3) = 0.67 | 0.77 |
| 2 | 0.5 | 0.5 / 2 | 0.5 + (0.5 / 2) = 0.75 | 0.85 |
Important note: Reliability is increased by using items with pi values enhanced by co educated guessing.
Discrimination - the ability of an item to differentiate prescribed performance levels from substandard performance levels. These are items which are answered correctly by students who have mastered the ability in question but are answered incorrectly by students who have not mastered the ability in question.
Point Biserial - The degree to which a question discriminates between a good student and a poor student. A correlation between the students who got an item correct, and the students who did well on the rest of the test. For point biserials, we are very happy when we get a value of 0.2 or higher (up to a possible, perfect correlation of 1.0). A negative point biserial indicates that poor students got the question right to a greater extent than good students did. We can calculate BPS for each answer (the right answer and the 3 wrong answers). Ideally, we hope to have a positive BPS of 0.2 for the correct answer and negative BPSs for the wrong answers (meaning that the poor students chose the wrong answers to a greater extent than the good students).