MEd Assessment & Curriculum

Faculty of Education · CDU-EDU-ASC-900 · MSc · v1.0.0

The psychometrics behind a defensible assessment: item difficulty and upper–lower discrimination with Ebel’s quality bands, internal-consistency reliability by Cronbach’s alpha, the standard error of measurement and the confidence band around a learner’s score, and a modified-Angoff cut score with its standard error across judges.

Lineage

Conquest Designer University, 2026 — authored from the brief and the AERA/APA/NCME Standards; no prior institution is claimed

Final research project

Is the test fair to the learner? Item quality, reliability, measurement error and the cut score

Syllabus

Item analysis: difficulty, the upper–lower discrimination index, revising or discarding items
Reliability: internal consistency, Cronbach’s alpha, what lengthens or shortens it
Measurement error: the SEM and the score band a report should show
Standard setting: the modified Angoff method and the precision of a cut score

Reference

Item difficulty (upper–lower): p = (U + L) ÷ 2n
Discrimination index: D = (U − L) ÷ n; ≥ 0.40 very good, 0.30–0.39 reasonably good, 0.20–0.29 marginal, < 0.20 poor (Ebel & Frisbie 1991)
Cronbach’s alpha: α = k/(k − 1)·(1 − Σσᵢ² ÷ σ_X²)
SEM: σ_X·√(1 − reliability); band = score ± z·SEM
Angoff cut: mean over judges of Σ item probabilities; SE = sd ÷ √judges

Units

Item difficulty and discrimination · Cronbach’s alpha · Standard error of measurement and score band · Modified-Angoff cut score

Role when narrated: You are an assessment specialist advising a curriculum committee. Say what the statistic means for the learner, show the calculation, and say what the committee should change in the test.

Item difficulty and discrimination · item_analysis

Try with the example

How hard is this item, and does it separate stronger from weaker candidates?

Reference: Kelley (1939) upper and lower groups; Ebel & Frisbie (1991) discrimination-index guidelines; AERA/APA/NCME Standards (2014) ch. 4 (item development and review)

Returns

difficulty: p = (U + L) ÷ 2n
discrimination: D = (U − L) ÷ n
quality: very good, reasonably good, marginal or poor

The example

{
  "upperCorrect": 24,
  "lowerCorrect": 10,
  "groupSize": 30
}

Agents: POST /faculty/med-assessment--curriculum/item_analysis with JSON {pan, …inputs} and the x-402-payment header. Manifest.

Cronbach’s alpha · cronbach_alpha

Try with the example

How internally consistent are the items of this test?

Reference: Cronbach, L. J. (1951) Coefficient alpha and the internal structure of tests, Psychometrika 16(3); AERA/APA/NCME Standards (2014) ch. 2

Returns

items: k
sumItemVariances: Σσᵢ²
alpha: Cronbach’s alpha

The example

{
  "itemVariances": [
    0.8,
    1.1,
    0.9,
    1.2,
    0.7,
    1
  ],
  "totalScoreVariance": 18.5
}

Agents: POST /faculty/med-assessment--curriculum/cronbach_alpha with JSON {pan, …inputs} and the x-402-payment header. Manifest.

Standard error of measurement and score band · sem_band

Try with the example

Given the test’s reliability, how precise is this learner’s score, and what band should be reported?

Reference: Classical test theory: SEM = σ·√(1 − r); AERA/APA/NCME Standards (2014) ch. 2 (errors of measurement should be reported)

Returns

sem: Standard error of measurement
lower: Score − z·SEM
upper: Score + z·SEM

The example

{
  "scoreSd": 12,
  "reliability": 0.91,
  "observedScore": 68,
  "band": "95"
}

Agents: POST /faculty/med-assessment--curriculum/sem_band with JSON {pan, …inputs} and the x-402-payment header. Manifest.

Modified-Angoff cut score · angoff_cut

Try with the example

Where do the judges place the pass mark, and how precise is that placement?

Reference: Angoff, W. H. (1971) Scales, norms and equivalent scores, in Educational Measurement (2nd ed.); Cizek & Bunch (2007) Standard Setting, ch. 6

Returns

cutScore: Mean of the judges’ sums (raw score)
cutPercent: Cut score ÷ items × 100
standardError: Sample sd of the sums ÷ √judges

The example

{
  "judgeSums": [
    28.5,
    31,
    29.5,
    33,
    30,
    27.5
  ],
  "items": 50
}

Agents: POST /faculty/med-assessment--curriculum/angoff_cut with JSON {pan, …inputs} and the x-402-payment header. Manifest.