Assessment is a Compromise
Cees van der Vleuten, PhD. Photo by Elisabeth Fall.
This year’s visiting Presidential Chair, Dr. Cees van der Vleuten, delivered his keynote address on programmatic assessment. All assessment mechanisms involve necessary trade-offs of reliability, validity, educational impact, acceptability, and cost. Our mission as educators is not to develop an assessment that can be perfect in every way, but rather to make informed decisions about what to compromise on, and when.
Reliability, for example, is somewhat of a myth—no method is perfectly reliable—but we can statistically improve the reliability of our methods by using multiple data points for each method (never just one observation of a learner, but multiple observations, preferably across multiple raters). Even very subject measures can be quite robust (decreasing fears around bias/non-standardization) if we have enough data points for a learner when making a summative judgment.