Clinical Versus Statistical Prediction: What 60 Years of Research Tells Us About Human Judgment

In 1954, psychologist Paul Meehl published a book called "Clinical Versus Statistical Prediction," which examined a specific question: when an expert reviews a set of information about a person and makes a holistic prediction, how does their accuracy compare to a simple statistical formula applied to the same information?

Meehl found, to the considerable discomfort of his profession, that the statistical formula performed at least as well as expert judgment in most cases, and often better. The experts were not incompetent. They were exercising genuine knowledge and professional skill. But the statistical model, by weighting and combining predictors consistently, outperformed the variable, impression-influenced processing of the same information by trained practitioners.

Grove et al.'s meta-analysis

In 2000, William Grove, David Zald, Boyd Lebow, Beth Snitz and Chad Nelson published a meta-analysis in Psychological Assessment that synthesised evidence from 136 studies comparing clinical and mechanical (statistical or actuarial) prediction across a wide range of contexts, including medical diagnosis, psychological assessment, academic performance, and various professional judgment tasks.

Their findings were striking in their consistency. Mechanical prediction was more accurate than clinical judgment in approximately 47 per cent of studies. The two methods performed approximately equally in 48 per cent of studies. Clinical judgment outperformed mechanical prediction in only 6 per cent of studies, and in many of those cases the clinician had access to information that the model did not.

The conclusion was clear: when the same information is available to both, a consistent, rule-based combination of predictors outperforms holistic expert judgment more often than not, and rarely underperforms it.

Why expert judgment loses to statistical models

The research has identified several mechanisms by which holistic expert judgment becomes less accurate than simple statistical combination. Experts weight information inconsistently: the same signal receives different weight at different points in the assessment, depending on what else the expert has recently reviewed. Experts are influenced by information that is vivid or memorable, regardless of its actual predictive validity. And experts engage in complex, interactive reasoning that can override the straightforward signal from the most predictive variables.

Paul Meehl himself described this phenomenon as "the clinician as a broken computer": attempting to do something statistical, but doing it inconsistently and with numerous systematic errors.

The limits of the research and what it means for admissions

The clinical-statistical prediction literature has limitations. Many of the studies involve relatively simple prediction tasks where the outcome is clearly defined and the predictors are well-established. School admissions involves more complex, multidimensional judgments about fit and potential, and the outcome, thriving in a specific educational environment, is harder to define and measure than recidivism or hospital readmission.

Nevertheless, the research points toward a clear implication for admissions practice. The academic comparison, the question of whether this student's performance is above or below the threshold for their entry grade, is exactly the kind of simple, rule-based judgment that statistical combination handles reliably and that human judgment handles inconsistently. Automating that judgment does not compromise quality. The research suggests it improves it.

What remains for human judgment is the more genuinely complex question: given the evidence, and given everything the assessor knows about the school and its programme, is this the right student for this place? That is a question worth preserving for experienced professional judgment. The underlying comparison does not need to be.

Sources: Meehl, P. E. (1954). Clinical versus statistical prediction: A theoretical analysis and a review of the evidence. University of Minnesota Press. Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., and Nelson, C. (2000). Clinical versus mechanical prediction: A meta-analysis. Psychological Assessment, 12(1), 19-30.

See how Evalent structures the academic comparison to support, not replace, professional admissions judgment.