Michael Lavine

Assessing Procedures vs Assessing Evidence

Many statistical analyses are characterized by how often a procedure works: how often an interval covers a true value, a null hypothesis is rejected, an item is correctly classified, etc. But assessing how often a procedure works differs from assessing the evidence in a data set. Understanding the difference is prerequisite to understanding what matters in a given analysis: the procedure, the evidence, or both.



    We begin with several examples that illustrate the difference between assessing evidence and assessing procedures. Then we work carefully through another example to motivate the Conditionality Principle (CP) and the Sufficiency Principle (SP), and show how CP and SP together imply that assessing evidence differs from assessing procedures.