Why Annotation Accuracy Is Misleading
EngineeringAnnotation accuracy measures the samples that were labelled, not model behaviour. Drawing on public research, this article explains why a dataset of positives alone cannot define a model's boundary, why annotator disagreement is signal rather than noise, why evaluation sets saturate with age, and what each of the three PEN data categories — Positive, Edge, Negative — solves.
2026-10-01