This article provides brief discussions of an eclectic set of problems, pitfalls, and paradoxes that challenge measurement theory and practice, especially for K–12 achievement testing. The technical issues considered are: true scores and error scores, fixed versus random facets, lower limits to reliability, the reliability‐validity paradox, reliability of difference scores, equating and linking, and scaling. The practical issues considered are: validation, standard setting, instructional improvement and high‐stakes accountability, multiple measures, adequate yearly progress, the National Assessment of Educational Progress as a confirmation model, and capacity.