Abstract

In psychology’s crisis of confidence, issues related to measurement did not receive much attention. However, ignoring challenges of measurement—one of the foundations of science—has serious consequences for the validity of the inference, replicability, and reproducibility of research. In our symposium, we highlight empirically studied measurement issues and present steps researchers can take to prevent them. Eiko Fried discusses the limited progress of research efforts into causes and treatments of depression, and argues that this lack of progress is in part due to lack of attention to the topic of measurement in clinical sciences. Esther Maassen explores reporting and reproducibility issues of measurement invariance checks in a set of 400 psychological articles with open data. Andrea Stoevenbelt is conducting a registered replication report on stereotype threat (ST), and uncovered that the data and conclusions of ST research suffer from measurement choices (such as time limits and missing data), if these are not addressed during the design or modeling phases of experiments. Finally, Jessica Flake and her team conducted systematic reviews of the instruments used in the Reproducibility Project: Psychology and Many Labs 2, to identify empirically based threats to validity of replication research, and identified four broad measurement challenges that replicators are likely to face. Our aim is to emphasize to researchers that measurement is a crucial part of the research process that should not be neglected in the broader quest to improve research practices in the social sciences.