Abstract

The focus of this symposium is on evaluating the quality of peer-reviewed papers and publicly available datasets. We will identify shortcomings in the peer review process, the design of studies, the reporting of statistics, and the way data are shared. We will also discuss in our presentations recommendations and tools to counteract the shortcomings that were observed.

The symposium consists of four presentations.The first presenter, Hilde Augusteijn, will discuss how peer review functions as a gatekeeper of the quality of scientific manuscripts. A preregistered survey amongst 800 researchers and students investigated which aspects are considered to be of importance when assessing the quality of both manuscripts submitted for publication, as well as student theses.

Second, Marjan Bakker will discuss a preregistered study that investigates whether psychological studies’ sample sizes have increased over time, specifically in response to the replicability crisis and the resulting focus on open science.

Third, Robbie van Aert will describe a preregistered study that is currently ongoing to assess whether statistics are more frequently inconsistent in preprints on COVID-19 than in comparable preprints that are not about COVID-19.

Finally, Rick Klein will discuss an effort to systematically assess privacy risks (data sensitivity and risk of re-identification of participants) in a sample of 2,169 open datasets shared alongside peer-reviewed articles in three journals.