The “perfect” reader study

•Clinical benefits from technological advances in radiology are mostly proven by reader studies.•Reader studies are challenging to design and require significant resources and time to be accomplished.•Appropriate study design with minimized sources of bias allows you to get correct results with good chances of generalizability and publication. A reader study, i.e. a diagnostic accuracy study aiming to assess clinical performance of one technology versus another, on the basis of image interpretation by a group of human readers, requires an accurate design, with the precise definition of each element concurring to the study itself. In this paper, the main issues to take into consideration when designing reader studies are described, and the characteristics of a number of indices of diagnostic accuracy are summarized. Special care is given to the description of the multiple sources of bias, which should be controlled in order to assure study soundness and allow consistent conclusions. Although a “perfect” reader study does not exist, because bias cannot be completely eliminated, a good study would be designed to minimize the bias effects, and possible unavoidable bias should be described in study reporting and used to provide a correct interpretation of results.