Performing and analyzing non-formal inspections of entity relationship diagram (ERD)

•Two measures for defect detection performance and defect difficulty level are proposed.•Nine software engineers’ reasoning performances and eye tracking data are used to validate the measures.•Detecting the missing information type defects in the ERD is harder than detecting other types.•The defects in the ERD related with cardinality and optionality of the relationships are harder to detect. Designing and understanding of diagrammatic representations is a critical issue for the success of software projects because diagrams in this field provide a collection of related information with various perceptual signs and they help software engineers to understand operational systems at different levels of information system development process. Entity relationship diagram (ERD) is one of the main diagrammatic representations of a conceptual data model that reflects users’ data requirements in a database system. In today's business environment, the business model is in a constant change which creates highly dynamic data requirements which also requires additional processes like modifications of ERD. However, in the literature there are not many measures to better understand the behaviors of software engineers during designing and understanding these representations. Hence, the main motivation of this study is to develop measures to better understand performance of software engineers during their understanding process of ERD. Accordingly, this study proposes two measures for ERD defect detection process. The defect detection difficulty level (DF) measures how difficult a defect to be detected according to the other defects for a group of software engineers. Defect detection performance (PP) measure is also proposed to understand the performance of a software engineer during the defect detection process. The results of this study are validated through the eye tracker data collected during the defect detection process of participants. Additionally, a relationship between the defect detection performance (PP) of a software engineer and his/her search patterns within an ERD is analyzed. Second experiment with five participants is also conducted to show the correlation between the proposed metric results and eye tracker data. The results of experiment-2 also found to be similar for DF and PP values. The results of this study are expected to provide insights to the researchers, software companies, and to the educators to improve ERD reasoning process. Through these meas