Artifacts supplementing the EuroUSEC '22 paper "Assessing Real-World Applicability of Redesigned Developer Documentation for Certificate Validation Errors"

This upload supplements the conference EuroUSEC 2022 submission by providing the full questionnaire, anonymized dataset and all performed analyses presented in the paper specified below. Title: Assessing Real-World Applicability of Redesigned Developer Documentation for Certificate Validation Errors Authors: Martin Ukrop, Michaela Balážová, Pavol Žáčik, Eric Vincent Valčík, Vashek Matyas Paper details: https://crocs.fi.muni.cz/public/papers/eurousec2022 Paper abstract: We face certificate validation errors commonly, yet the related tools and documentation had been shown to have very poor usability. Previous research suggests that just improving the error messages and corresponding documentation can have significantly positive effects. Our work aims at increasing the usability of certificate validation by 1) redesigning the API error messages and the corresponding documentation, and 2) validating the real-world applicability of the redesign by investigating the opinions of 180 IT professionals. We focus on the perceived obstacles, desired ideal form and overall satisfaction. The redesigned documentation exhibits a reliable significant decrease in perceived incompleteness, with a small amount of perceived bloat and tangle. The redesigned documentation, now published on a dedicated website, is preferred by 89% of our study participants. The artifacts accompanying this paper contain three major parts: The questionnaire used in the main study (described in Sections 3.1 and 3.2 of the paper and mostly present in Appendices A and B of the paper). The anonymized dataset (multiple formats) of all valid questionnaire answers and qualitative coding performed. Analyses of this dataset are the core of the paper and are present in subsection 3.4 and all parts of Sections 4 and 5. The set of analyses files (IBM SPSS scripts and outputs) producing all statistical results presented in the paper are included. More details about the artifacts can be found in the README file in the artifacts archive.