User-driven prioritization of ethical principles for artificial intelligence systems

Despite the progress of Artificial Intelligence (AI) and its contribution to the advancement of human society, the prioritization of ethical principles from the viewpoint of its users has not yet received much attention and empirical investigations. This is important to develop appropriate safeguards and increase the acceptance of AI-mediated technologies among all members of society. In this research, we collected, integrated, and prioritized ethical principles for AI systems with respect to their relevance in different real-life application scenarios. First, an overview of ethical principles for AI was systematically derived from various academic and non-academic sources. Our results clearly show that transparency, justice and fairness, non-maleficence, responsibility, and privacy are most frequently mentioned in this corpus of documents. Next, an empirical survey to systematically identify users’ priorities was designed and conducted in the context of selected scenarios: AI-mediated recruitment (human resources), predictive policing, autonomous vehicles, and hospital robots. We anticipate that the resulting ranking can serve as a valuable basis for formulating requirements for AI-mediated solutions and creating AI algorithms that prioritize user's needs. Our target audience includes everyone who will be affected by AI systems, e.g., policy makers, algorithm developers, and system managers as our ranking clearly depicts user's awareness regarding AI ethics. •Participants prioritize human safety and security of the AI systems as most important principles across all investigated scenarios•The prioritization of privacy differed among the four real-life scenarios (e.g., high in Predictive Policing vs. low in Human Resources application)•Participants ranked transparency & understandability of the decision-making process of the AI system as not very important•Overall, there is a fit between the prioritization of values in literature and institutional ethics codes and the empirical results of user's priorities