A privacy scoring framework: Automation of privacy compliance and risk evaluation with standard indicators

[Display omitted] •A framework is designed to evaluate the privacy risks of personal data practices.•Privacy indicators are defined in terms of personal data based on standards.•A privacy policy analysis model is designed to identify data practices for evaluation.•The privacy indicators of mobile applications are evaluated with real-world datasets. Personal data have become the key to data-driven services and applications whereas privacy requirements are now strongly imposed by regulations. Meanwhile, people find it difficult to understand whether the services and applications handle personal data to comply with their agreements and regulations. Therefore, the need for privacy indicators, which summarize privacy contents as forms of privacy scoring, labels, etc., has increased to empower the users’ rights by providing understandable information about privacy. For firm privacy indicators, proper criteria and methods for evaluating the level of privacy risks and compliance are required. Accordingly, this paper proposes a privacy scoring framework for services in the context of handling personal data, inspired by six standardized indicators. This paper introduces detailed information on standardized indicators and proposes privacy indicators to quantify privacy scores. Also, this paper proposes methods for evaluating privacy policy based on a set of machine learning-based hierarchical binary classifiers and processes for quantifying the level of privacy risks and compliance from privacy-related information. Through analyzing privacy policies and data access lists of more than 10,000 mobile applications on Google Play Store and investigating case studies on privacy scoring of some mobile applications, this paper shows the feasibility of the proposed framework.