Evaluating Second Generation Cross-Country Open Legal Data Infrastructures Using Value Models

Access to legal information is of critical importance to socio-economic and political activity. Hence, the provision of capabilities to search for, locate and retrieve legal information in an efficient and structured manner to governments, businesses, lawyers and citizens is highly valuable. In order to satisfy the requirement for legal information, a ‘first generation’ of open legal data infrastructures that focus on providing access to national legislation and offer some basic functionalities for the providers and the users of these data, is already available in many countries. However, rapid globalization, the emergence of supra-national unions of nations, the advent of advanced data processing capabilities, and the ever-increasing complexity of legislation as it comes to mirror modern life have all contributed to the development of more advanced ‘second generation’ open legal data infrastructures that facilitate access to legal information from multiple national legal frameworks in multiple languages. In the face of the advent of these new technologies, it becomes vital to understand how their performance could be evaluated. This paper presents and validates a methodology for evaluating the emerging second generation of Big Open Linked Legal Data (BOLLD) e-infrastructures based on the concept of ‘value model’ estimations from users’ evaluation ratings. The proposed approach advances beyond the traditional Information Systems (IS) evaluation approaches, as it includes assessments not only of the magnitudes of a wide range of types of value generated by such an infrastructure, but also of the relations among them. The proposed model, therefore, enables a deeper understanding of the whole value generation mechanism, and also can provide a rational definition of priorities for system improvement based on the capabilities offered to users. A first application of the developed approach is made for the evaluation of an advanced second generation BOLLD e-infrastructure developed as part of the European project ManyLaws, leading to both interesting insights as well as improvement priorities.