Summary of the context and overall objectives of the project
The project builds on the emerging field of legal analytics (LA), which develops applications in the legal domain to extract legal knowledge, infer undiscovered relations, and engage in data-driven predictions. So far, applications have been developed generally as private-sector commercial initiatives, although initiatives specifically targeting judges and courts are increasing. In this domain, AI brings great opportunities, as it can contribute to improving the work of the courts and the efficiency of the legal systems, enhancing speed, quality and consistency of decisions while reducing human bias. LA can respond to the need for judges to reduce their effort in retrieval and elaboration of legal framework, concept evolution and trends in decision-making.
The overall objective of the ADELE project is to design a methodological framework of legal analytics (LA) for court decisions and implement a pilot tool designed for Italian and Bulgarian case law in the fields of Trademark and Patents, and Value Added Tax, and tested on German case law to check language transferability.
In particular, ADELE pursues the following specific objectives: (1) knowledge extraction and modelling: it provides judges with a rich lightweight ontology representing the structure and relations among concepts relevant to the targeted legal fields; (2) network analysis of case law: it provides a network citation analysis featuring links between case law, coupled with automated function to identify arguments in text and group according to their similarity and their function in determining the final outcome of the case; (4) detection of trends in judicial decision-making: it uses artificial intelligence techniques to enable the anticipation of the most likely outcome of the decision based on previous judgments contained in the database.
The different functionalities are openly made available in the ADELE pilot tool (https://adele-tool.eu/).
The tool has been tested and validated separately in Italy and Bulgaria with specialist judges. In particular, two preliminary tests aimed at assessing the demo-version of the platform and gain feedback from end-users, while two validation tests have been organised for validate the final version of the tool.
Work performed in the project and main results achieved
During the project, a multidisciplinary approach combining methods pertaining to the fields of law, legal informatics, computer science and AI has been implemented.
A large amount of Italian, Bulgarian and German (limited to the test set) decisions in the field of trademark and patents law, and value Added Tax have been collected and analysed in an incremental fashion. The legal analysis aimed at studying the structure, type of reference, reasoning of decision within and across the two legal systems.
Based on the legal analysis performed, two comprehensive sets of annotation guidelines have been developed, in order to guide legal experts in the task of annotating decisions. The guidelines were aimed at guiding annotators in identifying relevant information in judicial decisions, such as relevant sections and parts of the judgements and the type of judicial arguments. Annotation has been carried out for the purpose of automatically extract arguments from cases and predict outcomes of future cases.
Links between judgements have been automatically extracted from the text of decisions or part thereof. Then, a graph containing the citations and the relations between them was developed. Finally, new information has been extrapolated from the graph, containing statistics on most cited cases.
A lightweight ontology framework has been developed to represent main concepts in the two legal sectors under analysis. The ontology contains the most important concepts, derived from relevant EU or international legislation, as well as link between those concepts. AI methods to link ontology concepts to judicial decision contained in the datasets have been tested.
Based on the annotated corpus, different machine learning methods have been tested to automatically extract arguments from cases and predict the outcome of the decision. More specifically, arguments have been automatically detected and classified according to their role in the judicial reasoning (e.g., premises or conclusion) and their scheme. The outcome prediction module refers to the possibility of predicting the likely outcome of a party’s request, supported by claims and arguments, based on previous case law, contained in the training set. Moreover, different machine learning models have been tested to automatically extract relevant words and sentences from cases. Finally, experiments on German test set have been carried out.
The ADELE pilot tool has been developed and embedded into a modular platform, openly accessible online (https://adele-tool.eu/). The development of ADELE back-end infrastructure was aimed at implementing a multi-stage data processing cycle starting from collection of the raw and the annotated texts of court decisions and other legal documents. A demo version of the online platform was released in September 2022, which was then presented at the test events with judges held in Italy and Bulgaria. The final version of ADELE pilot tool was launched in April 2023, and contain the following functionalities: outcome prediction; automated argument extraction; ontology framework; citation and network analysis; search for similar cases; automated extraction of keywords and case summaries.
The demo-version of the ADELE pilot tool was tested in two events organised separately by EUI in Italy and by APIS, SAJUZ and LIBRe in Bulgaria. Furthermore, the final version of the pilot tool was tested. The validation events were organised both in Italy by the EUI and in Bulgaria by APIS, SAJUZ and LIBRe. The four events gathered a large number of judges and courts’ judicial staff and allow to received feedback about the technical and user-operative capacity, as well as the legal soundness of the ADELE pilot tool.
At the end of the project, the Final Conference of the ADELE Project was held at the European University Institute in Florence in a hybrid mode, and was attended by highly qualified speakers, including academics and doctoral students, judges, and other legal practitioners.
Progress beyond the state of the art, expected results until the end of the project and potential impacts
The ADELE project has produced a new methodology for the development of LA techniques for the judiciary and has delivered an innovative platform with several usable functionalities.
Therefore, the project is expected to positively impact the judicial system by showing how AI can be profitably used to facilitate understanding of the law, thereby increasing efficiency and transparency of decision-making.
In particular, the legal database provides a comprehensive access point to case law and legislation, enriched with metadata. Automatically extracted keywords and summaries are particularly relevant in that they provide concise information into the matter of legal cases, thus allowing judges to rapidly decide whether the case is relevant or not for the decision at hand. Next, the automated extraction of links provides judges with access to a comprehensive web of legal references, enabling them to explore the intricate web of legal precedent and interpretation. In addition, automated extraction of arguments provides a quick and structured view and analysis of the arguments contained in a judgement, thus empowering judges to engage in more in-depth, nuanced, and efficient deliberations, ultimately enhancing the quality of legal decision-making. Finally, the outcome prediction allows us to estimate what could be the most likely outcome of a decision, thus allowing judges to estimate how an average colleague would decide the case at hand considering previous case law, contained in the dataset.
The application of the developed machine learning models in three languages (Italian, Bulgarian, and German for the test set) has contributed to how AI can be used for the transferability of text-based analytics between multiple jurisdictions.
From an academic standpoint, the project has engaged with other academics, research projects and groups to exchange views on the potential of AI in the judiciary, thus stimulating further research and applications of judicial LA.
Finally, from a societal perspective, the project has fostered the development of trustworthy AI, through the development of legal analytics methodologies that comply with the relevant legal and ethical framework. The team has carried out an ethical self-assessment to show how ethical and legal issues have been taken into consideration in the different stages of the AI pilot tool development.