POLINE aims at developing an AI-powered pilot tool for the retrieval and analysis of judicial interpretative formulas in the CJEU and national case-law in Value Added Tax (VAT).
The project started with an in depth theoretical study focusing on the use of precedents, drafting style, creative function, and use of formulas in the case law of four legal systems involved in the project: the European Union, Italy, Sweden, and Bulgaria. Moreover, lawyers from these legal systems analysed the features of decisions adopted by the Court of Justice of the EU and the national Supreme Courts in the field of VAT.
Based on this analysis and on EU literature focusing on the use of recurrent formulas in the case law of the CJEU, which brought it to develop a unique drafting style based also on the so called 'copy-pasting technique', we provided a definition and the core features of Judicial Interpretative Formulas (JIFs), to be extracted from CJEU decisions. The theoretical analysis highlighted the existence of similar techniques (recurrent formulas, copy pasting of these formulas from previous national decisions and CJEU case law) in national Supreme courts' judgements. Hence we decided to use a unique definition for JIFs in EU and national. However, we identified distinctive features of JIFs in national case law of each legal system. [See 'Deliverables']
Based on this definition and features' analysis, we drafted annotation guidelines that were used by national lawyers to manually annotate national and EU decisions. The annotation consisted in identifying JIFs in a validation set of decisions. After the iterative double blind annotation procedure brought to satisfactory results, we started to develop prompts to automatically extract JIFs from case law. [See 'Deliverables']
The extraction process lasted many months and relied on many different prompts used in relation to different LLMs (GPT; Claude; DeepSeek; etc.). For each legal system we used different prompts and models. Moreover, we tried both one-shot and few-shots methods. The results of the automatic extraction were tested against the manually annotated decisions. The automatically extracted JIFs were used to train a NLP system to extract JIFs from new decisions not included in the dataset. [See 'Publications']
The extracted JIFs were connected to a taxonomy built manually based on both legislative and jurisprudential relevant concepts in the field of VAT. The linking were carried out through LLMs, relying and testing different techniques. [See 'Deliverables']
Network analysis of case law citations contained in extracted JIFs was performed as well to highlight the increasing use and importance of certain JIFs. In addition, network analysis demonstrated the increasing use of JIFs. Finally, similarity analysis was performed to detect the semantic similarity between JIFs so that users reading one JIF can discover similar ones.
Th results were used to build a modular platform, consisting of:
i) Legal Database,
ii) Link Visualization, and
iii) Customised Detection Module.
It covers the case-law of the CJEU and the Italian, Swedish and Bulgarian Supreme Courts and will be accessible to judges, other legal practitioners, tax policymakers and taxpayers.
The development of the tool was based on a multidisciplinary approach combining:
i) theory and practice of judicial decision making for the study of the concept of 'judicial interpretative formulas' and the analysis of the case-law;
ii) legal experts knowledge of the domain to identify the features of the case law in the specific field and the role that JIFs have in this domain;
iii) legal informatics methods for the creation of an ontology of judicial concepts in VAT and training datasets of annotated JIFs;
iv) AI, machine learning, and NLP techniques for the automatic extraction of JIFs, the detection of textual and semantic similarity;
v) network analysis for citation network analysis purposes.
The tool has been tested in 3 online national testing events and disseminated in 3 national demonstration events and 1 final international conference. The pilot tool provides a trustworthy use case of AI technologies for justice. Through its collection of JIFs and NLP-powered search engine, the tool assists judges in accessing legal knowledge reducing their work overload. Moreover, through the Customised Detection Test Module, the tool allows recipients of VAT measures to identify JIFs applied in a specific case and assess whether VAT law is correctly applied. By developing open-access automated techniques of knowledge extraction, the methods developed can be easily reused and expanded to include other fields of law and other legal systems.
