1. General

Category

Category

Public Employees

2. Project Details

Company or Institution

INPS (the Social Security Administration of the Republic of Italy) and Accenture

Project

An AI-powered classification email system to help the Italian Public Administration to better serve citizens

General description of the AI solution

The project, developed by INPS (the Social Security Administration of the Republic of Italy) with Accenture, improves the current manual process of classifying emails sent by citizens and dispatching them to the appropriate office. It achieves its goal through innovative AI techniques and automatizing analysis of content and context of massive amounts of emails. Italian citizens send INPS more than 4 Million emails each year (likely to increase with the pandemic): this automated email classification system is extremely valuable to INPS offices to respond more promptly to the citizens.

At its core, this AI solution is a classification system based on the most innovative Deep Learning architectures and models, called Transformers, which are drastically changing the NLP landscape. This solution employs these complex models to understand citizens’ requests written in unstructured emails; in addition to the text itself, this solution analyzes thread and embedded attachments. Finally, this solution is able to classify the topic of the email and dispatches it automatically to the appropriate employees for a quicker response. The content of these massive unstructured emails is very specific and covers wide varieties of topics that belong to the Social Security world.

The entire solution is based on a supervised AI and the system was trained, in compliance with GDPR policies, with the huge amount of data stored for each area in INPS databases. The Transformer architecture used is BERT developed by Google. This selected model was trained with INPS real data to generate the specific word embedding representation, used for the classification of email content, that is related to the Italian Social Security world.

The entire solution leverages Open Source technologies and public Machine Learning models (e.g., BERT). The solution has been released on premises within the INPS infrastructure, ensuring a greater data protection in accordance with GDPR policies.

Website

https://inps.it/

Organisation

INPS

3. Aspects

Excellence and Scientific Quality: Please detail the improvements made by the nominee or the nominees’ team or yourself if your applying for the award, and why they have been a success.

During the study phase of the project, we have conducted a research to identify the best type of AI architecture and model. The study was based on the comparison between different algorithms through the training and testing of each model, aiming to select the best model to manage the type of data involved in the project.

As a matter of fact, many models and algorithms were applied to complete the analysis, always using Open Source technologies and frameworks such as “Scikit-learn” and “HuggingFace”. The scope of the study was to understand whether more innovative (but more complex) algorithms like Transformers were able to exceed the “classic” NLP Machine Learning algorithms like Random Forest, Support Vector Machine, K-neighbors or Regressions, Multi-Layer Perceptron. Many Transformers were tested, and the following ones turned out to be the most relevant ones: BERT, GPT-2, DeBERTa-v2, XLNET, RoBERTa. The classification performance of both “classic” and “innovative” families were comparable; eventually, the BERT Transformer was selected, due to the ability of this kind of models to adapt also on different use cases, such as the generative AI. For this reason, and also for the application of Italian language data as transfer-learning on pre-trained BERT model, the solution is really innovative. The Transformers algorithms are continuously evolving, and this generates new pre-trained models based on more general architecture (like BERT). However, the BERT technology is mature and ready to be used in production environments. In fact, the research results are clear and detailed as the scientific papers are written by Google AI Language Group.

Scaling of impact to SDGs: Please detail how many citizens/communities and/or researchers/businesses this has had or can have a positive impact on, including particular groups where applicable and to what extent.

The solution falls within SDG 8 (Decent Work and Economic Growth) and SDG 9 (Industry, Innovation and Infrastructure), since it improves the efficiency of INPS employees (by not overburdening them and creating high-quality jobs), increase the quality of services to citizens (by cutting down response time), fosters innovation and modernizes Public Service industry, making it sustainable and more resource-efficient.

This solution will be deployed in production (i.e., in the real world) in the coming weeks. The application of the AI classification solution on the Italian Social Security Administration will generate efficiency and positive repercussions on a National scale. The effects are two-pronged: citizens will have their issues solved faster, and INPS employees will not have to spend significant amount of time to sort the massive unstructured communications, thus investing their time and skills to more added value activities. Nowadays INPS manages more than 4 Million emails per year (and growing) on 450 different territorial agencies throughout the country: all is done through manual employee intervention, with no automated sorting system. This new solution instead introduces a sustainable approach: it can be re-used in other Public Administration areas, and re-qualifies the work of INPS employees, saving also heavy amount of time.

The technical components of the solution are based on open technologies and their rationales are highlighted on scientific papers.

The usage of huge amount of Italian language data, present on INPS databases, could be the first step to integrate and generate new kind of word embeddings (Transformers), specific for Italian Language, that currently is a sector still in its early stage.

The solution is a good candidate to become, after new trainings, a global solution for all similar National Social Security Administrations across the world, being tailored on topics that have a lot in common with countries’ Public Administrations.

Scaling of AI solution: Please detail what proof of concept or implementations can you show now in terms of its efficacy and how the solution can be scaled to provide a global impact ad how realistic that scaling is.

Currently the system is able to properly classify about 40% of unstructured requests. Therefore, on an annual basis, approximately 1.6 Million communications (out of 4 Million) can be automatically classified, saving more than 10,000 days of work distributed across all INPS agencies throughout the country, considering an average time of 3 minutes to analyze the communication (email body and attachments).

The scalability of the solution aims to guarantee a greater coverage of the classification areas, saving further potential days of work, that could be dedicated to higher added value activities. Any amount of time not spent on these activities can be invested to enhance the overall services for the citizen, better manage their requests, and improve the general perception of the Public Administration.

For INPS, this is the first tangible application of such modern AI technology that provides real direct aid to employees. This is a first step towards the implementation of an ecosystem of AI-powered solutions that will support back office employees in more areas, with the main ultimate goal of offering more efficient and engaging services for the citizens.

The model has been trained with a large amount of data owned by INPS. It can definitely be applied and integrated into other areas of Public Administration in Italy (as it is now) and potentially across the world (as described earlier).

This solution has tremendous impact on two different sets of people:
– all ~20,000 INPS employees spread across the 450 local agencies in Italy
– the users, i.e. all the citizens who send an email to INPS local agencies (potentially, the entire 50 Million Italian adult population)

The entire solution leverages Open Source technologies and is being released within the INPS production infrastructure (i.e., in the real world), managing personal and sensitive data in accordance with GDPR policies.

Ethical aspect: Please detail the way the solution addresses any of the main ethical aspects, including trustworthiness, bias, gender issues, etc.

We have developed a solution that adheres with ethical principles, such as the “Policy and Investment Recommendations for Trustworthy AI” published by the European Commission High-Level Expert Group on Artificial Intelligence (AI HLEG), in compliance with the seven requirements of Trustworthy Artificial Intelligence (human agency and oversight; technical robustness and safety; privacy and data governance; transparency; diversity, non-discrimination and fairness; environmental and social well-being; accountability).

Through the automatic management of the unstructured communication, it has been possible to reduce the bias and error margin, since there is no longer a human person reading it. Moreover, if we look at the impact from an environmental point of view, speeding up communications helps to increase confidence in using digital tools, reducing the need for citizens to physically access INPS offices.

If properly designed and used, AI technologies can offer real prospects for improving the quality of life. In the relationship between citizens and Public Administration, it will be possible to allow greater accessibility to public services, leading to a considerable reduction in their costs, with benefits in terms of social spending, and a decrease of the time spent to handle the requests. It will be possible to strengthen many procedures with appropriate automation, giving citizens the opportunity to interact with the Public Administration in a more agile, effective and customized way. This will bring benefits to everyone, including the elderly, the disabled and the low income people, and will allow the Public Administration to recover and strengthen the trust from the citizens in order to make them feel as part of a community.