With the help of AI, Revenue NSW is using a range of indicators to identify and support vulnerable customers early who may be unable to pay their fines.
Operating since 2018, the program diverts vulnerable customers away from enforcement action and provides alternative resolution options. This results in fewer vulnerable people being forced to pay fines that they cannot afford. It also increases the overall effectiveness of the garnishee process.
For the purpose of this program, a vulnerable person may be:
- A person who is facing health, financial, or domestic hardship
- A person who is potentially homeless or is residing in public housing
- A person who is often involved in the criminal justice system
- A young person
With approximately 46,000 customers considered ‘vulnerable’, the community benefits of an effective AI solution are substantial.
Previously, Revenue NSW could only find out that a customer was vulnerable after debt recovery action had been undertaken. This Program helps to predict vulnerability earlier, and to offer alternate resolution options. To predict vulnerability, a number of factors are considered, such as how often they contact Revenue NSW, the number of serious fines they have, their age and predicted socio-economic status are, as well as the weekly payment amount if their debt was to be paid via an instalment plan.
This prediction is designed to support or augment, and not replace, human decision making. Revenue NSW staff can review the predictions made by the program and direct the identified customers to more appropriate resolution channels that provide targeted support. This may result in the lifting of sanctions, putting enforcement on hold, establishing repayment arrangements, or implementing a Work and Development Order to enable a customer to reduce their fine by participating in unpaid work, courses, counselling or treatment programs.
Deployment of an AI solution
Previous solutions explored by Revenue NSW to identify vulnerable customers had proven ineffective due to the limitations of manual methods in identifying vulnerable customers on a large scale. To overcome these challenges, Revenue NSW leveraged internal expertise to develop an in-house AI solution which uses machine learning paired with ongoing staff training to assess various data attributes and make a prediction about a customer’s vulnerability.
To confirm the accuracy and performance of the solution, and to make sure that it continues to deliver against its objectives, Revenue NSW performs ongoing tests and continues to iterate the solution.
Compliance with AI Ethics Policy
The AI Ethics Policy provides a set of key principles that guide the ethical use of AI by the NSW Government and ensure that any projects with an AI component align with best practice. Revenue NSWs’ solution was implemented before the development of the AI Strategy and Ethics Policy. Despite this, they recognised the sensitivity of using AI in this area and implemented appropriate governance and controls to ensure an effective outcome and to address community concern. Revenue NSW project leads were consulted as part of the development of the AI Strategy and Ethics Policy.
The project team retrospectively tested the project against the five ethical principles contained in the AI Ethics Policy and found it to be compliant. This finding has also been peer reviewed and confirmed by the AI Review Committee.
Revenue NSW continues to train their solution and experiments with adding new data to improve accuracy. The solution has been reviewed as part of the development of AI maturity in government and continues to receive feedback as part of its ongoing monitoring of the solution.
Assessment against AI Ethics Policy
Community Benefit
The solution has helped identify 15,000 people annually and allowed for more appropriate resolutions to their cases. Other ways to identify vulnerable people were considered and ruled out because they did not realise the same benefits as the AI solution.
Fairness
This AI solution applies a bias when making predictions. Whilst it is usually appropriate to avoid biases in AI solutions, in this case the intentional use of a bias in the model resulted in positive outcomes by enabling Revenue NSW to identify vulnerable people. Furthermore, the full fines data set is used for this solution to ensure full inclusion and diversity in the data. Data that does not assist the model in predicting vulnerability is excluded to ensure that it does not influence the model’s predictions. Revenue NSW conducts regular monitoring of the model to ensure that it does not drift and that unintended biases do not enter the model.
Privacy and security
As the data is collected and utilised for the administration of fines legislation and use of data is in accordance with relevant privacy legislation, consent for its use is covered by legislation and is not required by individuals. The solution does not disclose or consume any personal information (other than what Revenue NSW is legally allowed to use), and is used for a specific reason and not for broad use.
Transparency
Information about the program is publicly available and details of the program have been presented in public forums.
Accountability
This solution is designed to make predictions that assist Revenue NSW Officers to make decisions. The AI does not make any decisions itself. Accountability for the decisions to direct customers to alternative resolution pathways remains entirely with officers within Revenue NSW.
Outcomes
Revenue NSW’s use of an AI machine learning tool in this program represents a significant success story for the use of AI in NSW. The program has offered substantial benefits to the customers of Revenue NSW and met the original aims of using AI technology to solve a complex customer problem. The tool was developed to predict vulnerability across many thousands of fine recipients to achieve more effective resolutions to identified cases. Approximately 15,000 people per year have been identified as being vulnerable by the tool, leading to better outcomes for those individuals and the community. It would not have been possible to achieve this result using manual methods or algorithms that didn’t use AI.
Capability uplift
Revenue NSW leveraged internal capability, drawing the required capabilities, skills and expertise from different members of staff, to build this AI solution and execute the project.
Having a mix of capabilities and skills from a diverse range of teams supported a learning in the flow of work culture during the project. The project team included a project manager who had coding and programming skills and team members who had basic data and statistical analysis skills. During the project, the project manager upskilled the team to build their programming and coding skills whilst the team shared their data and statistical analysis skills with the project manager. This collaborative approach enabled the project team to share knowledge, test solutions and refine ideas and processes. The project team applied what they learnt, and this set up the project for success.
The following is an overview of the three teams in Revenue brought together to form the project team and the capabilities and skills required to executive the project.
- Revenue NSW Analytics Team: Technical and programming skills
- Revenue NSW Digital Team: Digital Skills
- Revenue NSW Fines and Debt Team:Business rules such as determination of vulnerability
The project team used a combination of both technical and digital capabilities and skills to bring this project to life. Outlined below are the capabilities and skills required in more detail.
Data Literacy, Artificial Intelligence, Machine Learning and Coding skills to build the machine learning model.
The project team built the first iteration of the model using a programming language and software called ‘R’. This language is available in an open-source environment (and thus free) which meant the team could being work immediately. They also used a package called ‘rpart’ which uses classification and regression trees.
Ethical Leadership, Risk Management and Talent Development Skills to supervise the machine learning model
The project team used open-source machine learning tools developed by trusted sources. Using open-source algorithms gave the team confidence in the algorithms because they had been tried and tested by numerous members of the community worldwide. Given the team already had the required had skills to build the product, they were able to take time to train other staff and develop their skills with diligence and care.
Human Centred Design, Customer Research and Community Engagement skills to create the business rules
To help determine how the product would detect vulnerability, the Fines and Debt team utilised internal data and research to determine the personas that they would need to account for. This resulted in over 20 variables that would assist in determining vulnerability such as existing debt owed, age, number of fines, and the customer’s location (to determine overall socio-economic status).
Agile Project Methodology and Working In the open skills to execute the project.
Applying collaboration and agility skills such as Agile Project Methodology and Working In The Open enabled the project to progress in a way that ensured transparency and effective communication across teams.
AI requires humans to apply both technical and digital and customer capabilities and there is a global shortage of these capabilities and skills. The NSW Government is committed to upskilling its workforce so it can respond proactively to the changing environment of work, including AI, to provide better outcomes for citizens.
Find out more about how the NSW government is building the digital and customer capability of its employees on the Public Service Commission website.