Best practices in procuring AI: Lessons from Chile

For countries grappling with questions about how to buy and use artificial intelligence (AI) in the public sector responsibly, Chile’s experience offers some valuable insights.

Over the last four years, the Latin American nation has introduced a series of measures to promote the responsible and ethical use of AI in government, including how to buy AI successfully.

Chile first published a national policy on using AI in government in 2021. The policy notably puts a strong focus on the governance and ethics of implementing AI, alongside other practical considerations. According to the country’s guidelines for using AI in public entities, AI should be centered on the public’s needs, and buyers should evaluate whether it is the most appropriate technological solution to meet these needs while promoting government efficiency.

A corresponding action plan features specific objectives to modernize public procurement processes for buying AI, including creating a regulatory framework and training public officials to enhance AI procurement efficiency and effectiveness, as well as exploring how to integrate responsible and ethical principles in buying AI.

A Directive issued in December 2023 summarizes this effort and provides specific recommendations on how to purchase AI and address common risks. 

It includes the key questions to ask and relevant actions to take at each stage throughout the procurement process – from investigating the problem, proposing a solution, through to managing the implementation of the contract. The Directive also addresses the key role of data and recommends mapping available data sources and quality as part of the process. At the same time, it provides flexibility with the ability to define clauses and differentiate them according to the project. The objective: Help deliver solutions that are more results-oriented and not standardized by the purchased service and supplier. 

In the words of the Directive: “If projects are planned with these best practices from the outset, they can contribute to public value and adequately address privacy risks, biases, and lack of transparency. With ethical and responsible use of data, public buyers will help build trust with citizens and harness the potential of data to improve the design and management of public policies.”

The Directive is currently being updated to align with the new public procurement law. 

Piloting buying AI with a focus on responsible data use 

The Directive also builds on the insights of a collaboration between the public innovation lab GobLab UAI and ChileCompra, the country’s public procurement agency, that developed best practices for the procurement of data science and AI projects.  

Over two years, the project involved a deep analysis of existing AI standards and guidelines, consultations with public procurement officials, AI experts, and private sector companies, and applied these lessons to two pilot processes. The project particularly focused on the responsible and ethical use of data in the context of AI used by governments. 

The first pilot was for the purchase of a model to optimize oversight activities and prevent fraud in the National Health Fund. The second, for the Public Defender’s Office, was for a solution to measure the quality of its outsourced legal services. 

The resulting standard bidding documents provided a framework for developing and evaluating AI projects. These were then carried over into the official Directive. 

Key insights and recommendations

We talked to Romina Garrido, acting director of GobLab, who coordinated the Ethical Algorithms project with ChileCompra. These are some of her key lessons: 

• Focus on the problem: What matters is the problem description, not focusing on a specific solution.  
• A multidisciplinary approach: Successful AI procurement requires a diverse team with expertise in data science, public procurement, and ethics.
• Data quality is critical: The quality of data used to train AI models matters. Data needs to be accurate, unbiased, and representative.
• Flexibility and adaptability: Given the rapid pace of AI development, procurement processes must be flexible enough to accommodate new technologies and approaches and avoid being locked into a specific solution.
• Market readiness: One of the biggest challenges has been the market’s lack of preparedness. Many potential vendors, even large ones, were unfamiliar with the concepts of ethics in AI and the associated requirements. Romina’s team had to develop tools and guides to facilitate the understanding and implementation of these standards and invest in training and development, particularly for the start-ups this project aimed to support.
• Continuous monitoring and evaluation: New requirements need to be introduced with responsibility and accountability, ensuring the openness of the algorithm.

ChileCompra continues to collaborate with GobLab, most recently advising on ethical considerations for tenders from the social security and medical insurance agency SUSESO (for an interesting deep dive into the lessons from this project, visit the World Privacy Forum). This post is part of our series on the impact of AI on public procurement.