Canada’s government is putting its faith — and $3.3 million — in software designed to assess which AI chatbots are trustworthy for the benefits system, which supports almost 16 million Canadians. To protect the commercial interests of its developers, the government is keeping key details about the software secret, raising serious concerns about transparency and accountability.

For Canadians who receive support from the federal government, there are concerns that this could lead to the introduction of programs that automate adjudication and denial of benefits claims.

Employment and Social Development Canada (ESDC) is leading the Benefits Delivery Modernization Programme, overhauling aging IT systems to improve the delivery of Old Age Security, Employment Insurance, and the Canada Pension Plan. Part of the modernization efforts includes integrating generative AI, like AI chatbots, to boost efficiency. 

The ESDC is working with a little-known Ottawa-based company called NuEnergy.ai to do this. 

On May 3, 2024, the Ministry signed a one-year contract to license its software, called the Machine Trust Platform. The Ministry used the software to help develop a framework of rules and policies for AI, and to evaluate the risk of individual tools. The original $1.3 million contract was later amended to more than $3 million. The government quietly outsourced its judgment on which AI tools to implement to a private company with little public transparency. Whether Canadians actually want such a system or AI in government in the first place, let alone whether the Machine Trust Platform works as advertised, or if the contract is even worth the price tag, were never discussed.  

“Through our collaboration with NuEnergy, we are building transparent and accountable governance mechanisms to ensure our use of generative AI is done responsibly, ethically, and securely,” Samuelle Carbonneau, an ESDC ministry spokesperson, told Ricochet Media. Yet despite those transparency claims, there is no publicly available information about what the Machine Trust Platform has achieved and whether it works as advertised. The ministry has declined to release the reports that the software generates when assessing AI tools. 

Ricochet has obtained a training manual, presentations, and a final report pertaining to the contract through an access to information request. To protect NuEnergy.ai’s commercial interests, the ESDC fully withheld 232 out of 251 pages under two clauses of the Access to Information Act — 20(1)(b) which protects financial, commercial, scientific or technical information and 20(1)(c) which protects against information that could lead to material financial loss, gain or prejudice if released.

The information they provided summarizes and links to the federal government’s public policies and procedures on AI use, alongside some internal resources. The Year 1 Final Report is completely withheld apart from the title page and parts of the appendix. Three partially redacted slides from a presentation explaining the rules and procedures for AI developed as part of the Benefits Delivery Modernization Programme are provided but partially redacted. 

The withheld information and heavy redactions make it impossible to tell whether the software actually works, or what precisely the Ministry paid for. 

Renée Sieber, an associate professor at McGill University who researches government AI adoption, warns such systems could justify further layoffs of public service workers and obscure accountability by blaming the AI or assessment tools that helped greenlight them when something goes wrong. For example, chatbots might provide incorrect information to an employee assessing benefits claims, leading to a wrongful denial. 

“We should be skeptical” of modernization efforts coming out of this contract, she told Ricochet Media.

‘Experimenting’ with AI on vulnerable benefits recipients

The presentation, obtained through an access to information request, mentioned five use cases for the Machine Trust Platform software, though the specifics were redacted. 

A Ministry spokesperson declined to describe specific use cases. But they explained that the software provided assessments of “knowledge retrieval” AI tools — chatbots that answer questions about benefits rules and procedures. Ricochet contacted government employees and contractors who were listed as attendees to an “executive education session” in one document obtained through the request for more insight. Of the 19 unredacted attendees, three could not be reached, and 16 did not respond for comment.

A favorable assessment could support the rollout of chatbots. Several are already operating within the department. One chatbot built by Accenture, for example, answers staff queries about employment insurance regulations. Others are intended to help employees search through information, edit text, code and support research tasks.

Other ministries are also experimenting with AI. In 2020, Canada’s Revenue Agency spent $18 million on a chatbot to help with tax filing, but a recent Auditor General’s report found it provided incorrect or incomplete information most of the time. 

The Machine Trust Platform might one day assess riskier tools in the future, which might affect benefit eligibility decisions or flag fraud. 

Around the world, attempts to automate welfare systems for this purpose have ended in disaster. In the Netherlands, an algorithm falsely flagged childcare benefits claims from 26,000 parents — predominantly immigrant families — as fraudulent between 2013 and 2019. In Australia, an illegally implemented AI system known as Robodebt demanded repayments from tens of thousands of welfare recipients for debts they didn’t owe.

“Government bodies are experimenting with uses of AI in areas where there’s already widespread systemic inequality,” Joanna Redden, co-director of the Starling Centre at Western University and the UK-based Data Justice Lab, told Ricochet Media. “This raises serious concerns about the negative implications of the uses of new and emerging tech in areas like benefits administration.”

NuEnergy is ‘Government of Canada Approved’

Taxpayers funded the development of its Machine Trust Platform. From 2019 to 2022, multiple federal ministries, including the RCMP, paid more than $2.4 million to pilot the software. The platform was developed as part of the Innovations Solutions Canada Testing Stream, which purchases and tests new technology before it hits the market. 

In April 2023, per emails obtained through a separate access to information request to Innovation, Science and Economic Development Canada (ISED), Bhargava unsuccessfully pitched ISED on a new contract. “You will be pleased to know that we are leveraging our research and experience and also applying AI Governance to today’s tech innovations like ChatGPT.” 

In a presentation to the York Regional Police Board at the end of November 2025, CEO Niraj Bhargava said the company “graduated out of a testing program” within the government of Canada. The accompanying slides claim the software is “Government of Canada approved.”

NuEnergy.ai’s  work with police agencies raises more concerns. “We are leaders in AI governance that have focused on law enforcement,” Bhargava said. The company lists Interpol, the Canadian Association of Police Chiefs, and the International Association of Police Chiefs as partners, and the York Regional Police as customers. 

“We know that some of the applications of AI in the world of criminal justice are questionable because they have built in racism into their approach,” Alexi White, director of systems change at Maytree, a charity focused on poverty reduction and inequality, told Ricochet. 

Bhargava and NuEnergy did not respond to multiple requests for comment through email and LinkedIn. As of May 2025, Bhargava is a founding chair of the non-profit Canada’s Voice for AI Sovereignty (CAISC) that aims to bring together Canadian companies and researchers to build Canada’s AI ecosystem.

Was the contract worth $3.3 million?

After reviewing the documents obtained through the access to information request, some experts have questioned the cost of the contract. 

“Usually these consulting companies [take] an existing framework that they just copy and paste,” Sieber said. An existing governance framework should not cost this much, “unless the government is being shafted.”

For Megan Linton, policy lead at the Disability Justice Network of Ontario, the contract makes “clear the State’s desire to save money by wasting billions on failing AI infrastructure and paying out wealthy friends. We know this will only further debilitate the most marginalized […] while profiting the elite few.”

According to the documents later obtained from ISED, the original scope of the proposal detailing how the ESDC wanted to use AI companies for modernizing benefits delivery “was more narrow that[sic] expected” and it would need to be expanded based on feedback from Public Services and Procurement Canada.

Trust, don’t verify

Despite Ottawa’s growing appetite for AI, oversight hasn’t kept up.

Federal ministries are required to complete risk evaluations of AI tools at the design stage and before deployment. These evaluations, called Algorithmic Impact Assessments (AIA), comprise dozens of questions to pinpoint potential risks, biases, and transparency issues with AI systems. The Machine Trust Platform also integrates AIAs into its evaluation of AI tools. 

When Sieber and PhD candidate Ana Brandusescu reviewed AIAs in a 2025 study, they discovered that almost every department refused to release them. To date, only 30 such assessments have been published.

At the ESDC, at least 17 different AI tools in use or development, but only six have public algorithmic impact assessments. ESDC did not conduct one on the Machine Trust Platform since it doesn’t technically use AI or automate decisions; it only assesses other AI tools. Still, Sieber is concerned the Machine Trust Platform will be a compliance tool — a box to check off before deployment — rather than a serious oversight tool. 

When the ESDC is allowed to operate software like the Machine Trust Platform without public transparency, “governance becomes a trade secret,” Brandusescu said.

Questions about the algorithm impact assessments fall outside the purview of Evan Solomon, Minister of Artificial Intelligence and Digital Innovation. His office directed Ricochet to the Treasury board, since Solomon does not oversee federal use of AI. In response, the Treasury Board told Ricochet that only automated tools that “directly impact clients’ legal rights, privileges, or interests” require an AIA. 

For example, the Employment Insurance Jurisprudence Tool that helps reassess appeals and denied claims, doesn’t have a public AIA.

Most Canadians receiving benefits do not know how AI is reshaping the system. 

“The people who are most affected by the kinds of systems that are being introduced to aid or replace human decision making aren’t informed and do not have an opportunity to discuss if, where, and how these systems should be used,” said Redden.