AI must be built with Indigenous Knowledge to avoid harm, researchers say

A University of Melbourne research review of 53 academic studies, published during NAIDOC Week's 50th anniversary, finds that artificial intelligence is simultaneously a tool for preserving Indigenous languages and a source of harm when trained on data that excludes or exploits Indigenous communities. The findings arrive as Pope Leo XIV's encyclical Magnifica Humanitas frames AI as a moral test, warning of discrimination against marginalized peoples and demanding accountability over data ownership and benefit.

AI for language preservation and cultural mapping

Researchers are deploying AI to revitalize endangered languages and strengthen cultural ties to land. A prototype app from Curtin University in Western Australia identifies wildflowers and plays their Noongar names to aid pronunciation. In California, a partnership with the Amah Mutsun Tribal Band uses machine learning to map plant distributions essential to the tribe's cultural revitalization. These applications show how computational tools can support Indigenous Knowledge holders in keeping traditions alive.

Community-led healthcare solutions

AI also assists health programs that communities themselves prioritize. Aboriginal medical clinics in remote Western Australia trial AI-assisted screening for diabetic retinopathy, a leading cause of preventable blindness. Other Australian providers use AI to detect middle-ear diseases in Aboriginal and Torres Strait Islander children. In Taiwan, predictive modeling assesses fall risks for Indigenous elders, addressing gaps caused by distance, skills shortages and high costs. One initiative shows how AI for Healthcare can address critical access gaps when co-designed with local communities.

Bias and data extraction

The same systems that offer support can also harm. A child protection algorithm in Aotearoa New Zealand referred Māori children to services at disproportionately high rates, while an AI eye-screening tool trained only on Chinese clinic data produced high false positives for Aboriginal patients, likely because it ignored retinal pigmentation variations. These failures stem from training models on generalized data that do not reflect specific populations, often collected without Free, Prior and Informed Consent - a practice that violates the principle "nothing about us, without us."

Indigenous data sovereignty frameworks

Two Indigenous-led frameworks explicitly reject extractive data practices. The CARE principles (Collective Benefit, Authority to Control, Responsibility and Ethics) and the OCAP principles (Ownership, Control, Access and Possession) confirm that communities must govern their own data. Most AI systems deployed today lack these safeguards, the review notes, leaving Indigenous peoples without control over how their knowledge is used or compensated.

From voluntary frameworks to binding regulation

The researchers call for regulators to turn principles like CARE and OCAP into law, making Indigenous Data Sovereignty a mandatory standard. Australia's voluntary AI Ethics Framework offers a starting point, but without enforcement it cannot prevent harm. The technology industry must treat co-design with communities as a precondition for deployment, not an afterthought, and only train models on data that Indigenous peoples have approved. This shift demands professionals who understand the intersection of policy and technology, including those working in AI for Government.

Why this matters for IT, development, science and research professionals

IT, development, science, and research professionals build the data pipelines, models, and evaluation frameworks that determine who benefits from AI. The evidence shows that skipping community consent and diverse training data leads to biased outputs with real-world harm. To avoid these failures, practitioners should audit training datasets for cultural representation, advocate for co-design processes from project inception, and support regulatory standards that enforce Indigenous data sovereignty. The cost of ignoring these steps is not abstract - it is measured in false diagnoses, unfair child protection referrals, and the erosion of trust in technology.