Four Strategies for the UN AI Panel to Assess and Track AI Risk

Four Ways the International Scientific Panel on AI Should Approach AI Risk

In August 2025, the United Nations agreed to form the International Scientific Panel on AI (ISP-AI)-a major step toward evidence-led AI governance. This follows the secretary-general's AI Advisory Body (2023) and the inclusion of the ISP-AI and Global Dialogues on AI in the Global Digital Compact, annexed to the Pact of the Future in 2024. Costa Rica and Spain then drove negotiations on the panel's terms of reference.

The panel's job is clear: deliver a yearly, evidence-based readout of AI risks, opportunities, and impacts. It will synthesize existing research, not run new studies-opening the door to global scientific contributions and a shared baseline for cooperation.

The mandate is large. The MIT AI Risk Repository tracks over 1,600 risks across domains like discrimination, misinformation, and socioeconomic harm. Policymakers already juggle different schemas-from the EU AI Act's risk tiers to the NIST AI Risk Management Framework's pillars of harm. The ISP-AI will need a consistent, repeatable method that enables year-over-year comparison to judge whether policy is actually reducing risk.

Four Approaches the UN Uses to Monitor Risk

Approach 1: Global Risk Assessment Conducted by the United Nations

Who collects and analyzes: United Nations. Scope: Global.

This mirrors the IPCC model and UNDRR/World Bank global assessments. The goal: a cohesive view of cross-border risks, common standards, and clear signals for international action.

• Value: Comparable data across regions; consistent methodology; clearer prioritization for funding and cooperation.
• Watch-outs: Local nuance can get lost; national differences in exposure and policy maturity are harder to capture.

Approach 2: National Risk Assessment Conducted by the United Nations

Who collects and analyzes: United Nations. Scope: National.

Inspired by IPC food insecurity monitoring and INFORM's Global Risk Index, this approach assesses countries one by one using shared thresholds to trigger support or intervention.

• Value: Builds national capacity; enables apples-to-apples comparisons; adapts global standards to local contexts.
• Watch-outs: Resource-heavy; requires tight coordination across many agencies.

Approach 3: National Assessments Reported to the United Nations

Who collects and analyzes: National entities (primary), with UN compiling. Scope: National and global.

Countries run their own assessments using a UN framework and report results. The UN aggregates, validates, and produces a global picture. WHO, ICAO, and IAEA use versions of this model.

• Value: Leverages national expertise; increases ownership and accountability; richer, context-aware data for global synthesis.
• Watch-outs: Quality varies; methodology drift hurts comparability; reporting can be slow or incomplete.

Approach 4: UN Data Hubs Inform National Assessments

Who collects: United Nations (and partners). Who analyzes: National entities. Scope: National and global.

Here, the UN curates open data from official statistics, citizen science, and thematic hubs-think World Bank climate portals-so countries and researchers can run their own risk analyses.

• Value: Broad access; encourages independent analysis; supports transparency and reuse.
• Watch-outs: Data without shared evaluation criteria can fragment; quality control matters.

What This Means for the ISP-AI

Expect the ISP-AI to start with Approach 1 to map global AI risks tied to human rights, sustainable development, peace and security, humanitarian aid, and rule of law. But risk exposure varies widely: low adoption often means lower immediate risk; innovation-heavy economies may accept more risk; policy maturity is uneven.

To judge whether policy works, the panel will likely add Approach 2 (UN-led national assessments, potentially via observatories) and Approach 3 (national self-assessments reported to the UN). Existing bodies could help, such as the OECD's AI Policy Observatory and UNESCO's Global AI Ethics and Governance Observatory. Approach 4 underpins all of this by making data widely available for analysis and reuse.

A staged plan to 2026

• Phase 1 (now-2026): Launch a global assessment (Approach 1). Lock a clear taxonomy aligned to multilateral objectives. Define indicators, data sources, and rating logic. Set baselines for year-over-year tracking.
• Phase 2 (in parallel): Partner with observatories (Approach 2/4). Publish a national metrics pack: definitions, templates, minimum datasets, and thresholds. Offer technical assistance for low-resource contexts.
• Phase 3 (post-2026): Roll out national self-assessments with validation (Approach 3). Standardize submission schedules, quality checks, and methods to reconcile gaps and late reports.
• Always-on: Maintain backward compatibility-keep indicators stable or map changes. Publish open data and methods so findings can be verified and improved.

Practical Actions for Governments and Research Teams

• Assign a lead entity: Standards institute, AI risk institute, or an AI ministry/department with a formal mandate to report.
• Map exposure: Track where AI is deployed across sectors and critical functions; identify vulnerable populations and systems.
• Adopt a simple schema: Use risk tiers or harm pillars compatible with major frameworks (e.g., NIST AI RMF) to ease future reporting.
• Set triggers: Define thresholds that prompt mitigation, audits, or temporary pauses in high-stakes use cases.
• Build data pipelines: Establish data-sharing agreements, metadata standards, and audit trails so assessments are repeatable.
• Publish transparently: Release methods, indicators, and summary findings; invite independent review.

• For researchers and civil society: Contribute systematic reviews, benchmark datasets, and evaluation methods. Test policy effectiveness with before/after and difference-in-differences designs. Collaborate across borders to reduce duplication and bias.

Global technical consensus will not emerge from opinions; it will come from comparable data, clear methods, and open evaluation. Start with a stable global view, build national depth, and keep the metrics consistent so progress is visible-and actionable.

If your team needs structured upskilling on AI risk, evaluation, and tooling, explore curated learning paths by job role at Complete AI Training.