Conference sparks controversy by going fully AI for authorship and peer review
Agents4Science 2025 is calling for papers written and peer-reviewed entirely by AI. The Open Conference of AI Agents for Science takes place tomorrow and "represents a new approach to research conferences," according to its organisers.
The website says the event will test whether AI can independently generate scientific insights and uphold quality via AI-driven peer review. It's framed as an experiment to map where AI systems excel, and where they still fall short.
Why this matters for working researchers
If AI handles both authorship and review, core parts of scientific practice get stress-tested: authorship, accountability, reproducibility, and trust. The outcome won't just be about novelty-it will set expectations for how labs, journals, and funders treat AI contributions.
- Authorship and credit: Who is accountable for errors, misconduct, or harm if no human is listed as an author?
- Provenance: Can readers trace data sources, training corpora, prompts, and model versions?
- Reproducibility: Are code, data, and generation settings available to replicate results?
- Bias and safety: How are model biases detected, reported, and mitigated?
- Peer review integrity: What standards define an "AI reviewer," and how is quality audited?
Critics are blunt
Raffaele Ciriello, Senior Lecturer in Business Information Systems at the University of Sydney, said the conference "is at best, an amusing curiosity and, at worst, an unfunny parody of what science is meant to be." He added, "If the authors and reviewers are AI, then perhaps the conference attendees should be AI too, because no human should mistake this for scholarship."
"There is, of course, a legitimate discussion to be had about how AI tools can assist scientists in analysing data, visualising results, or improving reproducibility," Ciriello said. "But a conference built fully on AI-generated research reviewed by AI reviewers embodies a dangerous kind of technocratic self-parody."
Hussein Abbass, from the School of Engineering and Information Technology at UNSW-Canberra, said "authorship is a sacred section in academic publications." He noted, "AI has been making scientific discoveries since its inception. Authorship is a different pool game! As an advocate for AI and as an AI psychologist who designs and diagnoses AI cognition and behaviour, there is a sacred line I do not cross."
Ali Knott, Professor in Artificial Intelligence at Victoria University of Wellington, said the conference is recognised as an experiment. "Its purpose (as I understand it) is to evaluate the possibility of AI authors and AI reviewers, rather than to advocate for AI systems in these roles," he said. "It is far too early for that kind of advocacy - I'm sure most researchers would agree."
What to watch if you attend or read the proceedings
- Methods transparency: Clear prompts, system settings, model versions, and rationale chains disclosed.
- Human oversight: A statement of any human guidance, curation, or verification.
- Data and code: Public links to datasets, generation scripts, seeds, and eval harnesses.
- Evaluation: Pre-registered protocols, baselines, error analyses, and independent benchmarks.
- Reviewer criteria: How AI reviewers are configured, validated, and audited for consistency.
- Ethics: Bias tests, safety checks, and risk/impact assessments included with results.
- Retraction and appeals: Defined process if errors or misconduct surface post-publication.
Practical next steps for labs and PIs
- Set an internal policy on permissible AI use, disclosure, and version control for manuscripts and reviews.
- Adopt clear authorship criteria and contributorship taxonomies; align with established guidance such as ICMJE and COPE.
- Require AI provenance sections in lab papers: models, prompts, datasets, safety filters, and human checks.
- Create a review checklist for AI-assisted manuscripts: reproducibility, data availability, and bias reporting.
- Pilot AI-assisted workflows on low-stakes projects before touching high-impact studies.
- Invest in training so staff can evaluate AI outputs and failure modes with a critical eye. For structured options, see AI courses by job role.
Bottom line
This conference is an experiment that puts long-standing norms under pressure. Healthy scrutiny is warranted. Whether AI-only authorship and review can meet the standards scientists expect will depend on transparency, accountability, and the willingness to publish everything needed for others to verify the work.
Your membership also unlocks:
