AI-designed DNA slips past biosecurity screens, Science study warns

AI Can Design Dangerous DNA That Slips Past Biosecurity Screens

Artificial intelligence can be used to design DNA for dangerous proteins in a way that reliably bypasses the biosecurity screening measures used by DNA manufacturers. This capability presents a new and significant challenge for the scientific community.

Major biotech companies that produce custom DNA have safeguards to prevent malicious actors from acquiring genes for agents like smallpox or anthrax. A new study published in the journal Science demonstrates how AI can be used to circumvent these very processes.

"Paraphrasing" Malicious Code

A team of AI researchers used protein-design tools to "paraphrase" the DNA codes of known toxic proteins. This process rewrites the genetic sequence in ways that could preserve the protein's structure and, potentially, its function, according to Eric Horvitz, Microsoft's chief scientific officer.

The AI program generated over 75,000 variants of hazardous proteins. When tested, these "reformulated sequences slipped past the biosecurity screening systems used worldwide by DNA synthesis companies to flag dangerous orders," Horvitz stated.

A software fix was quickly developed and applied to the screening software. However, it is not a perfect solution, as it still failed to detect a small fraction of the AI-generated variants.

The Challenge of Open Science

AI-powered protein design is making significant contributions to medicine and public health. "Yet like many powerful technologies, these same tools can often be misused," says Horvitz. This highlights a long-standing debate in biology: the risk of powerful DNA tools being used to create biothreats.

The scientific tradition of open discussion and independent replication is now in conflict with the need to secure potentially hazardous information. In an unprecedented move, the study's researchers and the publishing journal decided to restrict access to their data and software.

They enlisted a third-party non-profit, the International Biosecurity and Biosafety Initiative for Science, to manage access. This organization will determine who has a legitimate need for the information, creating a new model for managing risk in scientific publications.

Identifying and Fixing Vulnerabilities

Arturo Casadevall, a microbiologist at Johns Hopkins University, praised the work for identifying and attempting to correct these system vulnerabilities. The real trouble, he notes, is the unknown. "What vulnerabilities don't we know about that will require future corrections?"

Casadevall also points out that the team did not perform lab work to create the AI-designed proteins and test their biological activity. Such work is an important reality check but would be difficult to conduct, as it could be prohibited by international treaties against the development of biological weapons.

Getting Ahead of an AI "Freight Train"

This is not the first exploration of AI's potential for malevolent use. A few years ago, a different team tasked an AI with generating novel molecules with the properties of nerve agents. In under six hours, it produced 40,000 molecules, including known chemical weapons like VX and many new, potentially more toxic compounds.

That team also withheld its data, considering it too dangerous to publish. "They simply said, we're telling you all about this as a warning," notes David Relman, a researcher at Stanford University.

Relman sees this latest study as laudable but also as an illustration of a much larger problem. "How do we get ahead of a freight train that is just evermore accelerating and racing down the tracks, in danger of careening off the tracks?"

A Reassuring Perspective

Despite these concerns, some industry experts offer a dose of reality. James Diggans of Twist Bioscience, a major DNA provider, states that in the past decade, his company has referred fewer than five orders to law enforcement. "This is an incredibly rare thing," he says.

He contrasts this with cybersecurity, where countless actors are constantly trying to breach systems. "The real number of people who are really trying to create misuse may be very close to zero," Diggans suggests. The screening systems are an important safeguard, but the low frequency of actual threats offers some comfort.