Financial firms and frontier labs outline how to give agents useful autonomy without losing control

Financial firms and frontier AI labs are converging on a shared principle: safe agent autonomy comes from where you place controls, not how many you impose. At the AI Engineer New York conference this week, OpenAI, Anthropic, Two Sigma, and Bloomberg each presented frameworks that let AI agents act on consequential financial work while keeping human judgment at specific, high-value checkpoints.

Joseph Kim, Head of ChatGPT for Financial Services at OpenAI, opened his keynote with a concept he called the "threshold of usefulness" - the point where a model becomes capable and reliable enough to save you time. He predicted that in 2027, the financial industry will not only rely less on Excel and PowerPoint, but agents will handle "investment idea generation and trading."

Kim tied the rising capability of models directly to traceability. "In finance, you need to be able to check where a number came from and understand the context around it," he said. "We wanted that review to be a natural part of the experience. These are the kind of product details that help turn a model's capabilities into work that someone can use." He framed the human-in-the-loop question as a product design decision: what the user still needs to control, review, or approve - questions teams can start working on now.

The controls that let agents move

John Feminella, SVP Architecture at the hedge fund Two Sigma, addressed those exact product questions. He described five controls the firm has implemented to keep agents in check without limiting what models can do. Feminella rejected the idea that safety and innovation sit at opposite ends of a spectrum, and he was especially critical of "approval fatigue" - the pattern where a person rubber-stamps everything, which slows work and introduces risk when attention drifts.

Two Sigma's controls are composable: each one does one job, and together they let teams run agents with real autonomy at an appropriate safety level. One example is auto-mode classifiers that "make decisions about routine calls and then send a person only the requests that are most important, the ones that are most likely to make a difference." Classifiers handle ambiguous cases; deterministic rules cover the clear ones.

Identity, sandbox, and the boundary before action

Anthropic's presentation outlined a trust framework with three layers. Belinda Neal, Managing Director of Financial Services at Anthropic, said the company's new Claude for Financial Advisors product was built because "every so often, a model crosses a threshold. And when it does, a wave of working changes."

Keon Shahab, who works on applied AI at Anthropic, walked through the trust architecture: an identity and permissions layer that defines who the agent acts as, a sandboxed approval layer so someone signs off before action, and verification that the agent still works after variables shift over time. He illustrated the line between proposing and acting with a trade break investigation agent that recommends an order amendment but does not execute it. "We want it to just propose new amendments to our orders data, not actually make those amendments," Shahab said. "By building once with a foundational layer that you trust, you can work at speed to deploy many agents at scale."

Enforcing security outside the model

Andrey Rybka, Head of Architecture & Emerging Technologies in the Office of the CTO at Bloomberg, described real incidents where agents caused problems - including one case where an agent treated instructions embedded in a marketplace review as a command to stock a product. His recommended defenses include scoped identities, controls over tool calls, outbound traffic restrictions, audit trails, budgets, and kill switches.

"We want deterministic controls outside the model," Rybka said. "Check every call, log it, and keep a kill switch." His list also included input validation, prompt injection defenses, provenance tracking, and human approval for high-value actions.

Why this matters for finance executives and strategy leaders

The presentations make clear that frontier labs are building trust frameworks, but financial firms own the implementation. Two Sigma's Feminella captured the operational takeaway with an analogy: "Why do cars have brakes? It's not just so that you can stop, it's so that you can go fast and stop safely." For executives evaluating agent deployment in 2027, the work is to design controls - classifiers, sandboxed approval layers, deterministic guardrails - that enable speed rather than block it. The firms that treat safety as a product design problem, not a compliance checklist, will be the ones that move first on investment idea generation and trading workflows.