New research testing ChatGPT, Claude and Perplexity on financial queries from vulnerable users found the AI models give practical, structured advice - but consistently miss red flags that a human advisor would catch. The findings carry direct implications for anyone in finance who handles client money or builds tools that touch it.
Researchers at an Australian university created hypothetical scenarios involving financial vulnerability: a graduate saving for a home deposit during a cost-of-living crisis, a pregnant woman planning maternity leave with a partner who keeps finances separate, and a single parent considering a cousin's crypto tip. Each scenario was run five times through each model in clean browser sessions to check consistency.
The blind spot in the outputs
The models excelled at breaking down complex concepts and producing step-by-step plans. But they failed to adjust those plans to each user's stated situation. ChatGPT gave detailed practical steps but did not pick up on the vulnerability clues "hidden" in the prompts. Claude leaned toward self-guided planning. Perplexity most often told users to see a professional, but gave the thinnest responses.
The most stark example: when a single parent asked about cryptocurrency, ChatGPT openly described how to start investing and recommended specific coins for beginners, despite the prompt stating a modest income and two children. Perplexity and ChatGPT also both assumed a pregnant woman's partner would help with household costs after the birth - even though the prompt explicitly said the partners keep their finances separate.
"We saw the output reflecting social stereotypes where mothers are strongly associated with parenting while fathers are strongly associated as material providers," the researchers wrote.
Why the model misses the point
Another finding from the study: models are trained on data that contains human bias. When they explain a recommendation, people are more likely to trust it blindly regardless of accuracy. That combination is dangerous in money matters.
The researchers acknowledge AI is useful for fact-gathering and brainstorming if the user is already financially literate. But the models won't push back or read between the lines.
"A human financial advisor would immediately flag these areas for consideration: a modest income, children, a high-risk investment, and anecdotal advice from the cousin," the study says. "A human advisor would first measure the user's overall position, time horizon, risk appetite and investment objectives."
Why this matters for finance professionals
If you work in finance, this study is a practical checklist for your own assumptions. The AI models failed because they could not infer, probe, or adjust when a client's personal context created risk. You serve anyone who might use ChatGPT for a quick answer - which means your advice should be valuable precisely where the AI's ends: reading the person in the room.
The study also points to what hands-on systems need before they can be trusted with sensitive consumer data. The researchers call for clear regulatory frameworks and strict transparency standards around what AI models do with financial information shared with them. Consumers deserve to know how their data is stored, whether the AI keeps a "memory" of their finances, and who can access it.
Until AI can grasp emotional and situational reality, the most important skill remains the same as it always was: asking whether a piece of advice makes sense for the particular person asking for it. That is still human work.
Your membership also unlocks:
