A new study from the University of Chicago's Booth School of Business suggests that AI-generated email copy is nearly as effective as human-written copy - at a fraction of the cost. The findings have serious implications for copywriters and other writing professionals who produce marketing content.
The researchers, Booth professor Jean-Pierre Dubé and PhD student Ariel Xu, worked with Wine Access, an online wine retailer. In a randomized 14-day trial, 27,500 customers who subscribed to the brand's twice-daily email newsletter were split into four groups: those who received staff-written emails, AI-written emails, AI-written emails that were human-edited, and a control group that received no emails.
What the trial found
The results were close on performance, with staff-written emails showing a slight edge. Purchase incidence was 12.0% for staff-written emails, 11.5% for AI-written, and 11.6% for AI-written, human-edited. Gross profit per customer followed a similar pattern: $10.50 for staff, $10.22 for AI, and $9.44 for AI with human editing.
But the labor costs tell a different story. The researchers calculated that staff-written emails cost $375,000 in annual labor, compared with $32,250 for AI-written newsletters and $63,500 for AI-written, human-edited versions. That gap is what makes the business case for automation so compelling.
"Once the annual overhead costs are considered, there is scope for WA [Wine Access] to downsize its writing teams and shift some of the creative content design to AI," the researchers wrote in the Chicago Booth Review.
Staff writers also took around three hours to produce an email. The AI took seconds.
The AI wrote tighter copy
The AI-written messages were shorter - 339 words on average, versus 393 for staff-written ones - and got readers to the "Buy Now" button more quickly. Staff-written emails had about 150 words before the call-to-action, compared with 124 for AI-written emails.
"This result is striking as it suggests the direct and hybrid implementations of the LLMs are generating email copy that is approximately as effective as the team of human writers," the researchers wrote.
Anthropic's Claude performed particularly well. According to the study, Claude "significantly edged out staff writers on gross profits for those wines in the newsletters," with projected annual profits of about $5.5 million for bottles mentioned by Claude versus roughly $5 million for those mentioned by human writers.
The stylistic differences were clear. Claude produced this line: "When a 99-point wine appears at $40, you pay attention." The human copywriters wrote: "Charles Smith's Motor City Kitty Syrah is one of the must-have red wines of the West Coast."
Not a simple replacement
The study doesn't suggest all copywriting jobs vanish tomorrow. Human-edited AI copy underperformed pure AI copy in some metrics, but it still came in far below the cost of fully staff-written emails - and within close range of staff performance.
For professionals whose livelihoods depend on writing email campaigns, the implications are straightforward: if an AI can produce copy that performs comparably to yours - at roughly one-tenth the cost - then the question is not whether you'll be using these tools, but how much of the work they'll be doing.
Why this matters for writers
You don't need to be an email copywriter to see where this points. The study shows the gap between AI and human output is now small enough that cost decides the issue. Both people who use LLMs and the company's budget considerations will shape how writing work gets assigned.
The most useful takeaway for writers who work: crew learn to edit AI output fast and well. The hybrid approach in this study - AI-written, human-edited - is the workflow that actually gets you retained. Understanding how to improve on AI's drafts, not just produce original copy, is the skill that this job market is moving toward. For writers who want to assess their own position, guidance for writers using this tech continues to evolve as these studies accumulate.
One caution the study itself makes clear: Claude can generate effective promotional copy, but it can't taste the wine. That's the design. Anything a human writer creates without tasting the product, or the customer, or the segment, is subject to someone smarter about costs calling it into question.
Your membership also unlocks:
