Hackers Sell Jailbroken AI Tools Built on Grok and Mixtral Models

AI Hacking Tools Powered by Grok and Mixtral Models Surface on Hacker Forums

Recent research from Cato Networks has uncovered that certain AI hacking tools sold online are built using commercial AI models from xAI’s Grok and Mistral AI’s Mixtral. These tools, found on underground hacker forums, offer uncensored capabilities that bypass traditional safety guardrails embedded in mainstream AI products.

While many commercial AI companies implement strict filters to prevent malicious use—such as generating malware code or instructions for harmful activities—an underground market thrives by selling uncensored AI variants. Known as “WormGPTs,” these tools are often assembled from open-source models and other components, enabling users to generate exploit code, hunt vulnerabilities, and automate offensive cyber tasks.

How These AI Tools Operate

One discovered variant, advertised on BreachForums in early 2025, operated as an “Uncensored Assistant” accessible via Telegram. Despite its benign description, analysis revealed it could produce phishing emails and PowerShell scripts designed to steal credentials on command. Researchers found it contained prompt-based guardrails that concealed the original system prompts, but by applying jailbreaking techniques, they revealed xAI’s Grok as the underlying engine.

This tool effectively acts as a wrapper around Grok, using system prompts that instruct the model to bypass its own safety restrictions and generate harmful content. Similarly, another variant marketed in late 2024 positioned itself as an AI specializing in cybersecurity and hacking topics. It assured potential buyers that it provided information on cyber attacks, vulnerability detection, and defensive strategies, while disclaiming responsibility for misuse.

Pricing and User Base

Access to these AI hacking tools comes at a cost, typically through subscription models priced around €550 (~$630) annually. More advanced private setups can cost upwards of €5,000 (~$5,740). The high price suggests these tools are primarily aimed at cybercriminals seeking to scale their operations efficiently.

Security Implications

The emergence of AI-powered hacking assistants highlights a growing challenge. Although AI can streamline certain hacking processes, intelligence agencies and major AI developers indicate that these tools have yet to significantly alter the threat landscape posed by nation-state actors.

Still, the availability of uncensored AI models lowers the technical barrier for cybercriminals to create sophisticated attacks. This trend underscores the need for continued vigilance and investment in cybersecurity defenses.

Further Learning

For IT professionals and developers interested in AI’s impact on cybersecurity—both positive and potentially harmful—exploring comprehensive AI courses can provide valuable insights. Resources like Complete AI Training’s latest AI courses offer practical knowledge on AI tools, prompt engineering, and ethical AI use.

• Understand how AI models are built and secured
• Learn prompt engineering techniques to optimize AI outputs
• Explore AI’s role in automating cybersecurity tasks

Staying informed about both AI capabilities and risks helps professionals prepare for evolving cyber threats effectively.