AI news ·
Amnesty International says generative AI companies use unlawful data scraping that violates privacy rights
Amnesty International accuses major AI companies of scraping billions of personal data points without consent to train systems like GPT-3, Gemini, and Llama. The group calls the practice unlawful and wants governments to ban AI built on such data.
Major AI Companies Built Systems on Unlawfully Scraped Data, Amnesty Says
Amnesty International released a briefing accusing leading AI companies of extracting billions of personal data points from the web without consent to train generative AI systems, practices the organization says constitute unlawful mass privacy violations.
The research examined generative AI and LLM systems including OpenAI's GPT-3, Google's Gemini, Meta's Llama, DeepSeek, and tools from Midjourney and Stable Diffusion. All rely on automated web scraping-extracting data from websites at scale-to gather training material from billions of public posts and images without explicit permission from the people who created or appear in them.
The scraping includes personal data such as images and social media activity. This practice, Amnesty argues, violates privacy by design principles and opens systems to human rights abuse from inception.
Bias Embedded in Training Data
Because training datasets pull directly from the web, they inherit real-world biases. Amnesty found that racial, gender, and cultural biases appear consistently in AI outputs, with negative stereotypes and prejudices amplified as models scale.
Historically marginalized communities face particular risk. The systems can also influence user beliefs through predictive suggestions, raising concerns about freedom of thought, especially in larger models trained on expansive datasets.
Environmental and Resource Costs
Larger AI models demand more energy-intensive chips, bigger data centers, and greater water and electricity consumption. Google reported a 48 percent increase in greenhouse gas emissions since 2019, tied to data center and supply chain operations. Microsoft's emissions rose 29 percent between 2020 and 2024 for similar reasons.
Communities in Chile, Mexico, and Arizona have resisted data center construction in areas already facing drought and electricity shortages. The infrastructure needs of AI production disproportionately affect lands and resources belonging to marginalized populations.
What Amnesty Is Demanding
The organization called on governments to ban standalone generative AI systems built using unlawful web scraping. Companies must stop non-consensual scraping of personal data for training, and states should hold them accountable for human rights violations tied to their design choices.
Amnesty wrote to Google, OpenAI, Meta, Stability AI, Midjourney, and DeepSeek requesting responses. At publication, Microsoft, Amazon, Intel, OpenAI, and Meta had replied. Google and DeepSeek had not.
Understanding how these data analysis practices shape AI systems matters for anyone building or deploying these tools. The design choices companies make during training directly affect the outputs and harms their systems produce.