Brazil’s top electoral court moved this week to tighten rules on AI-generated campaign content weeks before its October 4 vote, as California lawmakers passed the first US state law governing how lawyers use generative AI in court.
But the same technology is inflicting a starkly different kind of harm on a far more vulnerable population: a new UNICEF-backed study estimates that over 1.1 million children aged 12 to 17 had AI-generated sexual images or videos made of them in a single year.
A Trigger; a Pattern
Brazil’s Tribunal Superior Eleitoral — the country’s top electoral court, and the body with final say over how the October 4 presidential race is conducted — is expected to rule this week on a clearer legal definition of what counts as a deepfake, part of a package that already bans AI candidate-recommendation chatbots, mandates content labeling, and imposes a blackout on new AI-generated material immediately around voting.
The trigger for the current urgency was concrete: an AI avatar of former President Jair Bolsonaro, who is barred from public communication under house arrest, appeared at his son Flávio Bolsonaro’s party convention in July and drew immediate legal challenges.
A study by Rio de Janeiro’s Institute of Technology and Society found that seven of the most widely used chatbots in Brazil ranked or ordered candidates in 90% of their responses — despite the explicit ban. Rules exist; enforcement, so far, is porous.
This is not a Brazil-specific anxiety.
A comparative study of eleven countries and jurisdictions found the same fault line recurring everywhere: high-trust democracies lean on voluntary codes, while countries with more polarization or recent experience of intrusive disinformation write binding law instead. The eleven, to wit:
- EU
- Norway
- Denmark
- Ireland
- UK
- South Korea
- Philippines
- India
- Uruguay
- US
And the pattern’s most severe instance wasn’t a near miss.
The same study points to Romania’s 2024 presidential election that was annulled outright, following a campaign involving AI-manipulated media and coordinated algorithmic manipulation — proof that the failure mode here isn’t hypothetical.
That same instinct (that voluntary trust in AI-generated material isn’t enough) has now reached an institution with nothing to do with elections. This week, California’s legislature passed SB 574, the first US state law governing how lawyers use generative AI, after a string of court sanctions against attorneys who filed AI-hallucinated case citations. Democratic state senator Tom Umberg, who introduced the bill and chairs the Senate judiciary committee, put the stakes in institutional terms:
“Our system, our courts, our judiciary has to rely on the integrity of the litigants and the advocates. And if they can’t do that, everything breaks down.”
What AI has Already Done to a Child Who Never Touched a Camera
The scale problem UNICEF documented this week is not primarily about artificial intelligence. Across 21 countries, an estimated 20 million children aged 12 to 17 (about one in five internet-using children surveyed) were subjected to at least one form of technology-facilitated sexual exploitation or abuse in a single year, according to Through Children’s Eyes, a report from the Disrupting Harm initiative run jointly by UNICEF, ECPAT International and INTERPOL.
Most of that harm happens through ordinary channels: unwanted sexual content, coercion into sexual conversations, images shared without consent. Most of it on mainstream platforms.
Within that figure sits a narrower, newer one.
As mentioned above, an estimated 1.1 million children aged 12 to 17 had AI-generated sexual images or videos created depicting them in the year prior to being surveyed — a statistic drawn from nine of the 21 countries, ranging from 0.4% of children surveyed in Montenegro to 4.3% in North Macedonia.
“It was during a school recess, and I opened [a photo received from a fake profile via Instagram]. They managed to take my photo, take my head and put it on someone else’s body and make it look like it was actually my photo, like it was me.”
— a young woman from Montenegro, in an interview with UNICEF researchers
The report notes plainly that these figures likely undercount the true scale; children may simply be unaware that AI-generated images of them exist at all.
UNICEF’s recommendation is correspondingly direct: governments should “update laws” to explicitly criminalize “the production of AI-generated sexual content depicting children,” alongside broader mandates that platforms detect and remove such material, including content generated by AI.
Set beside the electoral and legal cases above, the throughline holds: documented harm, not hypothetical risk, is what has moved regulators, courts and international bodies from guidance toward enforceable rule.
Unfortunately, that documented harm spans a population that never consented to being part of the story at all.
Aklatan’s news and analysis drills down to the structural mechanics, geopolitical shifts, and hidden constraints truly driving AI and Asian tech ecosystems and knowledge work.
See coverage span here: Aklatan’s News and Analysis
Generative AI Transparency:
This article was written primarily with generative AI, specifically SupraGraphos’ A.C.E. News Module. Reviewed with human post-editing, all sources and claims are confirmed as of the time of writing.