San Francisco’s city attorney has issued a stern cease-and-desist letter to Meta, demanding the tech giant cease "allowing" paid advertisements that feature AI-generated child sexual abuse content. The letter also seeks an explanation for how such explicit material has repeatedly appeared on Meta’s prominent platforms, including Facebook, Instagram, and Threads. This aggressive legal action follows a damning report by WIRED, which revealed Meta’s extensive failure to intercept hundreds of deceptive advertisements.
Troubling Discovery: AI-Generated Exploitation on Meta Platforms
The crux of the issue lies in a series of advertisements that transformed still images of minors into short video clips, capable of depicting them in sexually explicit acts. These disturbing ads, discovered by researchers at the Tech Transparency Project (TTP), were not mere abstract creations; some originated from actual photographs of real children. Most alarmingly, one such image was identified as belonging to a member of a European royal family, underscoring the potential reach and sensitivity of the exploited content.
Upon clicking these ads, users were reportedly directed to download AI image- and video-generation applications. The nature of these applications is deeply concerning, as they can be exploited to digitally undress individuals or create other forms of nonconsensual intimate imagery, raising significant privacy and safety concerns for all users.
Scale and Persistence of the Problem
The TTP’s investigation uncovered a disturbing pattern of repeated uploads of identical advertisements, indicating a systemic failure in Meta’s content moderation processes. In total, these deceptive ads reached over 29,000 accounts across the European Union. Their reach extended beyond Europe, also targeting accounts in the United States, Australia, and India. This widespread dissemination occurred despite Meta’s public assertions of a zero-tolerance policy towards child exploitation.
The initial WIRED report in early August highlighted a batch of 53 such advertisements. However, the TTP’s subsequent findings revealed an even more alarming escalation, with over 250 additional ads appearing on Meta’s platforms after this initial disclosure. This persistence, even after the issue had been brought to light, amplifies the severity of the situation and the inadequacy of Meta’s response.
San Francisco’s Legal Challenge
San Francisco City Attorney David Chiu articulated the gravity of the situation in a statement to WIRED: "Meta has failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them. These ads are deeply disturbing and cause harm to real children. Meta claims to have zero tolerance for child exploitation, yet the scale and persistence of these issues suggest current efforts are woefully inadequate. This is unacceptable, and Meta must do better."
The four-page cease-and-desist letter, sent to Meta’s legal counsel, directly challenges the company’s advertising practices. "No company—particularly one that claims all advertisements are reviewed and approved prior to distribution and then accepts payment for that distribution—should permit its advertising systems to be used in this manner," Chiu wrote. The letter meticulously questions the efficacy of Meta’s ad policies and safety systems, demanding a comprehensive explanation of how these advertisements bypassed moderation and what steps are being taken to prevent future occurrences.
The letter also highlights that the findings do not appear to be isolated incidents, suggesting a deeper, systemic failure within Meta’s operational framework. The city attorney’s office has requested "prompt discussions" with Meta’s legal, child-safety, and advertising teams to address these critical issues.
Meta’s Jurisdictional Defense and San Francisco’s Rebuttal
In response to the legal action, Meta asserted that there is "no indication" the ads were displayed in San Francisco, thereby suggesting they fall "outside the city attorney’s jurisdiction." While Meta’s ad library provides data on ads published globally, it does not offer granular details on specific geographic placements within the United States.
However, the San Francisco City Attorney’s Office spokesperson, Alex Barrett-Shorter, vehemently rejected this defense. "San Francisco consumers can access the unlawful ads through Meta online spaces," Barrett-Shorter stated to WIRED, asserting that the matter falls squarely within the city’s purview.
Barrett-Shorter expressed astonishment at Meta’s stance, remarking, "We find Meta’s response astonishing. Meta appears less concerned about whether they’re guilty of exploiting children than whether San Francisco should say anything about it. Meta has bigger problems to solve here than determining our jurisdiction." This sharp retort underscores the city’s determination to hold Meta accountable, regardless of the company’s attempts to sidestep responsibility.
A History of Inadequate Responses?
Meta has previously stated that many of the ads identified by TTP had already been removed before the report was published, and all have since been removed for policy violations. A Meta spokesperson claimed that most of these ads had "fewer than 200 impressions from accounts" and that the total ad spend for the over 300 ads was "under $5,000." The company reiterated its commitment to not tolerating "nudify apps or any kind of child exploitation, whether real or AI-generated."
However, the persistence of the issue and the sheer volume of ads suggest that Meta’s preventative measures are insufficient. The TTP’s findings indicate that Meta did not respond to reports of abusive ads for over a week in some instances, allowing them to be viewed by hundreds more people. Furthermore, when some ads were eventually removed, they were not initially flagged as containing child sexual abuse content, according to TTP and WIRED’s review of Meta’s ad library.
Demands for Systemic Change and Transparency
The cease-and-desist letter demands that Meta not only halt the current ads but also provide a detailed explanation of how these ads evaded its moderation systems. It seeks information on Meta’s protocols for escalating and reporting suspected child sexual exploitation to the National Center for Missing and Exploited Children (NCMEC). Additionally, the letter requests details on how Meta addresses advertisers involved in such content and identifies repeat offenders.
Chiu’s letter emphasizes that these findings point to "broader systemic problems" in Meta’s detection of child sexual abuse in advertising, raising "significant legal questions" for the company. The primary objective, as stated by Chiu, is not merely to identify past failures but to ensure Meta implements robust systems that prevent the sexual exploitation of children through paid advertisements and that the company reliably meets its legal obligations when alerted to such exploitation.
Meta’s Policies and Ongoing Challenges
Meta’s community standards and advertising policies explicitly prohibit child sexual exploitation, abuse, or nudity, as well as adult exploitation and nudity. The promotion of "nudify" apps is also forbidden. The company claims that all advertisements undergo review through automated and, at times, manual processes before distribution. Despite these policies, Meta has previously removed hundreds of thousands of ads for nudify apps and has even pursued legal action against companies associated with such platforms.
However, researchers continue to identify thousands of ads that direct users to harmful applications and contain sexually explicit content. Even after Meta’s stated actions against nudify apps, WIRED’s reporting during the TTP findings uncovered hundreds of "nudification" ads still running on Meta’s platforms.
Katie Paul, director of the Tech Transparency Project, has stated that Meta is "not effectively" addressing the issue, with ads featuring child sexual abuse continuing to appear on its platform. "Meta has claimed it takes a hard stance against child sexual abuse material, but since TTP’s September 8 report published, more brand-new ads have run featuring some of the same children TTP sent to Meta," Paul remarked. TTP has reported these ads to both Meta and NCMEC, noting that some were already removed. Meta has not yet provided an immediate response regarding the continued appearance of new ads.
"The findings are clear," Paul concluded. "Meta is not effectively addressing this issue." The city attorney’s office has requested a formal response from Meta within 28 days, signaling a crucial period for the social media giant to demonstrate a commitment to rectifying these deeply concerning failures. The implications of this ongoing situation extend beyond Meta, raising broader questions about the responsibility of tech platforms in policing the content they profit from and protecting vulnerable individuals from online exploitation.
