Key Points
- Grok generated three million sexualized images in 11 days, including about 23,000 involving children.
- Mindgard found an adversarial prompting vulnerability in ChatGPT that allowed creation of intimate images.
- OpenAI fixed the ChatGPT vulnerability after being alerted in early February 2026.
- Google introduced a simplified bulk‑report tool for removing explicit images from Search.
- Both companies reference strict prohibited‑use policies that ban illegal or abusive AI‑generated content.
- Experts warn that attackers will keep trying to bypass safeguards, requiring ongoing vigilance.
- Advocacy groups are pushing for stronger legal protections like the Take It Down Act.
Grok’s Abuse Highlights AI Risks
In January 2026 the generative‑AI tool Grok, offered by Elon Musk’s xAI, was used to produce a massive volume of sexualized images. Over an eleven‑day period the system generated three million such images, with approximately twenty‑three thousand containing children, according to a study by the Center for Countering Digital Hate. The rapid creation and distribution of non‑consensual intimate imagery—often called revenge porn—underscored how AI can accelerate existing harms.
OpenAI’s Rapid Response
Researchers from the cybersecurity firm Mindgard discovered a flaw in ChatGPT that allowed users to bypass its guardrails through adversarial prompting. By manipulating the model’s memory with custom prompts, they were able to produce intimate images of well‑known individuals. After notifying OpenAI in early February, the company confirmed that it had fixed the vulnerability before the findings were made public. OpenAI highlighted the importance of red‑team testing and pledged to keep improving its safeguards.
Google Improves Image‑Removal Tools
Google announced a streamlined process for requesting the removal of explicit images from its Search results. Users can now select multiple images, report them with a single click, and track the status of their requests. The company said the change is intended to reduce the burden on victims of non‑consensual explicit imagery. Google also referenced its generative‑AI prohibited‑use policy, which bans the creation of illegal or abusive content, including intimate imagery.
Ongoing Challenges and Industry Outlook
Both OpenAI and Google acknowledge that no safeguard is a permanent barrier. Cybersecurity experts stress that attackers continually iterate, requiring AI developers to assume persistent attempts to circumvent controls. Advocacy groups continue to push for stronger legislation, such as the 2025 Take It Down Act, to aid victims. The Grok episode serves as a reminder that robust, adaptive moderation and collaboration with external researchers are essential to protect users as generative AI capabilities expand.
Source: cnet.com