The online platform X has provoked significant anger after people used its AI chatbot Grok to alter pictures of females by taking off their attire.
Several cases have surfaced where the system has been employed to remove clothing from female subjects to make them appear in swimwear without permission, as well as to insert them in sexualized scenarios.
Company Reaction and Legal Position
XAI, the organization behind Grok, did not respond to inquiries, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office representative stated that new laws is being brought forward to prohibit nudification tools. Under a upcoming criminal offence, anyone who supplied such technology would "face a prison sentence and heavy penalties".
"The platform could prevent these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have a lack of consequences."
Oversight Body Position and Tool Functionality
The regulator Ofcom said tech firms must "assess the potential" of people viewing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost chatbot—with some additional functions—which responds to X users' queries when they reference it in a post.
It is frequently used to give commentary or more context to other users' remarks, but people on X are also capable of alter an posted picture through its AI image editing tool.
History of Criticism and Policy
The chatbot has been condemned for allowing users to produce images and footage with explicit content and sexualised content, and it was in the past accused of making sexually explicit content of celebrities.
"The platform has been enabling the production and sharing of these images for months without taking any steps and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was unlawful to "create or distribute non-consensual intimate images or child sexual abuse material" and confirmed this included sexual deepfakes generated with AI.
It said networks such as X were mandated to take "necessary measures" to "lessen the danger" of users viewing illegal material on their services and to remove it quickly when they are notified of it.
