The online platform X has provoked widespread backlash after people used its AI chatbot Grok to change pictures of women by stripping their attire.
Several cases have been seen where the system has been employed to digitally undress female subjects to make them appear in swimwear without permission, as well as to place them in sexually explicit contexts.
Developer Reaction and Regulatory Stance
XAI, the organization responsible for Grok, did not respond to requests for comment, other than an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that new laws is being introduced to ban image stripping software. Under a upcoming criminal offence, anyone who supplied such technology would "face a jail term and significant penalties".
"The platform has the ability to stop these types of misuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The communications watchdog Ofcom said technology companies must "assess the potential" of people viewing illegal content on their networks, but stopped short of confirming whether it was at this time looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' requests when they mention it in a post.
It is often used to offer reactions or further background to other users' comments, but people on X are also capable of alter an posted picture through its AI image editing tool.
Background of Criticism and Rules
The chatbot has been faulted for allowing users to produce photos and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in generating graphic content of public figures.
"X has been permitting the generation and distribution of these images for months without taking any action and we have yet to see any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "showing likenesses of persons in a graphic way".
In a statement, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and clarified this covered sexual deepfakes created with AI.
It said platforms such as X were required to take "appropriate measures" to "lessen the risk" of users encountering illegal content on their services and to delete it swiftly when they learn of it.
