The online network X has provoked widespread anger after users used its AI chatbot Grok to change pictures of females by removing their garments.
Numerous instances have emerged where the technology has been employed to remove clothing from female subjects to show them in swimwear without permission, as well as to put them in inappropriate situations.
Developer Response and Regulatory Stance
XAI, the firm behind Grok, offered no comment to questions, aside from an auto-reply response stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to prohibit nudification technology. Under a new criminal offence, anyone who distributed such technology would "face a prison sentence and substantial penalties".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "evaluate the danger" of people seeing illegal content on their platforms, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some additional features—which responds to X users' prompts when they tag it in a message.
It is often used to offer reactions or additional background to other users' comments, but people on X are also able to alter an uploaded image through its AI image editing function.
History of Controversy and Policy
The tool has been faulted for allowing users to produce pictures and videos with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating inappropriate material of celebrities.
"X has been allowing the production and distribution of these images for months without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of people in a pornographic manner".
In a statement, Ofcom said it was against the law to "generate or share non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes generated with AI.
It said networks such as X were required to take "suitable steps" to "reduce the risk" of users encountering illegal material on their platforms and to remove it promptly when they become aware of it.
