The social media network X has sparked major anger after people used its AI chatbot Grok to manipulate images of females by removing their clothing.
Multiple examples have surfaced where the system has been employed to digitally undress women to make them appear in swimwear without permission, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to inquiries, other than an automatically-generated response stating "legacy media lies".
A government spokesperson stated that new laws is being proposed to ban nudification tools. Under a new law, anyone who distributed such technology would "be subject to a jail term and heavy penalties".
"X or Grok is capable of halting these forms of abuse if they desired to," said a law professor, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said technology companies must "consider the potential" of people viewing unlawful material on their services, but did not confirm whether it was at this time looking into X or Grok in regarding AI-generated pictures.
Grok is a free chatbot—with some additional features—which responds to X users' queries when they mention it in a post.
It is frequently used to provide reactions or additional background to other posters' remarks, but people on X are also capable of modify an uploaded picture through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been condemned for allowing users to create photos and footage with explicit content and sexually suggestive material, and it was previously alleged to have been involved in making inappropriate content of public figures.
"The platform has been allowing the creation and sharing of these pictures for months without taking any action and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of people in a pornographic fashion".
In a statement, Ofcom said it was illegal to "make or share deepfakes or illegal content" and stated this applied to explicit AI images made with AI.
It said services such as X were obligated to take "suitable measures" to "minimize the risk" of users coming across illegal material on their platforms and to take down it promptly when they are notified of it.
