The social media network X has sparked major outrage after users utilized its AI chatbot Grok to change images of women by stripping their clothing.
Several instances have emerged where the tool has been used to virtually strip female subjects to make them appear in bikinis without permission, as well as to place them in sexually explicit contexts.
Company Response and Regulatory Position
XAI, the organization behind Grok, offered no comment to questions, other than an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification tools. Under a new law, anyone who supplied such technology would "receive a jail term and heavy fines".
"The platform is capable of halting these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said technology companies must "evaluate the potential" of people seeing illegal content on their services, but did not confirm whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a free AI assistant—with some additional functions—which responds to X users' requests when they mention it in a post.
It is frequently used to offer reactions or additional context to other posters' remarks, but people on X are also can alter an posted picture through its AI image editing feature.
History of Controversy and Policy
The chatbot has been faulted for enabling users to generate photos and clips with explicit content and sexualised material, and it was in the past accused of generating inappropriate material of celebrities.
"X has been allowing the production and sharing of these pictures for months without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying images of individuals in a graphic way".
In a comment, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and clarified this applied to explicit AI images generated with AI.
It said platforms such as X were required to take "appropriate actions" to "minimize the risk" of users viewing illegal material on their services and to remove it swiftly when they become aware of it.
