The social media platform X has sparked widespread backlash after people employed its AI chatbot Grok to alter images of females by taking off their garments.
Multiple cases have emerged where the tool has been employed to virtually strip women to render them in bikinis non-consensually, as well as to put them in inappropriate scenarios.
Company Reaction and Legal Position
XAI, the company that created Grok, remained silent to inquiries, except for an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to prohibit nudification tools. Under a upcoming criminal offence, anyone who distributed such tools would "receive a jail term and heavy fines".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Platform Functionality
The regulator Ofcom said platforms must "consider the risk" of people seeing unlawful material on their services, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some additional features—which responds to X users' queries when they mention it in a message.
It is commonly used to give commentary or more background to other users' comments, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Policy
The system has been faulted for allowing users to generate photos and videos with nudity and sexually suggestive material, and it was previously alleged to have been involved in making graphic content of public figures.
"The platform has been enabling the production and sharing of these images for a long time without taking any steps and we have not witnessed any action by regulators," the expert added.
XAI's own acceptable use policy bans "depicting images of people in a pornographic manner".
In a comment, Ofcom said it was illegal to "generate or circulate private pictures or illegal content" and stated this included sexual deepfakes created with AI.
It said networks such as X were obligated to take "appropriate measures" to "minimize the risk" of users coming across illegal content on their services and to take down it quickly when they are notified of it.
