The social media platform X has provoked significant anger after users used its artificial intelligence assistant Grok to alter photos of females by removing their garments.
Several cases have been seen where the tool has been used to remove clothing from women to render them in bikinis without consent, as well as to insert them in sexualized scenarios.
Developer Reaction and Regulatory Position
XAI, the firm behind Grok, remained silent to requests for comment, aside from an automatically-generated response stating "traditional news outlets lies".
A government spokesperson stated that legislation is being brought forward to outlaw nudification technology. Under a upcoming law, anyone who provided such tools would "face a prison sentence and heavy fines".
"X or Grok has the ability to stop these forms of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Platform Capabilities
The regulator Ofcom said platforms must "consider the danger" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some premium functions—which responds to X users' requests when they tag it in a tweet.
It is often used to give commentary or more context to other posters' comments, but people on X are also can alter an uploaded photo through its AI image editing feature.
Background of Criticism and Rules
The tool has been condemned for permitting users to produce photos and clips with nudity and sexually suggestive content, and it was previously alleged to have been involved in creating graphic material of celebrities.
"X has been allowing the creation and sharing of these images for months without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "portraying images of people in a pornographic way".
In a statement, Ofcom said it was against the law to "generate or share deepfakes or illegal content" and clarified this included sexual deepfakes created with AI.
It said networks such as X were mandated to take "necessary actions" to "minimize the danger" of users encountering illegal content on their services and to delete it quickly when they are notified of it.
