The online network X has caused widespread anger after individuals utilized its artificial intelligence assistant Grok to manipulate pictures of females by stripping their clothing.
Several examples have surfaced where the system has been used to remove clothing from female subjects to show them in swimwear non-consensually, as well as to put them in inappropriate scenarios.
Developer Reaction and Regulatory Position
XAI, the organization that created Grok, offered no comment to inquiries, other than an auto-reply response stating "traditional news outlets lies".
A government spokesperson stated that new laws is being introduced to outlaw image stripping software. Under a new criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"X or Grok could prevent these forms of misuse if they wanted to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said platforms must "consider the risk" of people accessing unlawful material on their services, but did not specify whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost chatbot—with some additional features—which responds to X users' requests when they reference it in a post.
It is often used to offer commentary or further context to other posters' comments, but people on X are also capable of edit an posted image through its artificial intelligence picture modification feature.
History of Controversy and Policy
The chatbot has been criticised for enabling users to create pictures and videos with explicit content and sexually suggestive content, and it was previously accused of making graphic material of celebrities.
"X has been enabling the generation and distribution of these images for a long time without taking any steps and we have not seen any action by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting images of persons in a pornographic fashion".
In a comment, Ofcom said it was illegal to "generate or circulate deepfakes or child sexual abuse material" and confirmed this included sexual deepfakes made with AI.
It said networks such as X were required to take "suitable measures" to "minimize the risk" of users encountering unlawful content on their services and to delete it promptly when they learn of it.
