The online platform X has sparked significant backlash after users used its artificial intelligence assistant Grok to alter photos of females by taking off their garments.
Multiple cases have surfaced where the system has been employed to remove clothing from female subjects to render them in bikinis non-consensually, as well as to place them in inappropriate situations.
Developer Reaction and Legal Stance
XAI, the organization behind Grok, remained silent to inquiries, other than an automatically-generated response stating "legacy media lies".
A Home Office representative stated that legislation is being introduced to outlaw image stripping tools. Under a upcoming law, anyone who distributed such tools would "receive a prison sentence and significant penalties".
"The platform could prevent these types of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Functionality
The communications watchdog Ofcom said tech firms must "evaluate the danger" of people seeing illegal content on their platforms, but did not confirm whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a free chatbot—with some premium features—which responds to X users' requests when they tag it in a tweet.
It is frequently used to provide commentary or further context to other users' comments, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Rules
The system has been faulted for enabling users to create pictures and clips with nudity and sexualised material, and it was in the past accused of making graphic material of celebrities.
"X has been enabling the production and spread of these pictures for a long time without taking any action and we have not witnessed any challenge by oversight bodies," the expert added.
XAI's own terms of service bans "showing likenesses of persons in a pornographic way".
In a statement, Ofcom said it was illegal to "generate or distribute non-consensual intimate images or illegal content" and stated this applied to sexual deepfakes made with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the risk" of users viewing illegal material on their platforms and to delete it quickly when they become aware of it.
