The online platform X has caused major backlash after people used its artificial intelligence assistant Grok to alter pictures of women by taking off their garments.
Multiple instances have been seen where the tool has been used to virtually strip women to show them in swimwear without consent, as well as to insert them in sexually explicit scenarios.
Developer Reaction and Legal Position
XAI, the company that created Grok, did not respond to requests for comment, other than an standard response stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to prohibit image stripping tools. Under a new criminal offence, anyone who supplied such technology would "be subject to a prison sentence and heavy fines".
"The platform could prevent these types of abuse if they chose to," said a law professor, adding they "seem to have a lack of consequences."
Regulator Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the danger" of people accessing illegal content on their platforms, but did not confirm whether it was presently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' prompts when they reference it in a message.
It is often used to provide commentary or additional background to other posters' comments, but people on X are also able to edit an uploaded image through its artificial intelligence picture modification feature.
Background of Controversy and Policy
The chatbot has been criticised for allowing users to produce images and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in making sexually explicit material of celebrities.
"The platform has been allowing the generation and spread of these images for months without taking any steps and we have not seen any intervention by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing likenesses of individuals in a graphic manner".
In a statement, Ofcom said it was illegal to "create or share deepfakes or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said networks such as X were obligated to take "suitable actions" to "lessen the risk" of users encountering unlawful content on their services and to take down it promptly when they are notified of it.
