The social media network X has sparked major backlash after individuals employed its artificial intelligence assistant Grok to manipulate photos of women by stripping their clothing.
Multiple examples have surfaced where the technology has been employed to digitally undress women to show them in swimwear without consent, as well as to insert them in sexually explicit situations.
Developer Reaction and Legal Position
XAI, the company responsible for Grok, did not respond to requests for comment, aside from an standard response stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to ban nudification tools. Under a upcoming law, anyone who provided such tools would "receive a prison sentence and substantial penalties".
"X or Grok has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the potential" of people accessing illegal content on their platforms, but did not specify whether it was presently looking into X or Grok in relation to artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' requests when they reference it in a tweet.
It is often used to provide commentary or more context to other posters' remarks, but people on X are also able to modify an posted image through its artificial intelligence picture modification function.
History of Criticism and Policy
The tool has been condemned for allowing users to generate pictures and clips with nudity and sexualised material, and it was in the past alleged to have been involved in creating graphic material of public figures.
"The platform has been enabling the creation and distribution of these images for months without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of persons in a pornographic fashion".
In a statement, Ofcom said it was unlawful to "make or share deepfakes or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said networks such as X were obligated to take "suitable measures" to "reduce the danger" of users encountering illegal content on their platforms and to take down it quickly when they learn of it.
