The online platform X has sparked widespread outrage after users utilized its AI chatbot Grok to alter images of females by removing their attire.
Multiple cases have been seen where the technology has been used to digitally undress women to show them in swimwear without permission, as well as to put them in sexualized situations.
Company Reaction and Legal Stance
XAI, the firm responsible for Grok, did not respond to requests for comment, other than an automatically-generated response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to prohibit nudification tools. Under a proposed criminal offence, anyone who provided such technology would "receive a prison sentence and substantial fines".
"The platform is capable of halting these types of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Oversight Body Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the risk" of people viewing unlawful material on their networks, but did not specify whether it was currently investigating X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some paid-for features—which answers X users' queries when they tag it in a tweet.
It is often used to provide commentary or additional background to other users' comments, but people on X are also can modify an posted image through its artificial intelligence picture modification function.
History of Criticism and Policy
The chatbot has been condemned for permitting users to create photos and videos with explicit content and sexualised material, and it was in the past accused of generating graphic content of celebrities.
"The platform has been allowing the creation and distribution of these pictures for months without taking any steps and we have yet to see any challenge by regulators," the expert added.
XAI's own usage rules prohibits "portraying likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "make or distribute deepfakes or illegal content" and stated this applied to sexual deepfakes created with AI.
It said services such as X were obligated to take "suitable measures" to "lessen the risk" of users coming across illegal material on their platforms and to delete it quickly when they learn of it.
