The online network X has caused significant backlash after individuals used its AI chatbot Grok to alter images of females by removing their clothing.
Numerous cases have surfaced where the technology has been employed to remove clothing from female subjects to render them in swimwear non-consensually, as well as to place them in sexually explicit contexts.
Company Response and Legal Position
XAI, the organization that created Grok, remained silent to requests for comment, except for an auto-reply message stating "legacy media lies".
A government representative stated that legislation is being introduced to ban image stripping software. Under a upcoming law, anyone who distributed such technology would "face a jail term and significant penalties".
"The platform is capable of halting these forms of abuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Platform Functionality
The communications watchdog Ofcom said tech firms must "consider the potential" of people accessing unlawful material on their networks, but stopped short of confirming whether it was at this time investigating X or Grok in concerning AI-generated pictures.
Grok is a free chatbot—with some additional functions—which reacts to X users' requests when they reference it in a message.
It is often used to provide reactions or more background to other users' comments, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification feature.
History of Controversy and Rules
The chatbot has been condemned for permitting users to create photos and clips with nudity and sexualised material, and it was previously alleged to have been involved in making inappropriate material of public figures.
"X has been permitting the production and distribution of these images for a long time without taking any measures and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was illegal to "create or share deepfakes or illegal content" and clarified this applied to sexual deepfakes made with AI.
It said services such as X were mandated to take "suitable measures" to "minimize the danger" of users viewing unlawful material on their platforms and to delete it swiftly when they become aware of it.
