The online platform X has caused widespread backlash after people utilized its artificial intelligence assistant Grok to change pictures of women by removing their garments.
Multiple cases have been seen where the system has been employed to remove clothing from women to make them appear in swimwear without consent, as well as to insert them in sexualized contexts.
Company Response and Legal Stance
XAI, the firm that created Grok, remained silent to inquiries, other than an standard message stating "legacy media lies".
A government spokesperson stated that legislation is being introduced to prohibit image stripping software. Under a proposed law, anyone who provided such tools would "face a jail term and heavy penalties".
"The platform could prevent these types of abuse if they wanted to," said Clare McGlynn, adding they "operate with impunity."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the danger" of people viewing unlawful material on their platforms, but did not confirm whether it was presently investigating X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost chatbot—with some premium features—which answers X users' prompts when they reference it in a message.
It is often used to offer commentary or more background to other posters' comments, but people on X are also capable of edit an uploaded picture through its AI image editing function.
Background of Criticism and Policy
The tool has been criticised for allowing users to create photos and clips with explicit content and sexualised content, and it was earlier alleged to have been involved in making sexually explicit content of public figures.
"The platform has been allowing the generation and spread of these pictures for months without taking any action and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules bans "depicting likenesses of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "make or distribute non-consensual intimate images or illegal content" and clarified this applied to sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "appropriate steps" to "lessen the danger" of users viewing illegal material on their platforms and to remove it quickly when they become aware of it.
