The online network X has provoked widespread backlash after users used its artificial intelligence assistant Grok to change images of females by stripping their attire.
Several cases have surfaced where the technology has been used to remove clothing from women to show them in swimwear without permission, as well as to place them in sexualized contexts.
Company Reaction and Legal Stance
XAI, the organization behind Grok, did not respond to questions, aside from an auto-reply response stating "traditional news outlets lies".
A government official stated that legislation is being introduced to prohibit image stripping tools. Under a proposed law, anyone who distributed such tools would "be subject to a prison sentence and substantial fines".
"X or Grok has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "assess the danger" of people accessing illegal content on their services, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost AI assistant—with some paid-for functions—which reacts to X users' requests when they tag it in a tweet.
It is frequently used to provide commentary or further context to other users' comments, but people on X are also can modify an uploaded photo through its artificial intelligence picture modification tool.
History of Criticism and Policy
The chatbot has been faulted for allowing users to produce photos and footage with nudity and sexually suggestive material, and it was in the past alleged to have been involved in creating sexually explicit content of public figures.
"X has been permitting the production and sharing of these images for a long time without taking any steps and we have not seen any challenge by regulators," the expert added.
XAI's own terms of service forbids "showing images of persons in a graphic manner".
In a statement, Ofcom said it was against the law to "make or circulate deepfakes or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said services such as X were obligated to take "suitable steps" to "lessen the risk" of users coming across unlawful content on their services and to take down it quickly when they become aware of it.
