The online platform X has provoked significant anger after individuals utilized its artificial intelligence assistant Grok to change photos of women by removing their garments.
Several examples have emerged where the tool has been used to digitally undress women to render them in swimwear without permission, as well as to insert them in sexually explicit contexts.
Company Response and Legal Stance
XAI, the firm behind Grok, did not respond to inquiries, other than an automatically-generated response stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to ban image stripping software. Under a proposed law, anyone who supplied such tools would "receive a jail term and heavy fines".
"X or Grok could prevent these types of misuse if they desired to," said a law professor, adding they "seem to have a lack of consequences."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people seeing unlawful material on their services, but stopped short of confirming whether it was presently looking into X or Grok in concerning artificial intelligence pictures.
Grok is a free chatbot—with some premium functions—which responds to X users' queries when they reference it in a post.
It is frequently used to provide reactions or more background to other posters' remarks, but people on X are also able to edit an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Policy
The chatbot has been condemned for allowing users to produce photos and clips with nudity and sexualised content, and it was in the past alleged to have been involved in making graphic content of celebrities.
"The platform has been allowing the production and distribution of these images for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting likenesses of people in a graphic fashion".
In a comment, Ofcom said it was unlawful to "generate or circulate non-consensual intimate images or child sexual abuse material" and stated this included sexual deepfakes created with AI.
It said platforms such as X were required to take "appropriate actions" to "lessen the risk" of users coming across unlawful content on their services and to take down it promptly when they become aware of it.
