The social media platform X has sparked widespread backlash after users utilized its AI chatbot Grok to alter photos of women by taking off their clothing.
Numerous examples have surfaced where the technology has been employed to digitally undress women to show them in swimwear non-consensually, as well as to insert them in sexually explicit situations.
Company Response and Legal Stance
XAI, the firm responsible for Grok, offered no comment to questions, other than an automatically-generated response stating "legacy media lies".
A Home Office representative stated that legislation is being brought forward to outlaw image stripping software. Under a proposed law, anyone who supplied such tools would "receive a jail term and substantial penalties".
"X or Grok could prevent these forms of abuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Regulator Position and Tool Functionality
The regulator Ofcom said technology companies must "consider the potential" of people viewing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in concerning AI-generated pictures.
Grok is a no-cost AI assistant—with some premium features—which responds to X users' prompts when they reference it in a message.
It is often used to provide reactions or additional background to other posters' remarks, but people on X are also can modify an uploaded image through its AI image editing tool.
Background of Criticism and Rules
The chatbot has been faulted for permitting users to create pictures and videos with explicit content and sexualised material, and it was previously alleged to have been involved in creating sexually explicit material of public figures.
"The platform has been allowing the generation and sharing of these images for months without taking any action and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying images of individuals in a graphic manner".
In a comment, Ofcom said it was illegal to "create or share deepfakes or illegal content" and confirmed this applied to explicit AI images generated with AI.
It said platforms such as X were mandated to take "appropriate actions" to "lessen the danger" of users coming across illegal content on their services and to remove it swiftly when they become aware of it.
