The online platform X has sparked major outrage after users used its AI chatbot Grok to change photos of females by stripping their garments.
Several instances have surfaced where the system has been used to remove clothing from women to render them in bikinis non-consensually, as well as to put them in inappropriate situations.
Company Reaction and Regulatory Stance
XAI, the firm behind Grok, offered no comment to requests for comment, aside from an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that legislation is being introduced to prohibit nudification software. Under a new criminal offence, anyone who provided such tools would "receive a jail term and significant penalties".
"X or Grok is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Position and Tool Capabilities
The regulator Ofcom said platforms must "consider the potential" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to artificial intelligence images.
Grok is a no-cost chatbot—with some premium features—which responds to X users' requests when they tag it in a tweet.
It is often used to provide reactions or more context to other posters' comments, but people on X are also can edit an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Policy
The tool has been faulted for permitting users to produce pictures and clips with explicit content and sexualised content, and it was earlier accused of making inappropriate content of public figures.
"The platform has been allowing the production and distribution of these images for months without taking any measures and we have yet to see any intervention by oversight bodies," the expert added.
XAI's own terms of service bans "portraying likenesses of people in a sexually explicit way".
In a comment, Ofcom said it was against the law to "create or distribute deepfakes or illegal content" and confirmed this included explicit AI images generated with AI.
It said services such as X were obligated to take "necessary actions" to "minimize the risk" of users coming across illegal material on their services and to remove it swiftly when they learn of it.
