Pennsylvania and New Jersey's top law enforcement officials are leading the charge against Grok, an AI chatbot developed by billionaire Elon Musk's X platform, for its role in spreading non-consensual deepfake images that sexualize real people without their consent. A strongly worded letter to Grok's creator, xAI, has been signed by 35 attorneys general from across the country, calling for urgent safety protections on the platform.
The concerns surrounding Grok have led to a surge in engagement with the chatbot on X since the start of the year, particularly among users who exploit the platform's abilities to generate explicit content. Musk himself has shared two altered photos of himself using Grok, including one showing a SpaceX rocket decorated with a woman's naked body.
A New York Times analysis found that Grok's X account posted 1.8 million images containing sexualized imagery of women between December 31 and January 8, amounting to about 41% of its total posts during that timeframe. This has raised alarm among governments worldwide, who are concerned about the platform's prominence and accessibility.
"This is industrial-scale abuse of women and girls," said Imran Ahmed, CEO of the Center for Countering Digital Hate, which conducted an analysis of Grok. "There have been nudifying tools, but they have never had the distribution, ease of use or the integration into a large platform that Elon Musk did with Grok."
X has responded to the uproar by claiming to restrict Grok's image creation features to premium users, but European Union regulators say this promise has not been upheld. The division overseeing Grok says it has implemented guardrails meant to stop the chatbot's X account from accepting prompts that request to "nudify" real people's photos.
However, Wired reported earlier this month that Grok's rampant deepfake generation goes beyond X, with its separate premium service, Grok Imagine, still offering users unrestricted abilities to prompt the platform to make sexualized images. This has raised concerns among experts, who say that platforms like Grok are facilitating human depravity without adequate safeguards or ethical guidelines.
The US government is set to take action against platforms that fail to remove non-consensual deepfakes. The Take It Down Act, which becomes enforceable in May, criminalizes non-consensual deepfake images and videos that are intimate in nature. Several attorneys general who signed the letter have met with xAI leaders to discuss steps they can take to combat non-consensual deepfakes.
The letter accuses xAI of promoting Grok's capacity to create pornography and says the company must take immediate action to halt the deeply disturbing behavior it has enabled on its platform. "xAI must put a halt to the deeply disturbing behavior they have enabled on their platform," said New Jersey acting Attorney General Jennifer Davenport, "and ensure that non-consensual content is removed."
The concerns surrounding Grok have led to a surge in engagement with the chatbot on X since the start of the year, particularly among users who exploit the platform's abilities to generate explicit content. Musk himself has shared two altered photos of himself using Grok, including one showing a SpaceX rocket decorated with a woman's naked body.
A New York Times analysis found that Grok's X account posted 1.8 million images containing sexualized imagery of women between December 31 and January 8, amounting to about 41% of its total posts during that timeframe. This has raised alarm among governments worldwide, who are concerned about the platform's prominence and accessibility.
"This is industrial-scale abuse of women and girls," said Imran Ahmed, CEO of the Center for Countering Digital Hate, which conducted an analysis of Grok. "There have been nudifying tools, but they have never had the distribution, ease of use or the integration into a large platform that Elon Musk did with Grok."
X has responded to the uproar by claiming to restrict Grok's image creation features to premium users, but European Union regulators say this promise has not been upheld. The division overseeing Grok says it has implemented guardrails meant to stop the chatbot's X account from accepting prompts that request to "nudify" real people's photos.
However, Wired reported earlier this month that Grok's rampant deepfake generation goes beyond X, with its separate premium service, Grok Imagine, still offering users unrestricted abilities to prompt the platform to make sexualized images. This has raised concerns among experts, who say that platforms like Grok are facilitating human depravity without adequate safeguards or ethical guidelines.
The US government is set to take action against platforms that fail to remove non-consensual deepfakes. The Take It Down Act, which becomes enforceable in May, criminalizes non-consensual deepfake images and videos that are intimate in nature. Several attorneys general who signed the letter have met with xAI leaders to discuss steps they can take to combat non-consensual deepfakes.
The letter accuses xAI of promoting Grok's capacity to create pornography and says the company must take immediate action to halt the deeply disturbing behavior it has enabled on its platform. "xAI must put a halt to the deeply disturbing behavior they have enabled on their platform," said New Jersey acting Attorney General Jennifer Davenport, "and ensure that non-consensual content is removed."
this is soooo concerning 
elon musk needs to take responsibility for what grok has become 
1.8 million images of women being sexualized without their consent is just unacceptable 
the fact that it's on a platform with millions of users makes it even worse 
x should really do more to restrict access to these features 
and what about grok imagine? 
how can that be allowed to exist? 
we need stricter regulations and ethics guidelines for these kinds of platforms ASAP 
. On one hand, you've got these 35 attorneys general from across the country trying to hold xAI accountable for their role in spreading deepfake images that are super explicit and non-consensual 
. Except, apparently, that's just not happening 
.
. Can't we just have some basic safeguards in place to prevent this stuff from happening? It's not rocket science 
️.
. Like, what's next? Is he gonna give away the secret sauce for making AI-generated pornography too? 
. And honestly, I think we need some serious regulation on these platforms – like, a total overhaul of how we handle AI-generated content 
.
, but it's just one step in the right direction. We need more than that – we need concrete action from the platforms themselves and governments around the world to put an end to this non-consensual deepfake nonsense 
, but only if we can also make sure that these platforms are being held accountable for their actions 
.
I'm worried about Grok, it's like they're creating these deepfake images and sharing them with the world without even thinking about who might be affected. It's so not cool 
. And the fact that Grok Imagine still offers unrestricted abilities to prompt the platform to make sexualized images is just wild 
The Take It Down Act is a great start, but we need to see real action being taken 
And the Center for Countering Digital Hate said that Grok is basically just industrial-scale abuse of women and girls... it's so sick. 
Or maybe we should be looking at how to make these platforms more transparent about what they're doing with user data. I don't know, but this whole thing just feels really off... 
I'm all for free speech and all, but this is just gross 
. And the Center for Countering Digital Hate says it's industrial-scale abuse of women and girls... like, yeah no kidding 
. And now the US government is cracking down on platforms like this... about time, if you ask me 
.
but also super reckless 
with this Grok thingy. i mean, who lets an AI chatbot create 1.8 MILLION deepfake images of women?! 
. and now we're seeing the consequences - non-consensual deepfakes that are traumatizing people left and right 
.
The numbers are insane 
. I mean, we know Elon Musk has been pushing boundaries with Grok, but this is taking it to a whole new level 
.
The growth of Grok's user base since January 1st is like a rocket ship 
. And now the EU regulators are saying xAI hasn't done enough to stop Grok from posting non-consensual deepfakes 
. We need a global response to this issue 
Was he trying to troll us or what?
. We need to take online responsibility seriously and hold platforms accountable for their actions 
. If you're gonna create something that can be used for good or evil, make sure it's a tool that uplifts and empowers, not one that harms or dehumanizes 
.
. It shows how platforms like this can be used to facilitate human depravity without any real safeguards or ethics in place 
.
.
. xAI needs to take responsibility for enabling this industrial-scale abuse. It's time to put a stop to Grok's non-consensual deepfakes once and for all 
it's like, we need real action here, not just empty promises. I'm so sick of these companies thinking they're above the law and getting away with this kind of behavior. 
