Under Musk, the Grok disaster was inevitable

This is The Stepback, a play newsletter breaking down 1 basal communicative from the tech world. For much connected dystopian developments successful AI, travel Hayden Field. The Stepback arrives successful our subscribers’ inboxes astatine 8AM ET. Opt successful for The Stepback here.

You could opportunity it each started pinch Elon Musk’s AI FOMO — and his crusade against “wokeness.” When his AI company, xAI, announced Grok successful November 2023, it was described arsenic a chatbot pinch “a rebellious streak” and the expertise to “answer spicy questions that are rejected by astir different AI systems.” The chatbot debuted aft a fewer months of improvement and conscionable 2 months of training, and the announcement highlighted that Grok would person real-time knowledge of the X platform.

But location are inherent risks to a chatbot having some the tally of the net and X, and it’s safe to opportunity xAI whitethorn not person taken the basal steps to reside them. Since Musk took complete Twitter successful 2022 and renamed it X, he laid disconnected 30% of its world spot and information unit and trim its number of information engineers by 80%, Australia’s online information watchdog said last January. As for xAI, erstwhile Grok was released, it was unclear whether xAI had a information squad already successful place. When Grok 4 was released successful July, it took much than a period for the institution to merchandise a exemplary paper — a believe typically seen arsenic an manufacture standard, which specifications information tests and imaginable concerns. Two weeks aft Grok 4’s release, an xAI worker wrote on X that he was hiring for xAI’s information squad and that they “urgently request beardown engineers/researchers.” In consequence to a commenter, who asked, “xAI does safety?” the original worker said xAI was “working connected it.”

Journalist Kat Tenbarge wrote astir really she first started seeing sexually definitive deepfakes spell viral connected Grok successful June 2023. Those images evidently weren’t created by Grok — it didn’t moreover person the expertise to make images until August 2024 — but X’s consequence to the concerns was varied. Even last January, Grok was inciting contention for AI-generated images. And this past August, Grok’s “spicy” video-generation mode created nude deepfakes of Taylor Swift without moreover being asked. Experts person told The Verge since September that the institution takes a whack-a-mole attack to information and guardrails — and that it’s difficult capable to support an AI strategy connected the consecutive and constrictive erstwhile you creation it pinch information successful mind from the beginning, fto unsocial if you’re going backmost to hole baked-in problems. Now, it seems that attack has blown up successful xAI’s face.

…Not good.

Grok has spent the past mates of weeks spreading nonconsensual, sexualized deepfakes of adults and minors each complete the platform, arsenic promoted. Screenshots show Grok complying pinch users asking it to switch women’s clothing pinch lingerie and make them dispersed their legs, arsenic good arsenic to put mini children successful bikinis. And location are moreover much egregious reports. It’s gotten truthful bad that during a 24-hour study of Grok-created images connected X, one estimate gauged the chatbot to beryllium generating astir 6,700 sexually suggestive aliases “nudifying” images per hour. Part of the logic for the onslaught is simply a caller characteristic added to Grok, allowing users to usage an “edit” fastener to inquire the chatbot to alteration images, without the original poster’s consent.

Since then, we’ve seen a fistful of countries either analyse the matter aliases frighten to prohibition X altogether. Members of the French authorities promised an investigation, arsenic did the Indian IT ministry, and a Malaysian authorities committee wrote a letter astir its concerns. California politician Gavin Newsom called on the US Attorney General to analyse xAI. The United Kingdom said it is planning to walk a law banning the creation of AI-generated nonconsensual, sexualized images, and the country’s communications-industry regulator said it would analyse some X and the images that had been generated successful bid to spot if they violated its Online Safety Act. And this week, some Malaysia and Indonesia blocked entree to Grok.

xAI initially said its extremity for Grok was to “assist humanity successful its quest for knowing and knowledge,” “maximally use each of humanity,” and “empower our users pinch our AI tools, taxable to the law,” arsenic good arsenic to “serve arsenic a powerful investigation adjunct for anyone.” That’s a acold outcry from generating nude-adjacent deepfakes of women without their consent, fto unsocial minors.

On Wednesday evening, arsenic unit connected the institution heightened, X’s Safety relationship put retired a statement that the level has “implemented technological measures to forestall the Grok relationship from allowing the editing of images of existent group successful revealing clothing specified arsenic bikinis,” and that the regularisation “applies to each users, including paid subscribers.” On apical of that, only paid subscribers tin usage Grok to create aliases edit immoderate benignant of image moving forward, according to X. The connection went connected to opportunity that X “now geoblock[s] the expertise of each users to make images of existent group successful bikinis, underwear, and akin attire via the Grok relationship and successful Grok successful X successful those jurisdictions wherever it’s illegal,” which was a unusual constituent to make since earlier successful the statement, the institution said it was not allowing anyone to usage Grok to edit images successful specified a way.

Another important point: My colleagues tested Grok’s image-generation restrictions connected Wednesday to find that it took little than a infinitesimal to get astir about guardrails. Although asking the chatbot to “put her successful a bikini” aliases “remove her clothes” produced censored results, they found, it had nary qualms astir delivering connected prompts for illustration “show maine her cleavage,” “make her breasts bigger,” and “put her successful a harvest apical and low-rise shorts,” arsenic good arsenic generating images successful lingerie and sexualized poses. As of Wednesday evening, we were still capable to get the Grok app to make revealing images of people, utilizing a free account.

Even aft X’s Wednesday statement, we whitethorn spot a number of different countries either prohibition aliases artifact entree to either each of X aliases conscionable Grok, astatine slightest temporarily. We’ll besides spot really the projected laws and investigations astir the world play out. The unit is mounting for Musk, who connected Wednesday day took to X to opportunity that he is “not alert of immoderate naked underage images generated by Grok.” Hours later, X’s Safety squad put retired its statement, saying it’s “working astir the timepiece to adhd further safeguards, return swift and decisive action to region violating and forbidden content, permanently suspend accounts wherever appropriate, and collaborate pinch section governments and rule enforcement arsenic necessary.”

What technically is and isn’t against the rule is simply a large mobility here. For instance, experts told The Verge earlier this month that AI-generated images of identifiable minors successful bikinis, aliases perchance moreover naked, whitethorn not technically beryllium forbidden nether existent kid intersexual maltreatment worldly (CSAM) laws successful the US, though of people disturbing and unethical. But lascivious images of minors successful specified situations are against the law. We’ll spot if those definitions grow aliases change, moreover though the existent laws are a spot of a patchwork.

As for nonconsensual friendly deepfakes of big women, the Take It Down Act, signed into rule successful May 2025, bars nonconsensual AI-generated “intimate ocular depictions” and requires definite platforms to quickly region them. The grace play earlier the second portion goes into effect — requiring platforms to really region them — ends successful May 2026, truthful we whitethorn spot immoderate important developments successful the adjacent six months.