Malaysia will take legal action against Musk's X and xAI over misuse of Grok chatbot

The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images.

It said it served notices to X and xAI this month to remove the harmful content but no action has been taken.

“Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments” stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon.

There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse.

Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children.

Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize “nudification apps.” Britain’s media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children.

Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn’t fully address the problem.

Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, “Legacy Media Lies.”