Fear of AI losing control

A technical error caused Snapchat's AI to post its own status lines, refuse to answer questions, and make many people worried.

My AI is built on OpenAI 's ChatGPT model . Instead of just making suggestions, answering questions, and having regular conversations, Snapchat's chatbots can customize names, design avatars, and appear in group chats.

Last week, My AI acted unusually. Many people said that chatbots post videos to their personal accounts and write self-composed content. Some people say chatbots "ignore" their messages. "Even a botAI doesn't have time for me," one person wrote. My AI's strange actions make some users think that AI has its own thoughts. Snapchat representatives later admitted that their AI system had some problems and had been fixed.

My AI on Snapchat. Photo: Foundry

The posting issue is just the latest problem with chatbots like My AI. Since its launch in April, Snapchat's tool has faced backlash from the community. Many people said they have banned their children from using My AI. Others accept to pay a fee of 3.99 USD (95 thousand VND) per month to turn off this tool.

In a letter to then-CEO Snapchat, Democratic Senator Michael Bennet pointed out concerns about how chatbots interact with users. He mentioned that AI can teach young children how to lie to their parents.

In response, Snapchat wrote on the company's blog: "My AI needs more time to become perfect but the company has made a lot of progress."

To date, Snapchat still maintains My AI on the platform, despite warnings from experts. Due to deeper customization than ChatGPT, My AI can chat with users so naturally that many people have difficulty distinguishing between AI and real people.

Lyndsi Lee, living in East Prairie, Missouri (Canada), forbids her 13-year-old daughter from approaching this AI. "I don't think I'm ready enough to know how to teach my children how to distinguish between humans and machines. Snapchat may be crossing the thin line between humans and AI," CNN quoted Lee as saying.

Alexandra Hamlet, a clinical psychologist in New York, said parents of some patients have expressed concerns about their children interacting with AI. "If a teenager is in a negative mood and can't control their emotions, chatbots can make them feel worse. Over time, negative talk can erode a young person's spirit. Even though they are aware that they are talking to a chatbot, it is still difficult for them to avoid negative psychological impacts," Hamlet warned.

What worries experts the most is that when OpenAI allows third-party applications to access ChatGPT for customization on the platform, chatbots like My AI will appear more and more.