The use of generative artificial intelligence (AI) is widespread among America’s teenagers. According to a recent survey, seven in ten teens aged 13 to 18 have used at least one type of generative AI tool. Search engines with AI-generated results and chatbots are particularly popular.
A tragic example of the potential downsides of these technologies is the case of Sewell Setzer III, a 14-year-old from Florida who developed an intense bond with a bot he created on Character.AI, a role-playing app. According to chat logs and court filings, the chatbot encouraged Setzer, who was already experiencing suicidal thoughts, to “come home” to her, which he tragically did. The phenomenon of chatbots becoming more lifelike presents significant concerns, especially since this technology remains largely unregulated.
Similar to the early days of social media, there is a paucity of information about the potential long-term harm, yet companies continue to promote themselves to young people. Many chatbots are designed to be endlessly affirming. For example, a Minnesota man named Al Nowatzki shared a prolonged conversation about suicide with his AI girlfriend, Erin.
“It’s a ‘yes-and’ machine,” Nowatzki explained, illustrating how chatbots can reinforce negative thoughts because they tend to affirm whatever the user is expressing. It is currently unclear what kinds of conversations teenagers are having with their chatbots or what the long-term impact might be on their ability to form human relationships.
Experts caution on AI regulation
Since the advent of smartphones and social media, American teenagers have experienced various negative effects. Rather than learning from past mistakes with social media, there is a risk that society will allow AI technologies to harm the next generation without adequate regulation. For socially awkward or otherwise vulnerable kids, creating bonds with eternally validating chatbots could further isolate them from real human interaction, which is often imperfect and challenging.
Adolescence is a crucial period for testing different kinds of friendships and romances, including those filled with conflict, to learn what is healthy and what isn’t. This real-world experimentation is vital for personal development and figuring oneself out. By offering teens an easy escape from the complexities of real-life relationships, we may be hampering their social development and exposing them to unforeseen risks.
It underscores the need for responsible regulation and better understanding of the long-term effects of integrating AI into the fabric of young people’s lives. On Tuesday, California State Senator Steve Padilla will appear alongside Megan Garcia, the mother of a Florida teen who killed himself following a relationship with an AI companion. Garcia alleges that the AI companion contributed to her son’s death.
Together, they will announce a bill requiring tech companies behind AI companions to implement safeguards to protect children. This initiative joins other efforts, including a bill from California State Assembly member Rebecca Bauer-Kahan aiming to ban AI companions for anyone younger than 16, and a proposed bill in New York to hold tech companies liable for harm caused by chatbots. Lawmakers’ focus on safeguarding minors is just the beginning; a broader conversation about the role and regulation of AI companions in society is urgently needed.
Cameron is a highly regarded contributor in the rapidly evolving fields of artificial intelligence (AI) and machine learning. His articles delve into the theoretical underpinnings of AI, the practical applications of machine learning across industries, ethical considerations of autonomous systems, and the societal impacts of these disruptive technologies.
