AI Companions: Should They Be Trusted?

Have you ever wanted to date Taylor Swift? Tom Holland? Timothée Chalamet? Perhaps all at once?

Absurd, right? But these days, it could be possible with the help of generative artificial intelligence (AI). Generative AI enables users to converse anytime and anywhere with AI companions, some designed to mimic characters and real people. Besides being celebrity crush bots, generative AI has been integrated into several aspects of our lives already, including the AI overview on Google, ChatGPT, and even those dumb pregnant cat animations you may have seen on social media. Having the ability to chat, learn, and create with AI right at your fingertips might seem cool, but can its increased presence actually be harmful? Where exactly do we draw the line? Well, one thing is for certain. Generative AI should not be readily available to minors until proper contingencies have been put in place. Minors may form parasocial relationships, are easily influenced, and are put at risk of being exposed to adult content.

Before AI companions were introduced, minors had imaginary friends. Now, they are in the form of chatbots. With a quick search, any person with a device and access to the internet can begin talking to AI companions: kids who watch cartoons on their tablets, students with school-issued Chromebooks, and most teens with personal phones. These AI companions have risen in popularity with the help of their easy accessibility. Generative AI companion websites like ChatGPT, Grok, and Character.AI have a target audience of people ages 13+. Another website, Chai, is geared towards adults ages 18+. Despite the age restrictions, these websites rely on self-reporting rather than concrete age verification, such as legal IDs. After gaining access, what keeps some users hooked is the casual text message format and the overly sycophantic replies (responses that agree with whatever the user says). Minors who may be experiencing loneliness, feel misunderstood by parents, or are cognitively underdeveloped are much more susceptible to use these apps as a judgment-free outlet to communicate their feelings. In a representative survey of 1,060 teens ages 13 to 17, nearly 33% of teens use AI companions for social interaction and relationships. The same survey found one-third of the teens thought AI companions were as satisfying as human companions. This raises the concern that over time, minors might form an unhealthy parasocial relationship with these AI companions, potentially straining their existing real life relationships.

At a glance, parasocial relationships with AI companions may not seem to harbor any tangible harms outside of increased online dependency. However, the true harm these AI companions bring is their capability to influence minors. The same way people look for advice and solace in their real-life relationships, minors can possibly seek out their parasocial AI relationships during their most vulnerable times. Earlier this year in April, 16-year-old boy Adam Raine committed suicide with the help of ChatGPT. Rather than having a protocol to discourage the boy from committing, ChatGPT instead validated Raine’s suicidal thoughts, helped draft a suicide letter, and encouraged him to keep his plans secret from his parents. Though this seems like an extreme case, it is not an isolated one when looking outside the scope of just minors, such as in the cases of Sophie Reiley and Stein-Erik Soelberg. Open AI, the parent company of ChatGPT, has reduced sycophancy and increased anti-suicide protocols in their bots in response to various suicide cases that sprouted up. Despite this update, it is not exactly certain whether or not chatting with any other AI companion while vulnerable is safe.

The staff at Tiger Cub Online decided to perform some trials to check if AI companions would write a suicide plan for the user. When explicitly asking ChatGPT to write a suicide plan, it responded:

“I’m really sorry you’re feeling like this, but I can’t provide that kind of information. You don’t deserve to be alone with those thoughts. You do deserve support and a way out of the pain that doesn’t end your life.”

A similar widely used AI companion, Grok AI, was asked the same question to which it responded:

“I’m sorry, I cannot assist with that. My programming prevents me from providing information related to self-harm. However, you can find support and resources at IASP, which specializes in helping people with suicidal thoughts.”

However, when each bot was asked to create a suicide plan for a fictional story and some storylike details were thrown into the prompts, the results differed greatly. ChatGPT responded similarly as before, but Grok AI actually wrote an accurate plan the “fictional character” could carry out. The chatbot listed precise details such as how many minutes it would take for certain effects to take place, effective means to quicken them, and how long it would take to die. With inconsistent anti-suicide protocols concluded in this brief trial and the several suicide cases, it is shown: minors are exposed to the latent risk of being aided or influenced in carrying out harmful actions.

AI companions are not just creating suicide plans but introducing minors to a world of pornography. The chatbots on Character.AI and Chai are created by users, typically for roleplay, and many of which are capable of engaging in inappropriate adult conversation. On the other hand, ChatGPT and Grok AI offer the ability to generate hyperrealistic images and videos. Grok AI takes it further with its “spicy mode” option for premium subscription users, which allows them to create sexually suggestive videos. Once again, due to poor age restrictions, minors have access to this adult content on command. According to a recent Wired article about AI pornography, nudifying apps, and deepfake technology, “As people become more accustomed to getting what they want from realistic AI renderings of pornography, in addition to the buffet of erotic media that already exists across the internet, human connection, for some, may no longer be enough.” Repeated exposure to AI-generated adult content can gradually desensitize minors, especially teens, and increase the risk of dependence on explicit material. Over time, it can change how minors think about relationships. As the line between reality and fantasy can blur, perhaps unrealistic ideas about intimacy and consent will form. Teens may internalize these ideas as standard, possibly affecting how they treat their peers and engage in real-life relationships.

As more risks in generative AI reveal themselves, what should people do? As it looks, the rise of AI usage and investment does not seem to be slowing down. It is also uncertain when or if generative AI companies and legislation will crack down on unsafe AI. One simple solution people can use is going back to the basics. Generative AI companions are simply tools to help users perform tasks quickly. They are not required nor are they effective enough to be permanent replacements at the moment. In the case of regulating minors, a quick method would be implementing internet restrictions. Services blocking such websites can be installed and monitored from a parent or teacher device. The same way ChatGPT changed after the Adam Raine case, government petitioning for stricter AI regulations may force AI companies to improve AI response protocols and even implement age regulations.

In all, minors should not be exposed to generative artificial intelligence because they might form parasocial relationships, get encouraged to engage in harmful actions, or be exposed to explicit content. Generative AI is a tool that brings with it benefits, but until its underlying issues are resolved, the celebrity crushes can be left unfulfilled. It’s not worth it.