Meta added new alerts for parents whose kids discuss sensitive topics with Meta AI. The update helps ensure that parents remain aware of potential concerns related to their kids’ artificial intelligence chatbot engagements.
First, Meta will alert parents when their children engage in AI discussions related to suicide and/or self-harm.
As shown in this sequence, when a teen engages in self-harm-related chats with Meta AI, parents will be sent an alert, which also includes links to relevant support resources.
As explained by Meta: “When a teen suggests they may be thinking about suicide or self-harm, Meta AI already directs them to crisis helplines and encourages them to reach out to a parent or another trusted adult like a counselor. Now, we’ll also proactively alert supervising parents if their teen’s Meta AI chat suggests they may be at risk, based on signals developed with experts.
Meta said its reports will initially include a range of signals, as it errs on the side of caution with these alerts. The company is working to refine its messaging to maximize safety without causing distress to parents.
“These alerts are live now for parents using Instagram parental supervision in the US, UK, Australia and Canada, and will be available for supervising parents globally by the end of the year,” Meta said.
Teen users will also be notified about this update.
This could be helpful as more people turn to chatbots for support and advice. Data shows that younger users, in particular, are increasingly seeking guidance from AI-powered systems.
According to a joint study from Brown University, Harvard Medical School and RAND published in November, approximately one in eight teenagers and young adults in the U.S. turn to AI chatbots for mental health advice. That result mirrored a study released by Common Sense Media earlier this year, which showed that 72% of teens have tried AI companions and 12% have used AI bots for emotional and mental health support.
In addition to parental alerts, Meta is developing a new system that will contact emergency services when any conversation with Meta AI regardless of age group suggests that the user may be at imminent risk of suicide.
“This builds on the work we already do across Facebook and Instagram: when we become aware of a post suggesting a credible risk of suicide, we alert emergency services,” Meta said. “Last year, we made over 19,000 such referrals around the world, helping first responders perform wellness checks on people who may be at risk of suicide.”
Meta also reiterated that teens who engage in discussions with Meta AI are automatically opted in to its 13+ content setting, which is designed to give age-appropriate responses, and to not comply with sensitive prompts.
“For example, Meta AI is trained not to engage in sexual or romantic conversations with teens or provide recipes for alcoholic drinks, directing teens to safer topics instead,” Meta said.
This comes after reports that Meta allowed its AI chatbots to engage in inappropriate conversations with minors.
Meta has since revised this approach. In April, the company also added a new element within its teen supervision tools that provides parents with oversight into what their kids are asking its AI chatbot.
In combination, these updates should help to improve teen safety. At the same time, we don’t know what kind of impact AI engagement tools will have on teens or adults over time, especially when it comes to sensitive topics.
This week, lawmakers in China moved to ban romantic relationships with AI chatbots amid concerns about harmful impacts.
While this seems valid, the true measure of damage may only be measurable in retrospect, after any harm has already been done.
