A first for Openai. The new company known by ChatgPT is attacked by a complaint filed by the parents of a teenager who finished his life. They accuse the chatbot of encouraged the latter to commit suicide. It is in this context that Optai announced new measures to “help people when they need them”, people “in great mental and emotional anguish.”
The new company had already expressed this initiative in early August, providing its conversation agent with mental health. This is to allow you to better detect signs of mental or emotional anguish.
It also suggests that users take a break during very long sessions. A novelty that is added to the security measures in force since 2023, such as the orientation of a person who expressed suicidal intentions towards professional aid.
Strengthen security measures
Openai recognizes, however, that these security measures are not enough. “It has happened that our systems do not behave as planned in sensitive situations,” said the startup in a blog article. That is why he works in other measures.
In particular, it will strengthen those regarding the safety of users during long conversations. Currently, exchanges “are more reliable during short and current exchanges”, they are likely to be during the long interactions. “As exchanges intensify, certain aspects of model safety can deteriorate,” Openai said.
While you can direct a user who shared suicidal intentions in a dedicated telephone assistance line, Chatgpt could, for example, “offer an response contrary to our security measures” after many messages for a long period. A problem that the Chatbot creator seeks to correct.
Similarly, for the content that should have been blocked, but that was not, OpenAi explained without going into details too much. It only specifies that these deficiencies are generally linked to an underestimation of the seriousness of a problem detected by its classifier.
Help people in mental anguish more
The company will not stop there. In addition to solving these problems, it also plans to facilitate access to emergency services and professional aid for people who express the intention of hurting themselves. After having begun to locate resources in the United States and Europe, Openai will extend them to other international markets. Emergency services accessibility will also be improved, with access to a single click.
The start -up will also help users in mental or emotional anguish to contact people of trusted and expand interventions to more people in crisis. Currently, its attenuation models favor cases of acute self -schening. But, thanks to a future GPT-5 update, Chatgpt can calm a situation trying to return the person to the person.
Finally, Operai works on new minor protections, including parents control. These ads come as the new company, as well as other companies that offer chatbots, are in the viewfinder of American justice. On August 26, 44 general prosecutors sent them a letter to remind them of their responsibilities and their obligations to protect children from illegal behavior (sexual conversations, suicide incentive …).
Source: BFM TV
