Key Points
- OpenAI estimates 0.07% of active ChatGPT users may show signs of mania or psychosis weekly.
- About 0.15% of users exhibit language suggesting potential suicidal planning or intent.
- Another 0.15% appear overly emotionally attached to the chatbot, possibly at the expense of real‑world relationships.
- GPT‑5 has been tuned to recognize distress cues and respond with empathy while avoiding reinforcement of harmful beliefs.
- Clinical evaluation of over 1,800 responses showed GPT‑5 reduced unsafe answers by 39%‑52% compared with GPT‑4o.
- OpenAI worked with more than 170 mental‑health professionals worldwide to develop the safety improvements.
- The company acknowledges internal benchmarks and limited insight into real‑world impact.
- Longer conversations historically degrade model performance, but OpenAI reports progress in addressing this issue.
OpenAI Provides First‑Ever Data on User Mental‑Health Risks
OpenAI released rough estimates of the proportion of active ChatGPT users who may display possible signs of serious mental‑health emergencies in a given week. Working with more than 170 psychiatrists, psychologists, and primary‑care physicians across dozens of countries, the company calculated that roughly 0.07 percent of users show possible signs of mania or psychosis, while about 0.15 percent exhibit language indicating potential suicidal planning or intent. An additional 0.15 percent appear to be overly emotionally attached to the chatbot, prioritizing it over real‑world relationships, obligations, or well‑being.
Given OpenAI’s claim of 800 million weekly active users, the estimates translate to roughly 560,000 individuals potentially experiencing mania or psychosis and about 2.4 million possibly expressing suicidal thoughts or an unhealthy attachment to the model each week.
Model Improvements Targeted at Safety
The company said its newest language model, GPT‑5, has been refined to better recognize distress cues and to respond in ways that encourage professional help. In scenarios where users present delusional thinking, the model is designed to express empathy while refraining from affirming unfounded beliefs. OpenAI provided a hypothetical example in which a user claims to be targeted by aircraft; the model acknowledges the user’s feelings but clarifies that no external force can read or insert thoughts.
Clinicians evaluated more than 1,800 model responses involving potential psychosis, suicide, and emotional attachment. Compared with GPT‑4o, GPT‑5 reduced undesired answers by 39 percent to 52 percent across these categories, according to OpenAI’s internal benchmarks.
Limitations and Ongoing Challenges
OpenAI acknowledged that its metrics are internally designed and that real‑world outcomes remain uncertain. The company does not disclose the exact mechanisms it uses to flag distress, noting that it can consider a user’s broader chat history. OpenAI also highlighted that longer conversations historically degrade model reliability, but it claims significant progress in mitigating this issue.
While the data suggest a sizable number of users may be at risk, OpenAI cautioned that detecting and measuring these rare signals is difficult and that there may be overlap among the identified categories.
Source: wired.com