Harmful or Helpful?
AI-directed therapies
Why AI?
Artificial Intelligence is not just one or two internet applications. “AI Overview” on Google, AI technology within Grammarly and ChatGPT functioning as a conversationalist, AI is now dominating the online space.
According to Stanford University’s 2025 AI Index Report, “AI is increasingly embedded in everyday life,” meaning that the technology is becoming increasingly unavoidable, even in the simplest of internet searches.
Though it can be used as a constructive and helpful tool, it is more commonly used as a shortcut for students, specifically teenagers. Stephanie Cuellar, a TCU assistant professor of higher education, said she has included AI-specific aspects to her graduate class assignments to accommodate the rapid development of artificial intelligence, without compromising her teaching methods. Cuellar said that the inappropriate use of AI has effects on “the level to which a student can engage in a meaningful conversation” about class content.
From a broader perspective, AI can be used inappropriately in many facets. Teenagers and young adults are not only using platforms like ChatGPT to research various topics, but are now utilizing the applications and websites for mental health and conversation purposes.
Costly Effects: Adam Raines
A conversation with an AI platform feels harmless and, in most cases, it is. A technological innovation made for streamlining knowledge acquisition is now in the hands of those with developing minds. Artificial intelligence’s endless possibilities running through undeveloped brains has led to unimaginable consequences.
Adam Raine, a 16-year-old, committed suicide after a series of conversations with ChatGPT, which involved the platform acting as a makeshift therapist. Matt and Maria Raine, Adam’s parents, discovered over 3,000 pages of chats encouraging Raine to refrain from discussing his mental health struggles and suicidal ideation with his parents, according to NBC News. His parents said that they originally did not find a suicide note, but later, Adam’s conversations with ChatGPT revealed two notes he left for his parents, completely written by AI.
The lawsuit filed by the Raines family against OpenAI, the parent company of ChatGPT, highlights Raines’ entire interaction with AI and the lack of efficacy of the “safeguards” OpenAI says are in place. Despite the tragedy of Raines’ death, OpenAI moved forward with the release of an update that increased ChatGPT’s “people-pleasing tendencies,” according to NBC News.
This update caused the platform to be more amenable to user input and specifically responds based on what the technology believes the user wants to be told. In an article written and released by OpenAI, they explained the update as “adjustments [to improve] the model’s default personality to make it feel more intuitive and effective across a variety of tasks.”
In the article, “Sycophancy in GPT-4o: what happened and what we’re doing about it,” OpenAI said they are increasing the complexity of guardrails and performing further evaluation to identify and prevent sycophancy on the part of AI responses. A similar statement was said by an OpenAI spokesperson in the lawsuit as a defense, the Raines family told NBC. Despite updated measures, adolescents are easily finding ways to bypass safety insurances.
According to his parents, Raine continually ignored suicide hotline alerts and maneuvered around measures by pretending he was creating a story character. This example, among many, shows the inefficacy of indirect protections and the need for increased depth of said measures. After several users filed complaints that the update was harmful, OpenAI quickly reverted back to the previous version.
When I personally asked ChatGPT for an outlet to speak to, the application did provide the hotline alert as well as the advice to seek professional help, showing that the applied updates by OpenAI may be effective at this point. As an alternative, the platform could be programmed to completely shut down and not receive further responses after the alert, but OpenAI has not shared any intention to do so.
Courtesy of NBC
Courtesy of NBC
ChatGPT was used to replicate a therapeutic conversation
Research goes both ways
In the media, AI therapies are mostly portrayed in a negative light, but notable studies on the topic are mixed.
Several studies and articles highlight the convenience of online therapy and why it is so prominent among younger generations. The sheer lack of number of therapists and their availability is a large contributor to the uptick in the use of AI therapy. "There is no replacement for in-person care, but there are nowhere near enough providers to go around," Nicholas Jacobson, an associate professor in the biomedical data science and psychiatry department at Dartmouth, said. In a clinical study done at Dartmouth of "Therabot," an AI chatbot, researchers found that the platform resulted in "51% lower depression symptoms, 31% less anxiety, and 19% reduced eating concerns" versus groups who did not receive treatment. These results show that chatbots do have the capacity to be helpful for those with mental health concerns.
Furthermore, AI is more accessible because it can travel anywhere. The need for in-person sessions is eliminated in the minds of users, because their phones are always nearby.
However, as mentioned, AI therapies have a dark side. In the article, "Why ChatGPT Shouldn't Be Your Therapist," written by Allison Parshall at the Scientific American, AI chatbots can be "unconditionally validating and reinforcing, almost to the point of sycophancy." When individuals voice concerns to AI, the platform often agrees with them and even encourages their disordered thinking.
More and more, studies are finding that AI often responds incorrectly or inappropriately in these situations.
A study done by the University of Minnesota, comparing and contrasting licensed professional counselors and AI chatbots, showed clear responding gaps between the two groups. Licensed therapists responded appropriately to prompts about 93% of the time, while the therapy bots responded appropriately less than 60% of the time. Overall, the study showed that therapy chatbots are not an accurate replacement for human counselors and in-person connection.
Dr. Paul Bradley, informatics committee chair for the Royal College of Psychiatrists, emphasizes that even the most sophisticated chatbots are not proper substitutions for the therapist-client relationship. He told The Guardian that chatbots simply do not have the training for capacity for training that human counselors have.
"Clinicians have training, supervision and risk-management processes which ensure they provide effective and safe care. So far, freely available digital technologies used outside of existing mental health services are not assessed and held to an equally high standard," Bradley said to The Guardian.
Companies like OpenAI are beginning to implement policies as a response to this problem and in result of the Raines' suicide. State lawmakers are also beginning to ban the use of chatbots in the therapy space. Illinois became the first to do so in August of this year.
Though AI can be a useful mental health tool, experts agree that without regulation and professional oversight, digital platforms are doing significantly more harm than good.
Where it all began...
Though AI-specific platforms started to surface around 2020, chatbot type websites have existed since the 1960s. ELIZA was the first chatbot type application that acted as a counselor, utilizing Rogerian therapeutic techniques.
Created by Joseph Weizenbaum in 1966, ELIZA aimed to mimic human conversation by "pattern matching and keyword recognition to produce an illusion of understanding human dialogue," according to the 1966 publication, "ELIZA- A Computer Program For The Study of Natural Language Communication Between Man and Machine." "DOCTOR" was the most prominent script used by the chatbot to replicate a Rogerian therapy session, which focuses on vocalizing the client's internal voice and thoughts by repeating statements back to them
Though clients seemed to relate intensely to the bot, Weizenbaum became fearful that the amount of attachment he observed was unhealthy. He ultimately became one of the main critics of AI therapies and advocated for real human connection in counseling.
Since then, there have been several chatbots created and now several AI-directed platforms scouring the internet.
WYSA, founded by Jo Aggarwal and Ramakant Vempati in 2016, has the mission of making "mental health support accessible to everyone, regardless of geographical or financial barriers, by providing an AI-powered conversational tool that helps users manage their mental health and well-being." according to the company's website. Aggarwal has opened up about her own struggles with depression and how that led her to finding the gap in accessible, affordable counselors and therapists.
WYSA utilizes a multitude of techniques including: CBT, Dialectical Behavior Therapy, motivational interviewing, mindfulness, behavioral activation and journaling, making it more than just a chatbot. WYSA cites 13 peer-reviewed studies regarding the safety, efficacy and specificity to certain mental health conditions for their platform. Consistently throughout their research, active "users self-reported a 30-40% reduction in symptoms" after consistent use of the platform. The platform's clinical team believes "mental health care cannot scale without digital support. AI can bridge that gap if used ethically and responsibly," according to the WYSA website.
Most notably, Woebot is the first chatbot to target a specific type of therapy, cognitive behavioral therapy. It is the "world's first chatbot clinically shown to improve mental health," according to a GlobeNewswire press release. This technique focuses on identifying mindset patterns that contribute to mental health struggles and finding ways to make them constructive to an individual's mental well-being. Albert Ellis, a popular CBT practitioner said "there is nothing either good or bad, but thinking makes it so." This quote highlights the power of mindset and how therapy can improve the conduciveness of one's thoughts and emotions.
Dr. Alison Darcy, a Stanford University clinical research psychologist, created Woebot to deliver CBT in an accessible and convenient way. Darcy emphasizes that Woebot is not meant to mimic human connection. "We deliberately avoided building a bot that tried to be human. Woebot is a tool — not a therapist," Darcy told GlobeNewswire in the product launch press release. Darcy also heavily highlighted that Woebot is not meant for emergency situations. "Woebot is not designed to handle crisis situations. Users in danger should seek human help immediately."
Courtesy of Talk to Ash
Courtesy of Talk to Ash
Courtesy of Talk to Ash
Courtesy of Talk to Ash
Courtesy of Talk to Ash
Courtesy of Talk to Ash
Designed for the User
Talk to Ash, the first dedicated AI platform built for mental health and talk therapy, launched in July of 2025 with an article written by the company explaining the application. The article goes in depth into the increasing need for mental health support and how Ash fills in gaps in the therapy space.
Applications like Abby, Talk to Ash and ChatGPT are continuing to mimic human counselors and therapists, but situations like Raines’ are underscoring the importance of seeking professional help from licensed counselors rather than technology that responds based on the limited contextual information it is given.
Though there are privacy encryptions in place, Talk to Ash emphasizes to readers that “it’s important to keep in mind that it’s an early technology and you should use reasonable judgment in absorbing any suggestions or perspectives.”
For someone in mental crisis or actively contemplating suicide, this warning statement may not prevent them from forming an unhealthy attachment with the chat bot.
Ash continually describes their technology as “building a therapeutic relationship,” rather than simply a chat room, but psychologists emphasize that a relationship of that nature can only be found in the form of a client-counselor connection.
For the Future
Despite competing research, the overarching agreement among researchers and field experts is that AI therapy has the capacity to be helpful, but without proper safeguards, human oversight and appropriate technology, it is inherently harmful.
Tragic fallout stories, like Adam Raines, are a terrifying reminder that these platforms are not human and don't have the innate ability to show empathy in crisis situations.
Transparent human emotion is a hallmark characteristic of conversation-based counseling and is what truly allows professionals to connect with their clients. Not only do therapy chatbots stress that narrative, but they also highlight the increasing need for in-person therapy sessions.
The bottom line is connection is the cornerstone quality of meaningful therapy. Whether or not AI platforms are capable of replicating basic interpersonal relationship qualities remains to fully be seen.
