AI as the Therapist: Student Insights on the Challenges of Using Generative AI for School Mental Health Frameworks

Abstract

The integration of generative AI (GenAI) in school-based mental health services presents new opportunities and challenges. This study focuses on the challenges of using GenAI chatbots as therapeutic tools by exploring secondary school students’ perceptions of such applications. The data were collected from students who had both theoretical and practical experience with GenAI. Based on Grodniewicz and Hohol’s framework highlighting the “Problem of a Confused Therapist”, “Problem of a Non-human Therapist”, and “Problem of a Narrowly Intelligent Therapist”, qualitative data from student reflections were examined using thematic analysis. The findings revealed that while students acknowledged AI’s benefits, such as accessibility and non-judgemental feedback, they expressed significant concerns about a lack of empathy, trust, and adaptability. The implications underscore the need for AI chatbot use to be complemented by in-person counselling, emphasising the importance of human oversight in AI-augmented mental health care. This study contributes to a deeper understanding of how advanced AI can be ethically and effectively incorporated into school mental health frameworks, balancing technological potential with essential human interaction.