image: ©Keeproll | iStock

Artificial intelligence is arriving in mental health care faster than the rules meant to govern it.

Apps track our mood and sleep; chatbots are available to us at any time, day or night; and algorithms claim they can flag a crisis before it happens. (1)

For a field shaped by long waits, stigma and scattered access, that is a tempting offer.

The catch: the same systems built to support us also turn our most private experiences into data that can be analysed, predicted and sold.

At Mental Health Europe (MHE), we treat the technology neither as good or bad on its own. What decides the outcome is how it’s built and by who, who controls it, and whether anyone is willing to regulate it properly. (2) We caution against viewing digital technologies as a panacea for complex societal issues. Policymakers must avoid techno-solutionism.

True innovation in mental health care goes beyond technological advancements; it lies in our approach to community-based care. Co-creation, rooted in collaboration and understanding, stands as a transformative path forward.

Used well, these tools could add some support, and people are using them anyway, whether they help or not. Symptom-tracking apps and wearables can catch early warning signs or just help us understand our habits a bit better. But the evidence is inconsistent: what works depends heavily on the tool, the person and the setting, and proper real-world testing is still thin on the ground. (3,4)

For us, it is important to remind everybody that digital tools should support human, community-based care, not replace it. An algorithm can’t offer the relationship on which recovery is built. Lean too hard on automated assessment and you quietly redefine mental health as a personal glitch to be flagged and fixed, while the things that actually frame and drive it (such as precarious situation, loneliness, discrimination, loss of job) slip out of the digital tool view.

None of this lands evenly. It lands hardest on people who are already struggling. Few things are more sensitive than a record of someone’s mental health, and yet that data is sometimes poorly guarded. When researchers looked at 36 of the most popular depression and smoking-cessation apps, most were quietly sending user data to Google and Facebook, and only a handful owned up to it in their privacy policy. (5)

Once that information leaks or is reused, the timely advertising and incentives for consuming certain products that would give the illusion to support someone’s vulnerability can be quite convincing. Or in other cases, it can trail a person into decisions about insurance, a job, a tenancy, a loan; sometimes the fear of that is enough to stop someone asking for help.

A lot of these systems are trained on data that is historical but doesn’t fully represent the diversity of users; for example, young people using more wearables, which isn’t accurate for older people, and skin tone differences that can alter the results of measures. (6) A model tuned to read one group’s physical indicators risks misreading how another group speaks or shows distress. It solidifies the discrimination people with mental health challenges and/or with disabilities already deal with. That current situation shows how much we need open-source datasets, diversified data, and better data quality and access.

The third risk is profiling. Once AI is scoring risk, forecasting behaviour or deciding who gets what, the issue isn’t only whether it’s accurate, it’s who holds the power. A prediction about your mental state, right or wrong, can trigger things you never agreed to, from an unwanted intervention to being quietly shut out of a service. The GDPR already gives people the right not to be judged by a purely automated decision. (7) But enforcement is years behind.

Europe already has some theoretical protections for Europeans: the GDPR, the DSA, the European Health Data Space and the AI Act, all recognise that rights don’t stop at the screen. (8) How those frameworks should be sharpened for mental health specifically is a conversation we are still having, and not one to settle in this single article. But what we can offer here is the lens we bring to it.

That lens is the psychosocial approach, and it is a simple and organic view with wide reach. Mental health is shaped mostly by the conditions of a person’s life, their relationships, their security, their work, their inclusion or exclusion, rather than by something wrong inside the individual on its own.

The same way of looking applies to AI. Privacy, biases and profiling tend to get framed as technical problems waiting for technical fixes. They are social problems too. A skewed dataset reflects who gets studied and who gets left out. A profiling system draws its power from inequalities that already exist in insurance, work and welfare. A tool that flattens a person into a risk score does to mental health exactly what the psychosocial approach warns against: it removes the context and keeps only the individual.

So the question we would put first is not ‘Is the algorithm accurate?’ but ‘Does this technology understand people the way good care does?’

The World Health Organization has already warned of ‘potential serious negative consequences if ethical principles and human rights obligations are not prioritized by those who fund, design, regulate or use AI technologies for health.’ (9) But there is a simpler worry underneath all of this.

What we need runs the other way. We need community-based care that is accessible for people who need it most. We need an approach to mental health that invites people to look for support off their phones, away from the lens that feeds so much fear, comparison, isolation and distress. We need to be face-to-face. None of that comes from a better app. It is a systemic change, and governments are the ones who have to push for it by investing in human, community-based care instead of quietly handing the job to a chatbot.

1. Mansoor MA, Ansari KH. Early Detection of Mental Health Crises through Artifical-Intelligence-Powered Social Media Analysis: A Prospective Observational Study. J Pers Med. 2024 Sep 9;14(9):958. doi: 10.3390/jpm14090958. PMID: 39338211; PMCID: PMC11433454., https://pmc.ncbi.nlm.nih.gov/articles/PMC11433454/
2. Mental Health Europe, Mental health in the digital age: Applying a human rights based, psychosocial approach as compass (2022). https://www.mentalhealtheurope.org/what-we-do/digitalisation/
3. Alberca-González A, Fernández-Jiménez E. Strengths and limitations of the use of Artificial Intelligence in Mental Health. European Psychiatry. 2025. https://doi.org/10.1192/j.eurpsy.2025.1426
4. Torous J et al. The evolving field of digital mental health: current evidence and implementation issues. World Psychiatry. 2025;24(2):156-174. https://doi.org/10.1002/wps.21299
5. Huckvale K, Torous J, Larsen ME. Assessment of the Data Sharing and Privacy Practices of Smartphone Apps for Depression and Smoking Cessation. JAMA Network Open. 2019;2(4):e192542. https://jamanetwork.com/journals/jamanetworkopen/fullarticle/2730782
6. Radanliev P (2025) Privacy, ethics, transparency, and accountability in AI systems for wearable devices. Front. Digit. Health 7:1431246. doi: 10.3389/fdgth.2025.1431246
7. Article 22 of the GDPR
8. Regulation (EU) 2024/1689 (Artificial Intelligence Act), which classifies many health-related and emotion-recognition systems as high-risk or restricted. https://artificialintelligenceact.eu/
9. World Health Organization. Ethics and governance of artificial intelligence for health: WHO guidance. Geneva; 2021. https://iris.who.int/server/api/core/bitstreams/f780d926-4ae3-42ce-a6d6-e898a5562621/content