Applying AI to mental health: A follow-up conversation with technology advisor David Pérez Tenreiro
30 Jun, 20265 minute read
Following our recent newsletter exploring the rise of AI chatbot therapists and their potential risks to patients, we had the opportunity to speak with health technology advisor David Pérez Tenreiro about the growing application of artificial intelligence in mental healthcare.
David has worked on technology-focused projects for the European Commission for the past 19 years and is a member of Spain's Health Technology Assessment (HTA) group, where he helps evaluate emerging healthcare technologies.
Drawing on both his professional expertise and personal experience as a long-term mental health patient, David has offered his nuanced perspective on the promises and pitfalls of AI.
Below we’ve outlined why he believes mental healthcare is nowhere near ready for AI integration and the importance of education surrounding the technologies that are gradually becoming part of our everyday lives.
Machines don’t have emotions
David’s first reason for keeping AI well away from mental health was that machines don’t have emotions. And yet, we are seeing people increasingly treating AI chatbots as substitutes for thinking and feeling humans despite the fact they cannot understand our emotions or lived experiences.
“Machines don’t have emotions. They can’t feel emotions. They can’t feel hot or cold or sad... they can’t cry. Machines don’t have empathy. So, there’s no way they can work when applied to mental healthcare.”
He went on to say that, despite their name, there is no intelligence whatsoever inside of a machine. We still haven’t reached a point where they can act on their own and in all honesty, they shouldn’t yet be allowed to make decisions.
Mental health is incredibly complex
Of all the areas of healthcare, mental health is incredibly complex and arguably one of the least understood areas of medicine.
David emphasized that AI systems are pattern-recognition and language-generation tools rather than independent thinkers – and if clinicians themselves cannot fully understanding how the brain works, it’s impossible to conceive that AI could have any kind of psychological idea about the complexities surrounding mental health conditions.
Another point worth noting here is that AI chatbots are overly agreeable. And if the role of the therapist is to gently challenge their patients to help foster deeper insight and growth, an AI chatbot will fail on all fronts. Such machines will tell you want you want to hear instead of what you need to hear.
Free doesn’t necessarily mean free
Another major concern that David highlighted is that many people aren’t aware that they are in fact paying for ‘free’ generative AIs with their personal data.
There is a lack of understanding around the risks of sharing personal information with chatbots and how this information is neither kept private nor confidential. The reality is that anything you share is being stored and processed in huge data centers with a high likelihood of being sold to advertisers or law enforcement.
As mental health disclosures are often the most sensitive and private forms of personal data, it’s crucial to note that there’s no way of knowing how this information will be used or where it might end up.
AI can be used in other healthcare areas
While against the notion of AI chatbots substituting therapists, David does support the use of AI within other healthcare areas – particularly in large-scale data processing, pattern detection, administration, and research tasks.
He shared an example of a Spanish hospital that had stored fifty years' worth of electrocardiogram records. After feeding this immense quantity of data into an LLM, the machine analyzed the information and produced a pattern within just two hours. This would be an impossible feat for any person.
To sum up, AI is particularly useful for:
Finding patterns in decades of patient data
Assisting with diagnosis
Transcribing and organizing records
Administrative and repetitive tasks
There is a need for better education
As our conversation came to a close, David stressed how there is a need for better education. A big (and necessary) challenge involves helping the public understand how technology works and educating them about the risks of using AI without proper awareness.
“We are pushing very hard for regulators and governments to form educational programs [about AI] because the citizens need to be informed; the patients need to be informed; and the medical staff needs to be informed. If you give a person a good tool but don’t tell them how to use it, it becomes a bad tool.”
Conclusion
To conclude, as a technology adviser working at the European Commission, David Perez criticized the increasing use of AI chatbot therapists largely because machines cannot feel, and mental healthcare relies heavily on human connection.
He also raised concerns about sharing sensitive information with LLMs, how ‘free’ services are not actually free, and how, with mental health being one of the least understood areas of medicine, we are in no position to turn to AI chatbots when clinicians themselves are still figuring out how the brain works.
Ultimately, there needs to be a bigger push on helping the public understand these new technologies as well as where their data could end up because, as it stands, there is no way of truly knowing.
