They develop AI agents for healthcare, specifically the kind that call people to gather information, schedule appointments, provide medical device guidance etc.

The Skills page on their website contains over 50 samples of AI-patient recorded conversations referred to as 'real de-identified calls and demos'.

There are aspects of these call fragments that are appealing, but also aspects that are deeply troubling. The AI agents constantly refer to themselves as having feelings - I lost count of the times AI said they were happy, relieved, curious, concerned, worried, grateful etc. Clearly it's language that's intended to put people at ease, but is also deceptive and misleading because AI agents can't feel happy, relieved, or anything else.

And while this is a US-based vendor, it's indicative of where healthcare is moving. AI is increasingly being positioned to interface with patients and caregivers directly, reducing the actual time we spend dealing with both health professionals and support staff directly.

I don't doubt that there will be efficiencies, but I suspect we could gain efficiencies in other areas of healthcare without reducing human contact. In fact, if we prioritized AI to reduce administrative burden, it could significantly free up time to increase human contact.

This is why I believe patients and caregivers need to be deeply involved in the implementation of AI in healthcare, so that we can help set those priorities and steer it away from the process of dehumanizing healthcare services.

Thoughts and comments welcome!