UnBias-Plus AI tool detects and rewrites bias in text

[Alies Maybee]

Interesting tool developed by the Vector Institute, Canada. Take a read: UnBias-Plus AI tool detects and rewrites bias in text

[Debra Turnbull]

@Alies-Maybee

Yeah, I read that this morning... !

Very cool! and there's a free public version!

Has HUGE implications in training data for AI-Scribes. Hopefully with this we can get the Canadian version of datasets...

Now, to work on the Consent component. Exciting times!

[Debra Turnbull]

For those of you that want to test out the free version:

• UnBias-Plus

[Chris Johnston]

@Debra-Turnbull @Alies-Maybee

It’s an interesting tool for sure, and I can see it being useful in various ways. But perhaps mostly as a teaching tool to help students tone down at least the more obvious forms of bias in their writing. Again in a teaching context, the concern would be does it weaken the development of critical thinking? If students rely on a tool that identifies a subset of biased language, can they still be taught to recognize more subtle ingrained biases? It’s arguable whether that’s actually happening now, though it should be.

Deb, I’m interested in your comment about getting the “Canadian version” of datasets using this tool. Can you elaborate on that a little? What kind of datasets in particular and how this tool would be used in that context?

[Debra Turnbull]

@Chris-Johnston

So, much like the Ethical use of AI project (the one we were both on), bias gets automatically built in to AI systems = because they are fed biased input... especially AI-Scribes. The one example that comes to mind is: "Combative patient refuses to take medication" as opposed to "Patient refuses to take medication". The output decision of an AI gets skewed.

If Canada is going to produce its own training datasets, it would be good to clean up the clinician notes set that would act as input. Flag the biases and remove or re-classify them (not sure what this would look like). This increases the value of the data.

So now, I have one for you.
You mentioned training students - but which students? Med students and writing up physician-notes? How does this benefit high school students? Isn't bias under the guise of "creative writing"?

[Chris Johnston]

@Debra-Turnbull

It would certainly be an interesting research project to run a broad sample of clinical notes through the tool and see how much bias it picks up and at what severity.

Equally it would be very interesting to see what it misses, but that would be much harder to quantify of course.

In terms of students, I was thinking broadly of higher ed, not just medical or health sciences. Though I suppose it could be used at high school level as well.

As a poet, science fiction and memoir writer, the comment 'Isn't bias under the guise of creative writing?' really made me giggle. Yes - absolutely - but only in so much as all human communication (and all human thought for that matter) are also prone to bias And while this tool only gets at a tiny portion of it - even that tiny portion could be used to raise awareness of the ways in which bias is encoded not just in everything we say, but how we say it And once you start to see it, it can be difficult to unsee it.

[Kim Locke]

@Debra-Turnbull There's also the "challenging behaviour" trope of people with neurodevelopmental disorders.

"Challenging behaviour" in the autism community is understood to be "behaviours I don't like" by people without neurodevelopmental disorders.

[Chris Johnston]

@Kim-Locke @Debra-Turnbull @Alies-Maybee

Very true Kim - I've fallen into many of the 'challenging behaviour' buckets myself and my still very new doctor (new to me, to Canada, & the BC health system - but with long clinical experience) shared that many of the male doctors have described me as intimidating and combative. She was wary of me at first, now she's more wary of the doctors I've seen.

My current thinking is I have a mix of Au/ADHD - undiagnosed - and a hunger for information that apparently manifests as threatening to doctors who think I should be pacified with a pat on the head.

I dread to think what the UnBias tool would make of my records - I've read a collection of ramblings from one of my doctors and I sound positively terrifying from his perspective. It's almost enough to make me feel sorry for him - but not.

[Jenna Kedy 0]

@Alies-Maybee

This is such a cool development, and as someone who lives at the intersection of healthcare and patient partnership, my first thought was imagine how different healthcare could feel if we paid as much attention to the language in our records as we do to our lab results. Words matter. As someone who's been a patient for more than half my life, I know firsthand that your medical record isn't just documentation as it's often the first impression every new healthcare provider has of you. Long before they meet you, they've already read your chart. The language in that chart can shape expectations, influence clinical decision-making, and impact the relationship before you've even said hello. I've read my own records over the years, and I've also spoken with countless other patients who have requested theirs. It's amazing how often language slips in that probably wasn't intended to be harmful but still carries judgment. "Non-compliant." "Poor historian." "Frequent flyer." "Dramatic." "Anxious." Those words don't exist in a vacuum. They become part of your story. Sometimes they follow you from appointment to appointment, hospital to hospital, for years. One clinician writes it. The next clinician reads it and suddenly a single phrase starts shaping how you're perceived before anyone has even heard your perspective. That's why I think tools like UnBias-Plus are so interesting. Not because AI is going to magically eliminate bias but because it's helping shine a light on something we've often accepted as "just the way charts are written." I especially appreciate that it doesn't just rewrite text as it explains why something may be biased. That's huge because the goal shouldn't be to hide bias. The goal should be to help people recognize it. Education changes culture. Simply swapping words doesn't. As someone involved in digital health projects, I also think this raises a much bigger conversation. We spend a lot of time talking about bias in AI models, algorithms, and datasets which is incredibly important but we sometimes forget that AI learns from us. If our clinical notes contain assumptions... If our documentation contains stigma... If our datasets reflect decades of inequitable care then those patterns don't disappear when AI enters the picture. They scale which means addressing biased language isn't just about better communication. It's about building safer AI from the ground up. That said, I don't think tools like this should ever replace meaningful patient involvement. Patients need to be part of deciding what respectful language actually looks like. Developers might flag one word. Clinicians might focus on another. Patients might say, "Actually... here's the phrase that's been following me around for years and here's how it affects my care." That's expertise no algorithm can generate. One thing I'd love to see is patient partners helping evaluate tools like UnBias-Plus before they're widely implemented. Questions like Does the rewritten language actually feel less stigmatizing? Does it preserve clinically relevant information? Does it work across disability, chronic illness, mental health, Indigenous health, racialized communities, LGBTQIA+ communities, and other populations? Those are questions that require lived experience alongside technical expertise. I also can't help but think about the future. Imagine if clinicians received gentle prompts while writing notes. Imagine if medical students learned about stigmatizing language alongside anatomy. Imagine if healthcare organizations routinely audited documentation for patterns of bias and not to blame clinicians, but to improve systems. Imagine if patients like me could review our notes and have a clear way to flag language that felt inaccurate or stigmatizing. That's the kind of future I'd love to see. Tools like UnBias-Plus won't solve bias overnight. Bias is a human problem before it's an AI problem but if a free, open-source tool gets more people thinking critically about the words they use, the assumptions they make, and the impact those words have on real people, that's a pretty exciting place to start. As healthcare becomes increasingly AI-enabled, I hope we remember something simple: The most advanced technology in the world still needs humanity. Better AI starts with better data. Better data starts with better language and better language starts with listening to the people those words are describing!

[Debra Turnbull]

@Jenna-Kedy-0 @Chris-Johnston @Kim-Locke @Alies-Maybee

Yup, there are a myriad of descriptors that get attached to patients, and travel the system in their charts. They usually get generated in physician notes.

Next week, the CPCRC: CATALYST-PC presentation will be discussing how to deal with these physician notes (locked into text ramblings) - in a way to make them interoperablen (I assume). This means slicing and dicing the text ramblings into discreet(?) information parcels for electronic exchange. There are many questions on how valid the output will be...

Now, for those of us that registered, the question that needs asking is - will these texts be filtered through the UnBias Tool?

[Jenna Kedy 0]

@Debra-Turnbull This is such an interesting question, and honestly, I hope someone asks it. If we're taking years' worth of physician notes that may contain subjective language and turning them into structured, interoperable data, then we also have to ask what exactly are we preserving? Interoperability is a fantastic goal. As someone who has navigated multiple healthcare systems, provinces, specialists, hospitals, and the transition from pediatric to adult care, I know how frustrating it is when information doesn't follow you. Better information sharing has enormous potential to improve care but if biased language is converted into structured data without being critically examined first, are we making healthcare more connected or simply making bias more portable? I've read parts of my own chart over the years, and like many patients, I've noticed language that didn't fully reflect my experience. Once something is written in a chart, it has a way of following you. Every new clinician reads it. Every future interaction is, at least in part, influenced by it. Now imagine that same information becoming even easier to share across organizations. That raises some really important questions. Would a tool like UnBias-Plus help identify language that could unintentionally introduce bias into structured records? and perhaps most importantly are patients involved in answering those questions? I also think we need to be careful not to assume that removing certain words automatically removes bias. Bias can exist in what is documented, what is omitted, how information is categorized, and even how AI interprets structured data. Technology can help identify patterns, but meaningful review still requires clinicians, informaticians, and patients working together. I'll definitely be interested to hear how the presenters address this. It feels like a great opportunity to ask not only "How do we make clinical notes interoperable?" but also "How do we make sure the information we're sharing is fair, accurate, and truly reflects the patient's experience?" because if we're redesigning the future of health information, this is the perfect time to improve both its interoperability!

[Debra Turnbull]

@Jenna-Kedy-0 said in UnBias-Plus AI tool detects and rewrites bias in text:

"How do we make sure the information we're sharing is fair, accurate, and truly reflects the patient's experience?"

The Validation Plan.

[Jenna Kedy 0]

@Debra-Turnbull AI can absolutely help catch biased language or wording we might miss, but it shouldn't be the final boss. The Validation Plan is where the magic happens. It's basically asking, "Hey, patient partners; did we actually get this right?" AI can be a great first draft, but lived experience is the gold standard. If the people whose stories we're trying to tell don't see themselves in the final product, we still have work to do!