Left unchecked, biased health care AI can do a lot of harm.
A University of California, Berkeley study found that Black patients were substantially less likely to have access to care management services when medical decisions were based on a biased algorithm, says Michael Cary, Duke School of Nursing associate professor. At Duke, Cary is part of the solution. He’s a member of Algorithm-Based Clinical Decision Support, which vets every algorithm used at Duke Health. His interdisciplinary team checks for holes in medical algorithms that patients can fall through – and fixes them. He also trains clinicians to recognize biased algorithms and use AI more responsibly.
“My research is instantly translated into practice,” Cary says, thrilled at the pace of change. “I’ve worked my entire life as a clinician and as a researcher to get to that point where it’s not an academic exercise.”
Cary is from Palmyra, Virginia, where he was raised in a self-sufficient rural Black community with little access to resources. The closest primary care was more than 45 minutes away, so anything that wasn’t life-threatening was treated by Robitussin or Tylenol, he says. Cary wondered why this gap existed and went into medicine so he could be an agent of change.
“I was going to bring that knowledge back to Palmyra,” says Cary.
Read More on AI at Duke
Over the years, his interest in health care intersected with data analysis. Then, on the advice of his supervisor at the time, Cary went into nursing. In the University of Virginia’s nursing program, Cary saw his own future when he encountered a Black male nurse for the first time.
“He was like a unicorn to me,” Cary says. “Like, where did you come from? What kind of food do you eat?”
An inspired Cary eventually got his doctorate in nursing. He wanted to improve care of older adults, and especially older adults of color. Machine learning had begun to crop up in medical settings, so Cary wondered whether AI could predict and prevent bad outcomes for these patients. Now on faculty at Duke School of Nursing, he designed a curriculum to teach himself machine learning and become equal parts nurse and data scientist. Through his algorithms, data analysis that once took months or years could now happen during an appointment. The nurse or doctor could know in real-time whether a patient has risk factors for poor outcomes like hip fracture, readmission or even death.
The bottom line: “You can couple data-informed decisions with clinical judgment at the point of care,” Cary says.