Artificial intelligence and machine learning techniques hold a great potential in solving complex world problems. They can facilitate clinical decision making by providing actionable insights through ‘learning’ from large volumes of patient data. Among others, deep learning algorithms were proved able to accurately identify head CT scan abnormalities requiring urgent attention, significantly increasing the efficiency of health services.
This is because data-driven AI models make inferences by finding ‘patterns’ from the data they analyse, but disparities such as those on racial and ethnic basis have long existed in health and care. Without effective mitigation approaches, inferences learnt from such biased data are inevitably channeling embedded inequities into the decisions they make.
In addition to those embedded in data, biases could also arise from methodological choices for AI development and deployment. Obermeyer and colleagues analysed an algorithm widely employed in the US that uses health costs as a proxy for health needs and they demonstrated that it falsely concludes that Black patients are healthier than equally sick White patients.
Increasingly data-heavy medical practice might require novel skills from the next generation of medical professionals with consequences for the future of medical education. In order to address these issues, implementation of critical frameworks such as embedded ethics in the development of medical AI is required.