Is Your Artificial Intelligence Biased?

SUMMARY:

Pre-existing healthcare bias in artificial intelligence (AI) models may impact performance outcomes.
Unknown encoded patient characteristics into an AI models will result in significant performance disparities between groups of patients.
Current understanding of these limitations is limited, however, AI models need fairness, transparency and accountability prior to selection for use

The use of AI is revolutionizing clinician accuracy, providing timely diagnoses and in the development of treatment plans.
However, if the AI model has a pre-existing healthcare bias toward inequality, the results could cause a significant disparity in performance between patient subgroups.
Existing AI models may already be encoded with biases which are not effectively explained by the clinical features of the model resulting in an amplification of suboptimal outcomes.
In addition to evaluating the clinical features and readiness of AI models, clinicians need to evaluate potential bias.

4 COMMON (but not only) SOURCES OF BIAS:

Dataset Composition:
- Underrepresented disadvantaged subgroups in the development group.
- Bias can also be introduced as the AI learns from the dataset.
- Solution:
  - Ensure dataset is pre-processed.
  - Bias within the dataset should be documented.
  - Reweighting of unintended features to provide statistically independent outcomes.
Unintended Patient Sensitive Information:
- AI models can identify patient characteristics (race, gender) and use them in outcomes, even when the characteristics are not associated with the diagnosis.
- Even after removal of sensitive information, AI models can use other factors for predictions.
- Solution:
  - A framework offering a metric based approach to ensure similar patients are indeed grouped together and similar.
Inherited Disparities:
- AI models can inherit disparities from medical data by learning to depend on relationships from unrelated input features and using them to generate outcomes.
- Lack of clarity of which data is used in model predictions contributes to these disparities.
- Solution:
  - Detailed understanding how the encoded input parameters are to be used in the model prediction algorithm.
  - Algorithmic transparency
AI Learning and Evolution Bias:
- Limited data exists on the impact of AI learning and algorithm evolution from additional data on potential bias of outcomes.
- Solution:
  - De-biasing techniques employed during the intra and post-processing phase to minimize performance disparities.

For many existing AI applications, inherent bias may already be embedded within the algorithm, contributing to questionable outcomes.
The concern includes encoded patient characteristics as well as AL model learning, leading to disparities in patient subgroups.
Evaluation of AI models to minimize bias would include evidence of advanced fairness, transparency and accountability.