4 Key Factors to Bridge the Gap Between Artificial Intelligence Development and Clinical Acceptance
SUMMARY:
The promise of artificial intelligence (AI) to provide timely, patient specific, evidence-based point of care decisions has remained unfulfilled.
Numerous causes for this gap include: transparency of algorithms, lack of useful clinical context, and insufficient validations across diverse populations.
The various suggestions for improvement can all be summarized into 4 key segments toward improved development and deployment.
BACKGROUND
AI consists of clinical decision support (CDS) and machine learning (ML) algorithms.
AI will be one of the most important computing applications to be developed for patient care.
AI is designed to provide patient specific, timely and appropriate recommendations including:
o Risk Assessment
o Diagnosis
o Prognosis
o Predicting Treatment Response
Despite over a decade of development, significant gaps exist between the design and development of AI algorithms and its integration into meaningful clinical workflow, care and outcomes.
The public perception of AI in healthcare based on a recent survey revealed (Khullar et al JAMA Network Open 2022, n=926):
o 55.4% believe AI will make healthcare better
o 71.5% are uncomfortable receiving a computer generated diagnosis that is incapable of explaining it’s rationale.
o The largest Concerns by the public include:
Data Confidentiality: 70.9%
Loss time with provider: 69.6%
Increased healthcare costs: 68.4%
Public Perception of AI in Healthcare
REVIEW:
Addressing technological challenges alone is unlikely to be sufficient to optimize AI adoption.
Numerous categories and sub-categories for the lack of adoption has been proposed by various authors.
All of them can typically be placed into one of 4 key factors.
Main Factors to Improve Clincial Outcomes with AI
KEY FACTOR #1: Transparency of Algorithm Development:
Specific intellectual property may remain within the “Black Box”, however, the AI calculation needs to be explained and understood by the clinician.
Descriptions are needed of the:
Data acquisition process
Selection criteria of patient cohorts
Prevalence of attributes likely to influence the model
Users should be able to determine if potential bias is or is not representative in the training data.
KEY FACTOR #2: Clinical Context:
Typical AI metrics (sensitivity, specificity, area under the receiver operating curve) do not normally include clinical context.
All correct diagnoses are treated as equally correct
All incorrect diagnoses are considered equally negative
One size DOES NOT fit all - Different clinical problems require different metrics
The goal should be to identify the salient features of a clinical problem in the design of the AI algorithm with performance metrics evaluating improved utility and outcomes.
KEY FACTOR #3: Clinical Validation:
Validate AI algorithm in populations where they will be used.
Stated clinical utility.
Transparent metrics.
Address potential algorithm bias (socioeconomic, age, etc) between training and validation data.
KEY FACTOR #4: Ongoing Monitoring:
Anticipate modifications to algorithm.
Re-validate any changes.
When should the AI algorithm become part of the standardized care process.
How will the AI algorithm change clinical behavior?
Adaptive AI systems which can learn and change performance over time can only fully be transparent post-implementation and requires performance monitoring
CONCLUSIONS:
AI implementation represent a greater disruption change than the typical new practice introduction.
Concerns do not lie with specific technical distinctions, but rather how AI can be placed into routine daily practice.
Clinicians should choose algorithms that are transparent, clinically useful and proven effective across diverse patient populations.
Subscribe to the monthly newsletter here: http://eepurl.com/gFJKYD