SRFP009: Assessing How Clinicians and Staff Would Use a Diabetes Artificial Intelligence Prediction Tool
Winston Liaw, MD, MPH; Angela Stotts, PhD; Yessenia Ramos Silva; Erica Soltero, PhD; Alex Krist, MD, MPH
Abstract
Context: Nearly one third of those with diabetes are poorly controlled (hemoglobin A1c of ≥9.0%). Identifying at-risk individuals and providing them with the right treatment is an important strategy for preventing poor control.
Objectives: To assess how clinicians and staff would use a clinical decision support tool based on artificial intelligence (AI) and identify factors that affect adoption.
Study Design: Mixed-methods study of semi-structured interviews and surveys to assess the perceived usefulness and ease of use, intent to use, and factors affecting tool adoption.
Population: Clinicians and staff in clinics that manage diabetes (N=9).
Instrument: During interviews, participants review a sample electronic health record alert and are informed that the tool uses AI to identify those at high risk for poor control. Participants discuss how they would use it, whether it would contribute to care, and the factors affecting its implementation. In a survey, participants report their demographics, rank order factors influencing adoption, and quantify (on a 7-point Likert scale from strongly disagree to strongly agree) their perceived usefulness, intent to use, ease of use, and organizational support for use.
Outcomes: We use descriptive statistics to report demographics, mean ranked score for 6 factors influencing adoption, and the percentage of participants that intend to use the tool and perceive it as useful, easy to use, and a tool their organization would support. Qualitative data are analyzed using a thematic content analysis approach.
Preliminary Results: Two-thirds of respondents are physicians, and another 66.7% identify as female. Forty-four percent work in academic health centers while a third work in Federally Qualified Health Centers. Two-thirds found the tool easy to use. Fifty-six percent found the tool to be useful, while forty-four percent thought their clinics would support their use of the tool. Half (N=8) intended to use it. The two highest-ranked factors affecting adoption were whether the tool improves health and accuracy of the tool. While interviewees thought it was useful, they were concerned about alert fatigue and the magnification of biases. To trust the tool, they needed to know why patients were identified as high risk.
Conclusion: A majority found the tool to be easy to use and useful though they had concerns about burnout, bias, and transparency. These data will be used to enhance the design of an AI tool.
Objectives: To assess how clinicians and staff would use a clinical decision support tool based on artificial intelligence (AI) and identify factors that affect adoption.
Study Design: Mixed-methods study of semi-structured interviews and surveys to assess the perceived usefulness and ease of use, intent to use, and factors affecting tool adoption.
Population: Clinicians and staff in clinics that manage diabetes (N=9).
Instrument: During interviews, participants review a sample electronic health record alert and are informed that the tool uses AI to identify those at high risk for poor control. Participants discuss how they would use it, whether it would contribute to care, and the factors affecting its implementation. In a survey, participants report their demographics, rank order factors influencing adoption, and quantify (on a 7-point Likert scale from strongly disagree to strongly agree) their perceived usefulness, intent to use, ease of use, and organizational support for use.
Outcomes: We use descriptive statistics to report demographics, mean ranked score for 6 factors influencing adoption, and the percentage of participants that intend to use the tool and perceive it as useful, easy to use, and a tool their organization would support. Qualitative data are analyzed using a thematic content analysis approach.
Preliminary Results: Two-thirds of respondents are physicians, and another 66.7% identify as female. Forty-four percent work in academic health centers while a third work in Federally Qualified Health Centers. Two-thirds found the tool easy to use. Fifty-six percent found the tool to be useful, while forty-four percent thought their clinics would support their use of the tool. Half (N=8) intended to use it. The two highest-ranked factors affecting adoption were whether the tool improves health and accuracy of the tool. While interviewees thought it was useful, they were concerned about alert fatigue and the magnification of biases. To trust the tool, they needed to know why patients were identified as high risk.
Conclusion: A majority found the tool to be easy to use and useful though they had concerns about burnout, bias, and transparency. These data will be used to enhance the design of an AI tool.
Debora Goldberg
dgoldbe4@gmu.edu 11/20/2021Yessenia, Erica, Alex, Angela and Winston, Very interesting study of the acceptance and implementation of AI to identify patients at high-risk for poor control of diabetes. Interested in learning more about the local contextual factors and environmental factors influencing providers acceptance and organizational implementation of this type of tool. Great work! Debbie