Evaluating Perceived Workload, Usability and Usefulness of Artificial Intelligence Systems in Low-Resource Settings: Semi-automated Classification and Detection of Community Acquired Pneumonia

The use of Artificial Intelligence (AI) techniques in radiological workflows is increasingly becoming mainstream. However, the uptake of AI techniques is still low in low-resource settings in places such as the Global South. This paper presents a study conducted in a setting with low AI uptake, to determine the impact of AI on Radiologists’ workload when interpreting medical images. Two (2) AI models—a classification model and detection model indicating potential areas of interest—were implemented to facilitate the semi-automated interpretation of medical images for Pneumonia. In addition, a Web-based DICOM Viewer was implemented to interface the AI models. To determine the appropriate model configuration, two (2) experts—a Radiologist and Radiology Resi- dent—participated in a focus group discussion aimed at determining how the AI models could facilitate interpretation processes. A comparative controlled experiment was subsequently conducted with 12 Radiology Residents at a large University Teaching Hospital, to assess the impact of AI on the workload and its perceived usefulness. NASA Task Load Index (TLX) and Technology Acceptance Model (TAM) 2 questionnaires were employed to measure the workload and usefulness. The results in- dicate that the perceived workload is significantly less when using the AI solution, with an overall NASA-TLX score of 1.86. Furthermore, the perceived usefulness of the AI solution is demonstrated through the positive responses for all the eight TAM 2 constructs. This study experimen- tally demonstrates the potential of utilising AI for the semi-automated interpretation of medical images in low-resource settings.