Evaluation of satisfaction and usability of a Clinical Decision Support System (CDSS) targeted for early obstetric risk assessment and patient follow-up

Introduction: Usability of computer systems, and satisfaction of their users, has been related to user performance, especially for clinical decision support systems (CDSS). The purpose of this paper is to present an evaluation of satisfaction and usability of a web-based CDSS called HADA. HADA was developed to improve antenatal healthcare by assisting in obstetric risk assessment. Methods: This research used an observational study of blocks of user interactions with multiple treatments. Twenty physicians of two rural hospitals were HADA was tested performed eleven independent tasks, using it. Subjects answered the NASA TLX self-assessment questionnaire1 after performing each task. Once they finished the set of tasks, they answered the Post-Study System Usability Questionnaire (PSSUQ) 2,3. Results: The data was not normally distributed; therefore non-parametric approaches were used for analysis. Workload was measured to be low and satisfaction to be high. Main tasks related to CDSS scored in the lowest range of findings. There was a statistically significant difference of Overall NASA TLX cognitive load medians between tested users at both hospitals. Median differences among types of physicians were not statistically significant. PSSUQ results showed a high overall acceptance. System usefulness was highest on the independent subscales, followed by interface quality and then information quality. Discussion: Experiences of users does not seem to be a factor that affected user evaluations of HADA. The reduction on workload on similar tasks suggested a learning effect. HADA’s match to current workflow seems to be a factor in perception of workload and acceptance of the tool. The users at the hospital that had no comparable processes experienced a higher workload and lower acceptability. Conclusions: HADA proved to be a tool that generally results in low additional workload for clinicians independent of their experience, and is rated high in acceptability. The design and development techniques used for HADA have resulted in a tool with low workload and high acceptability. The evaluation methods used have proven useful in this clinical system context and confirm previous research that usability of computer systems, user satisfaction levels and user performance, are related.