Developing a Qualitative Evaluation Framework for Explainable Decision Support Systems from a User Experience Perspective
DOI:
https://doi.org/10.59890/ijist.v4i7.29Keywords:
Explainable Decision Support Systems, User Experience, Explainable Artificial Intelligence, Qualitative Evaluation Framework, Transparency.Abstract
The growing adoption of Explainable Decision Support Systems (EDSS) across various decision-making domains has created a need for evaluation mechanisms that assess not only system performance but also the quality of user experience in understanding and trusting system recommendations. This study aims to develop a qualitative evaluation framework for EDSS from a user experience perspective by identifying the key factors influencing users’ perceptions of transparency, understandability, trust, and system usefulness. A qualitative exploratory approach was employed in this research. Data were collected through in-depth interviews and focus group discussions involving 25 participants consisting of system users, developers, and user experience experts. The collected data were analyzed using thematic analysis to identify the principal themes relevant to EDSS evaluation. The findings reveal that explanation transparency, information relevance, ease of interpretation, and user control are the primary dimensions affecting user experience and system acceptance. Based on these findings, a structured qualitative evaluation framework was developed to assess the quality of explainability in decision support systems. The proposed framework contributes to the advancement of Explainable Artificial Intelligence research and provides practical guidance for developers in designing systems that are more transparent, understandable, and user-centered.
References
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018). Trends and trajectories for explainable, accountable and intelligible systems: An HCI research agenda. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (pp. 1–18). Association for Computing Machinery. https://doi.org/10.1145/3173574.3174156
Adadi, A., & Berrada, M. (2018). Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access, 6, 52138–52160. https://doi.org/10.1109/ACCESS.2018.2870052
Al-Ansari, N., Al-Thani, D., & Al-Mansoori, R. S. (2024). User-centered evaluation of explainable artificial intelligence (XAI): A systematic literature review. Human Behavior and Emerging Technologies, 2024, Article 4628855. https://doi.org/10.1155/2024/4628855
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/10.1016/j.inffus.2019.12.012
Bernardo, E., & Seva, R. (2023). Affective design analysis of explainable artificial intelligence (XAI): A user-centric perspective. Informatics, 10(1), Article 32. https://doi.org/10.3390/informatics10010032
Braun, V., & Clarke, V. (2022). Thematic analysis: A practical guide. SAGE Publications.
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021). To trust or to think: Cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW1), Article 188, 1–21. https://doi.org/10.1145/3449287
Campbell, S., Greenwood, M., Prior, S., Shearer, T., Walkem, K., Young, S., Bywaters, D., & Walker, K. (2020). Purposive sampling: Complex or simple? Research case examples. Journal of Research in Nursing, 25(8), 652–661. https://doi.org/10.1177/1744987120927206
Chazette, L., Klös, V., Herzog, F., & Schneider, K. (2022). Requirements on explanations: A quality framework for explainability. In 2022 IEEE 30th International Requirements Engineering Conference (RE) (pp. 140–152). IEEE. https://doi.org/10.1109/RE54965.2022.00019
Creswell, J. W., & Poth, C. N. (2024). Qualitative inquiry and research design: Choosing among five approaches (5th ed.). SAGE Publications.
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021). Expanding explainability: Towards social transparency in AI systems. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (pp. 1–19). Association for Computing Machinery. https://doi.org/10.1145/3411764.3445188
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2023). Measures for explainable AI: Explanation goodness, user satisfaction, mental models, curiosity, trust, and human-AI performance. Frontiers in Computer Science, 5, Article 1096257. https://doi.org/10.3389/fcomp.2023.1096257
Kim, J., Maathuis, H., & Sent, D. (2024). Human-centered evaluation of explainable AI applications: A systematic review. Frontiers in Artificial Intelligence, 7, Article 1456486. https://doi.org/10.3389/frai.2024.1456486
Kostopoulos, G., Davrazos, G., & Kotsiantis, S. (2024). Explainable artificial intelligence-based decision support systems: A recent review. Electronics, 13(14), Article 2842. https://doi.org/10.3390/electronics13142842
Liao, Q. V., & Varshney, K. R. (2021). Human-centered explainable AI (XAI): From algorithms to user experiences. arXiv. https://doi.org/10.48550/arXiv.2110.10790
Malterud, K., Siersma, V. D., & Guassora, A. D. (2016). Sample size in qualitative interview studies: Guided by information power. Qualitative Health Research, 26(13), 1753–1760. https://doi.org/10.1177/1049732315617444
Meske, C., Bunde, E., Schneider, J., & Gersch, M. (2022). Explainable artificial intelligence: Objectives, stakeholders, and future research opportunities. Information Systems Management, 39(1), 53–63. https://doi.org/10.1080/10580530.2020.1849465
Miller, T. (2023). Explainable AI is dead, long live explainable AI! Hypothesis-driven decision support using evaluative AI. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (pp. 333–342). Association for Computing Machinery. https://doi.org/10.1145/3593013.3594001
Mohseni, S., Zarei, N., & Ragan, E. D. (2021). A multidisciplinary survey and framework for design and evaluation of explainable AI systems. ACM Transactions on Interactive Intelligent Systems, 11(3–4), Article 24, 1–45. https://doi.org/10.1145/3387166
Naveed, S., Stevens, G., & Robin-Kern, D. (2024). An overview of the empirical evaluation of explainable AI (XAI): A comprehensive guideline for user-centered evaluation in XAI. Applied Sciences, 14(23), Article 11288. https://doi.org/10.3390/app142311288
Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024). Towards human-centered explainable AI: A survey of user studies for model explanations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4), 2104–2122. https://doi.org/10.1109/TPAMI.2023.3331846
Stahl, N. A., & King, J. R. (2020). Expanding approaches for research: Understanding and using trustworthiness in qualitative research. Journal of Developmental Education, 44(1), 26–28.
Suresh, H., Lao, N., & Liccardi, I. (2020). Misplaced trust: Measuring the interference of machine learning in human decision-making. In Proceedings of the 12th ACM Conference on Web Science (pp. 315–324). Association for Computing Machinery. https://doi.org/10.1145/3394231.3397922
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (2019). Cognitive architecture and instructional design: 20 years later. Educational Psychology Review, 31(2), 261–292. https://doi.org/10.1007/s10648-019-09465-5
Tracy, S. J. (2020). Qualitative research methods: Collecting evidence, crafting analysis, communicating impact (2nd ed.). Wiley-Blackwell.
Vessey, I. (1991). Cognitive fit: A theory-based analysis of the graphs versus tables literature. Decision Sciences, 22(2), 219–240. https://doi.org/10.1111/j.1540-5915.1991.tb00344.x
