Dataset from User Study on Comprehensibility of Explainable AI Algorithms
This article presents a dataset originating from a user study focused on how people understand explanations generated by explainable artificial intelligence (XAI) algorithms. The study involved three distinct groups: mycology experts (domain experts, DE), students with data science and visualization backgrounds (IT), and students from social sciences and humanities (SSH).
The core dataset includes 39 interview transcripts where participants engaged with a machine learning model designed to classify mushrooms as edible or inedible. Participants interpreted explanations of the model's decisions, provided feedback, and suggested improvements. This material is enriched with visualizations of the explanations, thematic analysis outcomes, and initial survey data assessing participant domain knowledge and data literacy.
Background and Summary
As black-box machine learning models like deep neural networks and gradient boosting trees become prevalent, the need to explain their decisions grows, especially in critical sectors such as healthcare, law, and industry. Legislative frameworks like the EU's GDPR and the proposed AI Act emphasize AI transparency, driving interest in explainable AI.
Despite advances in XAI algorithms, a key challenge remains: ensuring explanations are comprehensible to diverse users. Research increasingly highlights that human factors must be central to XAI design and evaluation, as comprehension varies widely depending on individual knowledge and skills.
This dataset fills a gap by providing empirical data from a multidisciplinary user study. It allows replication of the original study and supports extensive analyses of how different user groups perceive and interact with XAI explanations. The study’s design acknowledges that effective explainability depends on understanding users’ cognitive processes and preferences.
Study Design and Participants
- Groups: 13 domain experts in mycology (DE), 8 IT students with data science backgrounds, and 18 SSH students.
- Model: Extreme Gradient Boosting (XGB) trained on a publicly available mushroom dataset.
- Explanations: Included methods such as SHAP, LIME, DICE, and Anchor, combined in a presentation format.
- Methodology: Think-aloud protocol (TAP) interviews recorded and transcribed to capture participants’ reasoning and reactions.
The mushroom classification task allowed participants to engage with realistic, domain-specific data. The study gathered rich qualitative data, including verbalizations during explanation interpretation and feedback on explanation clarity and usability.
Methods
Machine Learning Model and XAI Algorithms
The mushroom dataset from the University of California Irvine Machine Learning Repository contains over 61,000 specimens labeled as edible or poisonous. The dataset is balanced with approximately 55% inedible entries, covering 173 species. Artificially generated data supplemented real observations to enrich the dataset.
The XGB classifier was chosen for its high accuracy (99.97%) and efficiency. Preprocessing included one-hot encoding for categorical variables, median imputation for missing numeric data, and scaling.
The study presented multiple explanation types to participants to cover a wide range of XAI approaches:
- Feature importance methods: SHAP and LIME
- Rule-based explanations: Anchor and Local Universal Explainer (LUX)
- Counterfactual explanations: DICE
- Statistical data visualizations
These explanations were compiled into a 14-slide PDF for participants, with a 15th slide marking the study's end.
Empirical Data Collection
Data collection had two stages: an online survey and individual think-aloud protocol (TAP) interviews. The survey assessed participants’ domain knowledge and data literacy, informing group allocation.
Interviews were audio-recorded with participant consent and transcribed. Sessions with students were conducted in person, while expert interviews were remote via Microsoft Teams. Transcripts were manually corrected and tagged to align specific text fragments with tasks and questions.
Interview Tasks
- Task 1: Answer 17 questions related to each explanation type and classify mushrooms using the explanations.
- Task 2: Arrange the explanations by usefulness, remove unnecessary ones, and suggest additional explanations.
- Task 3: Provide general recommendations for improving the explanatory materials.
These tasks aimed to evaluate comprehension, gather preferences, and identify potential improvements to explanation methods.
Thematic Analysis
Thematic analysis was applied to interview transcripts to identify recurring patterns and themes across user groups. Two researchers independently coded the data, developing a codebook aligned with the study’s goals. The analysis highlighted differences in how user groups understand and value various explanations.
Tools like MAXQDA and Tableau supported systematic coding and visualization of results.
Dataset Description
The complete dataset is publicly available in the Zenodo repository. It includes:
- Manually tagged interview transcripts in Polish.
- Visual explanations presented during the study.
- Results of thematic analysis, including codebooks and theme occurrences.
- User recommendations and survey data assessing background knowledge.
- Source code to reproduce the machine learning model and explanation generation.
Transcripts are linked with other dataset components by unique identifiers, enabling integrated analysis of qualitative and quantitative data.
The mixture of Polish and English content preserves the original material's integrity, while allowing future translation as needed with modern language tools.
Applications and Impact
This dataset offers multiple practical applications:
- Replicating and extending user studies on XAI comprehension.
- Developing multimodal explainability tools combining text and visual explanations.
- Creating personalized XAI systems that adapt explanations to user knowledge and preferences.
- Informing design of interactive interfaces and recommender systems for better AI transparency.
By providing diverse user perspectives, this dataset supports more human-centered AI explanation methods and evaluation frameworks.
Further Resources
For those interested in expanding skills around AI and explainability, platforms like Complete AI Training offer courses covering AI fundamentals, interpretability, and practical applications.
Access to the dataset and detailed documentation is available at the Zenodo repository.
Your membership also unlocks:
