EXPLAINS 2024 Abstracts


Area 1 - Technology

Full Papers
Paper Nr: 15
Title:

Feature Importance for Deep Neural Networks: A Comparison of Predictive Power, Infidelity and Sensitivity

Authors:

Lars Fluri

Abstract: This paper evaluates the effectiveness of different feature importance algorithms employed on a neural network, focused on target prediction tasks with varying data complexities. The study reveals that the feature importance algorithms excel with data featuring minimal correlation between the attributes. However, their determination considerably decreases with escalating levels of correlation, while the inclusion of irrelevant features has minimal impact on determination. In terms of predictive power, DeepLIFT surpasses other methods for most data cases, but falls short in total infidelity. For more complex cases, Shapley Value Sampling outperforms DeepLIFT. In an empirical application, Integrated Gradients and DeepLIFT demonstrate lower sensitivity and lower infidelity, respectively. this paper highlights interesting dynamics between predictive power and fidelity in feature importance algorithms and offers key insights for their application in complex data scenarios.
Download

Paper Nr: 18
Title:

XAIMed: A Diagnostic Support Tool for Explaining AI Decisions on Medical Images

Authors:

Mattia Daole, Pietro Ducange, Francesco Marcelloni, Giustino Claudio Miglionico, Alessandro Renda and Alessio Schiavo

Abstract: Convolutional Neural Networks have demonstrated high accuracy in medical image analysis, but the opaque nature of such deep learning models hinders their widespread acceptance and clinical adoption. To address this issue, we present XAIMed, a diagnostic support tool specifically designed to be easy to use for physicians. XAIMed supports diagnostic processes involving the analysis of medical images through Convolutional Neural Networks. Besides the model prediction, XAIMed also provides visual explanations using four state-of-art eXplainable AI methods: LIME, RISE, Grad-CAM, and Grad-CAM++. These methods produce saliency maps which highlight image regions that are most influential for a model decision. We also introduce a simple strategy for aggregating the different saliency maps into a unified view which reveals a coarse-grained level of agreement among the explanations. The application features an intuitive graphical user interface and is designed in a modular fashion thus facilitating the integration of new tasks, new models, and new explanation methods.
Download

Paper Nr: 26
Title:

Prediction of Alzheimer Disease on the DARWIN Dataset with Dimensionality Reduction and Explainability Techniques

Authors:

Alexandre Moreira, Artur Ferreira and Nuno Leite

Abstract: The progressive degeneration of nerve cells causes neurodegenerative diseases. For instance, Alzheimer and Parkinson diseases progressively decrease the cognitive abilities and the motor skills of an individual. Without the knowledge for a cure, we aim to slow down their impact by resorting to rehabilitative therapies and medicines. Thus, early diagnosis plays a key role to delay the progression of these diseases. The analysis of handwriting dynamics for specific tasks is found to be an effective tool to provide early diagnosis of these diseases. Recently, the Diagnosis AlzheimeR WIth haNdwriting (DARWIN) dataset was introduced. It contains records of handwriting samples from 174 participants (diagnosed as having Alzheimer’s or not), performing 25 specific handwriting tasks, including dictation, graphics, and copies. In this paper, we explore the use of the DARWIN dataset with dimensionality reduction, explainability, and classification techniques. We identify the most relevant and decisive handwriting features for predicting Alzheimer. From the original set of 450 features with different groups, we found small subsets of features showing that the time spent to perform the in-air movements are the most decisive type of features for predicting Alzheimer.
Download

Paper Nr: 27
Title:

LLM-Generated Class Descriptions for Semantically Meaningful Image Classification

Authors:

Simone Bertolotto, André Panisson and Alan Perotti

Abstract: Neural networks have become the primary approach for tackling computer vision tasks, but their lack of transparency and interpretability remains a challenge. Integrating neural networks with symbolic knowledge bases, which could provide valuable context for visual concepts, is not yet common in the machine learning community. In image classification, class labels are often treated as independent, orthogonal concepts, resulting in equal penalization of misclassifications regardless of the semantic similarity between the true and predicted labels. Previous studies have attempted to address this by using ontologies to establish relationships among classes, but such data structures are generally not available. In this paper, we use a large language model (LLM) to generate textual descriptions for each class label, aiming to capture the visual characteristics of the corresponding concepts. These descriptions are then encoded into embedding vectors, which are used as the ground truth for training the image classification model. By employing a cosine distance-based loss function, our approach considers the semantic similarity between class labels, encouraging the model to learn a more hierarchically structured internal feature representation. We evaluate our method on multiple datasets and compare its performance with existing techniques, focusing on classification accuracy, mistake severity, and the emergence of a hierarchical structure in the learned concept representations. The results suggest that semantic embedding representations extracted from LLMs have the potential to enhance the performance of image classification models and lead to more semantically meaningful misclassifications. A key advantage of our method, compared to those that leverage explicit hierarchical information, is its broad applicability to a wide range of datasets without requiring the presence of pre-defined hierarchical structures.
Download

Paper Nr: 30
Title:

TX-Gen: Multi-Objective Optimization for Sparse Counterfactual Explanations for Time-Series Classification

Authors:

Qi Huang, Sofoklis Kitharidis, Thomas Bäck and Niki van Stein

Abstract: In time-series classification, understanding model decisions is crucial for their application in high-stakes domains such as healthcare and finance. Counterfactual explanations, which provide insights by presenting alternative inputs that change model predictions, offer a promising solution. However, existing methods for generating counterfactual explanations for time-series data often struggle with balancing key objectives like proximity, sparsity, and validity. In this paper, we introduce TX-Gen, a novel algorithm for generating counterfactual explanations based on the Non-dominated Sorting Genetic Algorithm II (NSGA-II). TX-Gen leverages evolutionary multi-objective optimization to find a diverse set of counterfactuals that are both sparse and valid, while maintaining minimal dissimilarity to the original time series. By incorporating a flexible reference-guided mechanism, our method improves the plausibility and interpretability of the counterfactuals without relying on predefined assumptions. Extensive experiments on benchmark datasets demonstrate that TX-Gen outperforms existing methods in generating high-quality counterfactuals, making time-series models more transparent and interpretable.
Download

Short Papers
Paper Nr: 13
Title:

Fitness Landscape Analysis of a Cell-Based Neural Architecture Search Space

Authors:

Devon Tao and Lucas Bang

Abstract: Neural Architecture Search (NAS) research has historically faced issues of reproducibility and comparability of algorithms. To address these problems, researchers have created NAS benchmarks for NAS algorithm evaluation. However, NAS search spaces themselves are not yet well understood. To contribute to an understanding of NAS search spaces, we use the framework of fitness landscape analysis to analyze the topology search space of NATS-Bench, a popular cell-based NAS benchmark. We examine features of density of states, local optima, fitness distance correlation (FDC), fitness distance rank correlations, basins of attraction, neutral networks, and autocorrelation in order to characterize the difficulty and describe the shape of the NATS-Bench topology search space on CIFAR-10, CIFAR-100, and ImageNet16-120 image classification problems. Our analyses show that the difficulties associated with each fitness landscape could correspond to the difficulties of the image classification problems themselves. Furthermore, we demonstrate the importance of using multiple metrics for a nuanced understanding of an NAS fitness landscape.
Download

Paper Nr: 21
Title:

Explaining Explaining

Authors:

Sergei Nirenburg, Marjorie McShane, Kenneth W. Goodman and Sanjay Oruganti

Abstract: Explanation is key to people having confidence in high-stakes AI systems. However, machine-learning-based systems – which account for almost all current AI – can’t explain because they are usually black boxes. The explainable AI (XAI) movement hedges this problem by redefining “explanation”. The human-centered explainable AI (HCXAI) movement identifies the explanation-oriented needs of users but can’t fulfill them because of its commitment to machine learning. In order to achieve the kinds of explanations needed by real people operating in critical domains, we must rethink how to approach AI. We describe a hybrid approach to developing cognitive agents that uses a knowledge-based infrastructure supplemented by data obtained through machine learning when applicable. These agents will serve as assistants to humans who will bear ultimate responsibility for the decisions and actions of the human-robot team. We illustrate the explanatory potential of such agents using the under-the-hood panels of a demonstration system in which a team of simulated robots collaborates on search task assigned by a human.
Download

Paper Nr: 23
Title:

Explainability Applied to a Deep-Learning Based Algorithm for Lung Nodule Segmentation

Authors:

Arman Zafaranchi, Francesca Lizzi, Alessandra Retico, Camilla Scapicchio and Maria Evelina Fantacci

Abstract: Deep learning and computer-aided detection (CAD) methods play a pivotal role in the early detection and diagnosis of various cancer types. The significance of AI in the medical field has become particularly pronounced during the coronavirus pandemic. This study aims to develop a deep learning-based system for segmenting and detecting nodules in the lung parenchyma, utilizing the Luna-16 challenge dataset. The algorithm is divided into two phases: the first phase involves lung segmentation using the previously developed LungQuant algorithm to identify the region of interest (ROI), and the second phase employs a specifically designed and fine-tuned Attention Res-UNet for nodule segmentation. Additionally, the explainable AI (XAI) technique, Grad-CAM, was used to demonstrate the reliability of the proposed algorithm for clinical application. In the initial phase, the LungQuant algorithm achieved an average Dice Similarity Coefficient (DSC) of 90%. For nodule segmentation, the DSC scores were 81% test sets. The model also achieved average sensitivity and specificity metrics of 0.86 and 0.92.
Download

Paper Nr: 16
Title:

Analyzing Exact Output Regions of Reinforcement Learning Policy Neural Networks for High-Dimensional Input-Output Spaces

Authors:

Torben Logemann and Eric MSP Veith

Abstract: Agent systems based on deep reinforcement learning have achieved remarkable success in recent years. They have also been applied to a variety of research topics in the field of power grids, as such agents promise real resilience. However, deep reinforcement learning agents cannot guarantee behavior, as the mapping of the entire input space to the output of even a simple feed-forward neural network cannot be accurately explained. For critical infrastructures, such black box models are not acceptable. To ensure an optimized trade-off between learning performance and explainability, this paper relies on efficient regularizable feed-forward neural networks and presents an extension of the algorithm NN2EQCDT to transform the networks into pruned decision trees with significantly fewer nodes to be accurately explained. In this paper, we present a methodological approach to further analyze the decision trees for high-dimensional input-output spaces and analyze an agent for a power grid experiment.
Download

Paper Nr: 22
Title:

Graph Hierarchy and Language Model-Based Explainable Entity Alignment of Knowledge Graphs

Authors:

Kunyoung Kim, Donggyu Kim and Mye Sohn

Abstract: This paper proposes a Graph hierarchy and Language model-based Explainable Entity Alignment (GLEE) framework to perform Entity Alignment (EA) between two or more Knowledge Graphs (KGs) required for solving complex problems. Unlike existing EA methods that generate embedding for entities using KG structure information to calculate the similarity between entities, the GLEE framework additionally utilizes graph hierarchy and datatype properties to find entities to be aligned. In the GLEE framework, the semantically similar hyper-entities of the entities to be aligned are discovered to reflect graph hierarchy in the alignment. Also, the semantically similar datatype properties and their values of the entities are also utilized in EA. At this time, language model is utilized to calculate semantic similarity of the hyper-entities or datatype properties. As a result, the GLEE framework can trustworthy explain why the two entities are aligned using the subgraphs that consist of similar hyper-entities, semantically identical properties, and their data values. To show the superiority of the GLEE framework, the experiment is performed using real world dataset to prove the EA performance and explainability.
Download

Paper Nr: 25
Title:

Computer Vision Based Smart Security System for Explainable Edge Computing

Authors:

N. Nisbet and J. I. Olszewska

Abstract: Smart cities aim to reach a high quality of life for their citizens using digital infrastructure which in turn need to be sustainable, secure, and explainable. For this purpose, this work is about the development of an explainable-by-design smart security application which involves an intelligent vision system capable of running on a small, low-powered edge computing device. Hence, the device provides facial recognition and motion detection functionality to any electronic motion picture input such as CCTV camera feed or video. It highlights the practicality and usability of the device through a support application. Our resulting explainable edge computing system has been successfully applied in context of smart security in smart cities.
Download

Area 2 - Applications

Short Papers
Paper Nr: 24
Title:

Neuromorphic Encoding / Decoding of Data-Event Streams Based on the Poisson Point Process Model

Authors:

Viacheslav Antsiperov

Abstract: The work is devoted to a new approach to neuromorphic encoding of streaming data. An essential starting point of the proposed approach is a special (sampling) representation of input data in the form of a stream of discrete events (counts), modeling the firing events of biological neurons. Considering the specifics of the sampling representation, we have formed a generative model for the primary processing of the count stream. That model was also motivated by known neurophysiological facts about the structure of receptive fields of sensory systems of living organisms that implement universal mechanisms (including central-circumferential inhibition) of biological neural networks, particularly the brain. To list the main ideas and consolidate the notations used, the article provides a brief overview of the features and most essential provisions of the proposed approach. The new results obtained within the framework of the approach, related to the analysis of neuromorphic encoding (with distortions) of streaming data, are discussed. The issues of possible decoding/restoration of the original data are discussed in the context of what Marr called the primary sketch. The results of computer modelling of the developed encoding/decoding procedures are presented, approximate numerical characteristics of their quality are given.
Download

Paper Nr: 14
Title:

Design of an Iterative Method for Deep Multimodal Feature Fusion in Heart Disease Diagnostics Utilizing Explainable AI

Authors:

Sony K. Ahuja, Deepti D. Shrimankar and Aditi R. Durge

Abstract: This research addresses the critical need for advanced diagnostic methodologies in heart disease, a leading cause of mortality worldwide. Traditional diagnostic models, which often analyze genomic, clinical, and medical imaging data in isolation, fall short in providing a holistic understanding of the disease due to their fragmented approach. Such methods also grapple with significant challenges including data privacy concerns, lack of interpretability, and an inability to adapt to the continuously evolving landscape of medical data samples. In response, this study introduces an innovative approach known as Deep Multimodal Feature Fusion, designed to integrate genomic data, clinical history, and medical imaging into a cohesive analysis framework. This method leverages the unique strengths of each data modality, offering a more comprehensive patient profile than traditional, one-dimensional analyses. The integration of Explainable Artificial Intelligence with Clinical Data Interpretation enhances model transparency and interpretability, crucial for healthcare applications. The use of Transfer Learning with Pre-trained Models on medical imaging data and Continual Learning for Adaptive Genomics ensures diagnostic accuracy and model adaptability over temporal instance sets. Federated Learning for Privacy-Preserving Analysis is employed to address data privacy, allowing for collaborative model training without compromising patient confidentiality. Testing across diverse datasets demonstrated substantial improvements in diagnostic Precision, Accuracy, Recall, and other metrics, indicating a major advancement over existing methods. Practically, it exemplifies the application of advanced AI techniques in clinical settings, narrowing the gap between theoretical research and practical healthcare solutions.
Download

Paper Nr: 17
Title:

Deep Learning and Multi-Objective Evolutionary Fuzzy Classifiers: A Comparative Analysis for Brain Tumor Classification in MRI Images

Authors:

Giustino Claudio Miglionico, Pietro Ducange, Francesco Marcelloni and Witold Pedrycz

Abstract: This paper presents a comparative analysis of Deep Learning models and Fuzzy Rule-Based Classifiers (FBRCs) for Brain Tumor Classification from MRI images. The study considers a publicly available dataset with three types of brain tumors and evaluates the models based on their accuracy and complexity. The study involves VGG16, a convolutional network known for its high accuracy, and FBRCs generated via a multi-objective evolutionary learning scheme based on the PAES-RCS algorithm. Results show that VGG16 achieves the highest classification performance but suffers from overfitting and lacks interpretability, making it less suitable for clinical applications. In contrast, FBRCs, offer a good balance between accuracy and explainability. Thanks to their straightforward structure, FRBCs provide reliable predictions with comprehensible linguistic rules, essential for medical decision-making.
Download