Explainable Concept Generation through Vision-Language Preference Learning for Understanding Neural Networks' Internal Representations

Aditya Taparia, Som Sagar, Ransalu Senanayake
International Conference on Machine Learning (ICML), 2025

In one sentence. RLPO frames concept-based explanation as an image generation problem and uses reinforcement learning preference optimization to fine-tune a vision-language generative model, automatically producing concept images that reveal what a neural network has internally learned.

picture_as_pdf PDF code Code play_circle Video

Abstract

Understanding the inner representation of a neural network helps users improve models. Concept-based methods have become a popular choice for explaining deep neural networks post-hoc because, unlike most other explainable AI techniques, they can be used to test high-level visual concepts that are not directly related to feature attributes. For instance, the concept of stripes is important to classify an image as a zebra. Concept-based explanation methods, however, require practitioners to guess and manually collect multiple candidate concept image sets, making the process labor-intensive and prone to overlooking important concepts. Addressing this limitation, in this paper, we frame concept image set creation as an image generation problem. However, since naively using a standard generative model does not result in meaningful concepts, we devise a reinforcement learning-based preference optimization (RLPO) algorithm that fine-tunes a vision-language generative model from approximate textual descriptions of concepts. Through a series of experiments, we demonstrate our method's ability to efficiently and reliably articulate diverse concepts that are otherwise challenging to craft manually.

Note. An earlier version appeared at the NeurIPS 2024 Workshop on Interpretable AI: Past, Present and Future.

BibTeX

@inproceedings{taparia2025explainable,
  title     = {Explainable Concept Generation through Vision-Language Preference Learning for Understanding Neural Networks' Internal Representations},
  author    = {Taparia, Aditya and Sagar, Som and Senanayake, Ransalu},
  booktitle = {International Conference on Machine Learning (ICML)},
  year      = {2025}
}

Topics

explainable AI, concept generation, preference optimization, reinforcement learning, vision-language generative models, interpretability