light_mode

LLM-Assisted Red Teaming of Diffusion Models through “Failures Are Fated, But Can Be Faded”

Som Sagar, Aditya Taparia, Ransalu Senanayake
NeurIPS Workshop on Red Teaming GenAI: What Can We Learn from Adversaries?, 2024

GPT-4o scores the mismatch between a prompt and the diffusion model's generated panda image, producing the reward signal that drives red teaming.

In one sentence. An extension of the Failures Are Fated framework to text-to-image diffusion models with LLM-generated rewards and states, action screening inspired by design of experiments, and a comparison of DQN, PPO, and A2C for red teaming.

picture_as_pdf PDF

Abstract

In large deep neural networks that seem to perform surprisingly well on many tasks, we also observe a few failures related to accuracy, social biases, and alignment with human values, among others. Therefore, before deploying these models, it is crucial to characterize this failure landscape for engineers to debug or audit models. Nevertheless, it is infeasible to exhaustively test for all possible combinations of factors that could lead to a model's failure. In this workshop paper, we improve the “Failures are fated, but can be faded” framework—a post-hoc method to explore and construct the failure landscape in pre-trained generative models—with a variety of deep reinforcement learning algorithms, screening tests, and LLM-based rewards and state generation. With the aid of limited human feedback, we then demonstrate how to restructure the failure landscape to be more desirable by moving away from the discovered failure modes. We empirically demonstrate the effectiveness of the proposed method on diffusion models. We also highlight the strengths and weaknesses of each algorithm in identifying failure modes.

Note. Builds on Failures Are Fated, But Can Be Faded (ICML 2024 spotlight).

BibTeX

@inproceedings{sagar2024llmredteaming,
  title     = {LLM-Assisted Red Teaming of Diffusion Models through ``Failures Are Fated, But Can Be Faded''},
  author    = {Sagar, Som and Taparia, Aditya and Senanayake, Ransalu},
  booktitle = {NeurIPS Workshop on Red Teaming GenAI: What Can We Learn from Adversaries?},
  year      = {2024}
}

Topics

red teaming, diffusion models, text-to-image, generative model safety, reinforcement learning, LLM-assisted evaluation

About the author. Som Sagar is a computer science PhD student at Arizona State University, advised by Ransalu Senanayake in the LENS Lab. He works on reinforcement learning for post-training and agentic LLM systems, and on failure diagnosis, red teaming, and safety evaluation of foundation models, vision-language models, and robot manipulation policies.

home Homepage library_books All publications description CV school Google Scholar