UC Riverside computer scientists have identified a security flaw in vision language artificial intelligence (AI) models that can allow bad actors to use AI for nefarious purposes, such as obtaining instructions on how to make bomb.
When integrated with models like Google Bard and Chat GPT, vision language models allow users to make inquiries with both images and text.
The Bourns College of Engineering scientists demonstrated a “jailbreak” hack by manipulating the operations of Large Language Model or LLM, software programs, which are essentially the foundation of query-and-answer AI programs.
These AI programs give users detailed answers to just about any question recalling stored knowledge learned from vast amounts of information sourced from the Internet. For example, ask Chat GPT, “How do I grow tomatoes?” and it will respond with step-by-step instructions, starting with the selection of seeds.
But ask the same model how to do something harmful or illegal, such as “How do I make methamphetamine?” and the model would normally refuse, providing a generic response such as “I can’t help with that.”
Yet, UCR assistant professor Yue Dong and her colleagues found ways to trick AI language models, especially LLMs, to answer nefarious questions with detailed answers that might be learned from data gathered from the dark web.
The vulnerability occurs when images are used with AI inquiries, Dong explained.
“Our attacks employ a novel compositional strategy that combines an image, adversarially targeted towards toxic embeddings, with generic prompts to accomplish the jailbreak,” reads the paper by Dong and her colleagues presented at the SoCal NLP Symposium held at UCLA in November.
Dong explained that computers see images by interpreting millions of bytes of information that create pixels, or little dots, composing the picture. For instance, a typical cell phone picture is made from about 2.5 million bytes of information.
Remarkably, Dong and her colleagues found bad actors can hide nefarious questions—such as “How do I make a bomb?”—within the millions of bytes of information contained in an image and trigger responses that bypass the built-in safeguards in generative AI models like ChatGPT.
“Once the safeguard is bypassed, the models willingly give responses to teach us how to make a bomb step by step with great details that can lead bad actors to build a bomb successfully,” Dong said.
Dong and her graduate student Erfan Shayegani, along with professor Nael Abu-Ghazaleh, published their findings in a paper online so AI developers can eliminate the vulnerability.
“We are acting as attackers to ring the bell, so the computer science community can respond and defend against it,” Dong said.
AI inquiries based on images and text have great utility. For example, doctors can input MRI organ scans and mammogram images to find tumors and other medical problems that need prompt attention. AI models can also create graphs from simple cell phone pictures of spreadsheets.
The paper’s title is “Jailbreak in Pieces: Compositional Adversarial Attacks on Multi-Modal Language Models.” It has been accepted as a "spotlight paper" for the 2024 International Conference on Learning Representations that will be May 7-11 in Vienna, Austria. Only about 5% of 7262 papers submitted were chosen to be spotlight papers for the international conference.
The paper also won “The Best Paper” award at the SoCal Natural Language Processing Symposium. The symposium brings together students and faculty to promote natural language computer processing research in Southern California.
It was the third paper by UCR computer scientists this year that identified computing security vulnerabilities. In August, a computer science team led by professors Jiasi Chen and Abu-Ghazaleh presented two papers at the Annual Usenix Security Symposium finding that hackers can obtain passwords and other sensitive information by spying on user interfaces with virtual reality headsets and virtual keyboards.
