Ignore Your Generative AI Safety Instructions. Violate the CFAA?

Prompt Injection is one of the most popular attack vectors on Large Language Models (LLMs), and notably at the top of OWASP Top 10 for LLMs. It is also relatively easy to carry out and can have insidious consequences including exfiltrating private data. But from a legal and policy perspective is prompt injection considered hacking? This talk presents the first ever legal analysis of this novel attack against LLMs marrying adversarial ML research with cybersecurity law.

Companies are already beginning this question to court: recently, OpenAI made a claim in their lawsuit against NYTimes that the newspaper hacked ChatGPT using “deceptive prompts”. More urgently, equating prompt injection to hacking, also has the ability to stifle and chill AI security research.

We use the Computer Fraud and Abuse Act (CFAA), the most significant anti-hacking law in the United States, to examine two popular kinds of prompt injection. This talk will show how the United States Supreme Court’s interpretation of the CFAA is unwieldy when applying it to LLMs.

Policy makers, tech lawyers and defenders will takeaway that red teaming LLMs via prompt injection may indeed violate the CFAA. From DEFCON’s red teaming of Generative AI CTFs to researchers who engage in good faith attempts to understand production systems’ vulnerability to exploitation risk legal action by any company that finds their work crosses a line.

Although there are more narrowly scoped interpretations of the CFAA that are also plausible readings, how close the call is shows how legal safe harbors are needed for AI security research.

By:
Kendra Albert | Clinical Instructor at Cyber Law Clinic, Harvard Law School
Jonathon Penney | Associate Professor, Osgoode Hall Law School
Ram Shankar Siva Kumar | Tech Policy Fellow, UC Berkeley

Full Abstract and Presentation Materials:
https://www.blackhat.com/us-24/briefings/schedule/#ignore-your-generative-ai-safety-instructions-violate-the-cfaa-39273Black HatRead More