Judge Cautions AI Prompt Injection Risk

A Connecticut judge has identified what appears to be the first instance of a US plaintiff attempting to hide text within court filings, designed to be read only by artificial intelligence systems. Judge Walter Spader Jr. noted that while this hidden text did not impact the case's outcome, the tactic sets a concerning precedent.

The plaintiff, Matthew Elliott, attempted to use prompt injection by shrinking tiny text and coloring it white on a white background. These instructions, intended to be invisible to humans but legible to AI, aimed to sway the court's decision after his earlier arguments were dismissed.

Elliott continued to hide messages in new pleadings even after being warned, calling later prompts "jokes." These included a link to a YouTube video and nonsensical messages. Judge Spader found this behavior stunning and a serious litigation abuse.

Connecticut's judicial branch does not currently use AI to review filings, so there was no direct risk of the court's AI being misled. Elliott claimed his intent was to audit the court for fears of AI bias, but the judge suggested that visible communication would have been appropriate if that were his concern.

Sanctions were deemed warranted because the hidden messages attempted covert communication, excluding defendants from a fair process. However, the judge prohibited Elliott from e-filing in the future, opting against monetary penalties for the pro se litigant.

This case highlights a growing trend where individuals, particularly pro se litigants inexperienced with AI, may misuse chatbots. Judge Spader noted that building arguments backward by only seeking favorable AI responses can entrench litigants in their positions, leading to dishonest advocacy.

While this US case appears to be the first of its kind, a similar attack occurred in Brazil, resulting in monetary sanctions. However, prompt injection has largely proven ineffective, with AI systems or human review exposing the hidden text. Judge Spader emphasized that courts need to develop rules addressing prompt injection as AI technology and its misuses rapidly advance.