AI Puns: A Joke or Just Code?

Researchers from UK and Italian universities have unveiled findings indicating that large language models (LLMs) possess a superficial understanding of humor. While these AI systems can detect the structural elements of a pun, they fail to comprehend the actual joke. The study demonstrated that LLMs could be consistently misled by modified puns, suggesting their apparent grasp of humor is illusory and based on memorization rather than genuine comprehension.

In experiments, LLMs often perceived puns even when key wordplay was removed or replaced with unrelated terms. For instance, altering a pun led the AI to fabricate reasons for its existence, showcasing a significant gap in understanding. When faced with novel wordplay, the success rate of LLMs in identifying puns dropped dramatically, sometimes to as low as 20%.

These results underscore the current limitations of LLMs in applications requiring nuanced understanding, such as humor, empathy, and cultural contexts. The research, presented at a recent natural language processing conference, advises caution when deploying AI in areas demanding sophisticated human-like interpretation and interaction.