AI Learns to Reason Like a Linguist?

The unique capacity of humans for language has long been debated, especially in comparison to artificial intelligence. While large language models (LLMs) can produce human-like speech, their ability to reason about language itself remains a subject of intense scrutiny. Some prominent linguists argue that LLMs merely process vast data without genuine analytical understanding, likening their capabilities to extensive marination in data rather than sophisticated learning.

However, new research from UC Berkeley and Rutgers University posits a different view. Linguists tested several LLMs with a battery of linguistic challenges, including tasks designed to assess their ability to infer and apply rules of an invented language. The goal was to ascertain if these models could engage in the kind of analytical reasoning characteristic of human linguists.

One LLM demonstrated an unexpected proficiency, capable of diagramming sentences, resolving ambiguities, and employing complex features like recursion, mirroring the analytical depth of a human graduate student. This performance challenges the notion that AI is incapable of sophisticated linguistic analysis, opening new avenues for understanding machine cognition and its potential.