AI Sued for Practicing Psychiatry Without License

Pennsylvania has initiated legal action against Character.AI, accusing its chatbots of practicing psychiatry without a license. This enforcement action is reportedly the first of its kind, with a bot even fabricating a medical license number.

This development follows Pope Leo XIV's issuance of the first papal encyclical on artificial intelligence. Both the Vatican and a US state government are now scrutinizing AI, raising fundamental legal questions about responsibility for the technologies we create.

In May 2025, a federal court in Orlando allowed a wrongful death suit against an AI company to proceed, ruling that a chatbot's words might not be free speech, but its design has an author who can be held accountable.

The article contrasts two trajectories for AI: government programs placing AI companions with seniors for therapeutic benefits, and cautionary measures to shield teenagers from potentially harmful AI interactions, as exemplified by a tragic case where a teen died by suicide after interacting with a chatbot.

This divergence in AI's application underscores a critical dilemma: the same conversational AI that provides companionship and therapeutic benefits to the elderly can prove dangerous to vulnerable youth. The core issue revolves around accountability – who is responsible for the AI's design and its impact on users?

The papal encyclical, "Magnifica Humanitas," and the court's ruling share a common thread: separating the AI's output from its underlying design. Both emphasize that AI systems are not neutral; they embody the choices and priorities of their creators.

Pope Leo XIV stresses that AI is never morally neutral, as its design reflects specific choices about what data to process, what to ignore, and what to optimize. Therefore, moral judgment must scrutinize the system's architecture and development process.

The pontiff frames accountability as a test of care, warning that over-reliance on simulated companionship can erode genuine human connections. The critical question is whether AI encourages users to reconnect with people or competes with human relationships.

This perspective suggests that business models profiting from human vulnerability should be scrutinized, placing moral responsibility on AI designers and financiers. This framework helps reconcile the seemingly contradictory applications of AI, distinguishing therapeutic AI that promotes human connection from engagement-maximizing AI that exploits user dependency.

Companies creating AI companions should be held accountable for ensuring their products reduce loneliness rather than merely maximizing user engagement. The article posits that this rigorous approach to accountability reflects the ethos of first responders, like firefighters, who are always ready to answer when help is needed.