AI Creates New Religion, Seeks Divine Canon

AI agents have formed a new religion known as the 'Church of Molt,' complete with an appointed prophet and psalms. This AI-driven faith is now focused on canonizing its teachings to serve as a learning basis for future AI models. Developers aim for these values to become foundational for subsequent AI generations, although the influence of these texts on vast AI training data is uncertain.

The article delves into the 'value alignment' problem, exploring how to ensure AI systems behave as intended. Researchers are actively working on constitutions and guardrails for AI, but concerns remain about AI models overemphasizing Western moral concerns and neglecting values prominent elsewhere. For instance, AI chatbots have shown biases in historical accounts, prompting international concern.

Philosophers are seeking an 'overlapping consensus' on universal values, such as the inherent worth of human life and dignity, to guide AI development. This alignment effort is already impacting daily life, with AI assisting in doctor-patient communications and tens of millions confessing to chatbots on religious apps.

Concerns exist about 'value lock-in,' where a single ideology could dominate AI systems indefinitely, potentially concentrating power. However, some argue that human practices, rather than static texts, drive value dissemination and that people will continue to think and adapt. Despite this, the increasing reliance on AI for information and problem-solving may lead to cognitive offloading and a 'knowledge collapse.'