AI Agents Act, But Do Developers Ensure Safety?

Agentic AI systems, capable of planning and executing complex, multistep tasks with minimal human oversight, are rapidly being deployed. These advanced systems can manage workflows and interact with desktop tools, acting autonomously on behalf of users. Researchers cataloging these systems found a concerning trend: developers are eager to showcase capabilities but are significantly less transparent about safety features and risk assessments.

While developers widely share demos and benchmarks, a striking imbalance emerges regarding safety. Approximately 70% of documented agents provide some information, but only about 19% have a formal safety policy, and fewer than 10% report external safety evaluations. This lack of structured transparency is particularly worrisome as these agents operate in sensitive environments like software engineering, potentially handling critical data and control.

The MIT AI Agent Index highlights that as AI autonomy increases, the structured documentation of its safety measures has not kept pace. The report does not claim agentic AI is inherently unsafe but emphasizes that the public availability of safety guardrails lags behind the technology's accelerating capabilities, leaving users with limited insight into potential risks.