Delegation Risk Homeostasis: Why More Capable AI Agents Need Not Produce Fewer Incidents

Abstract

Frontier AI systems are becoming more capable, but enterprise incidents from AI agents need not fall in proportion to model improvement. This paper develops a theory of delegation risk homeostasis. When firms delegate tasks to AI agents, better models reduce the failure risk of any fixed task. But governance-constrained firms may respond by expanding delegated scope, authority, and task complexity until their incident-absorption capacity binds again.

The result is a stable or slowly changing incident rate per unit of agent-task exposure, even as the underlying model generation improves. The framework also predicts severity drift: if incidents occur at similar frequency while delegated authority rises, failures migrate toward more consequential tasks. It further predicts release-cycle overshoot, because organizations can expand authority faster than field performance and control quality are learned. The paper identifies the relevant underwriting objects for AI-agent liability: governance capacity, exposure volume, delegated authority, control quality, release-window risk, and claim-layer containment. The implication is not that model quality is irrelevant. It is that model generation alone is not a sufficient rating variable when organizational delegation adjusts endogenously.

Keywords: AI agents, delegation, governance capacity, operational risk, insurance, incident severity

Request paper