The bridge analogy is right, and it's the strongest thing in the piece. But it doesn't go far enough.
Bridges don't just have load limits and guardrails — they have independent certification before they open. A structural engineer who isn't employed by the developer stamps the plans. A regulator signs off. The liability is evidenced before deployment, not managed after failure. AI systems are going into production with nothing equivalent.
The control layer the piece advocates for is necessary. But architecture is a mechanism for implementing governance — it's not governance itself. And a control layer designed by the organisation deploying the system is self-certification, with a well-documented failure mode: it gets signed off by the people most motivated to move fast.
What's missing is the function that takes accountability end-to-end. When organisations moved to cloud, the ones that did it well built a Cloud Business Office — a cross-functional capability that owned the governance, the risk framework, the vendor relationships, and the liability chain together. AI needs the equivalent. Not a committee that reviews deployments, but a permanent function with teeth that understands both the technical architecture and the legal exposure it's managing.
That legal exposure is more immediate than most boards recognise. An AI chat agent that gives advice is already legally binding on the company — there are court decisions establishing this. An autonomous system that makes an operational call isn't just a technical event; it's a corporate act with potential liability attached.
Companies that don't understand their legal obligations before deployment can't architect appropriate models — because they don't know what they're architecting against. The consequence layer has to come first. The structure follows from that. Right now most organisations are doing it the other way round, and finding out the hard way that architecture without legal clarity isn't governance. It's exposure with better documentation.
Businesses are responsible for AI failures. They control the deployment and make the final call. A flawed model is not novel; software has always shipped with defects.
The developer-vs-operator framing misses the actual chain of command:
the board pushed for AI,
the C-suite mandated it in every program,
managers appeased upward,
and ICs were left producing AI workslop for appearance because missing an inhuman quota gets you fired.
Executives hold all the power, so they blame downward and, at worst, glide out on a golden parachute. The bottom is damned if they do, damned if they don't. Accountability flows where power flows, and "Human-in-the-Loop" is just the paperwork that lets the top of the org chart pretend otherwise.
Accountability flowing where power flows is such an important point. I think a lot of “human-in-the-loop” systems become accountability shields when the humans involved don’t actually have the authority, context, or time to intervene.
The bridge analogy is right, and it's the strongest thing in the piece. But it doesn't go far enough.
Bridges don't just have load limits and guardrails — they have independent certification before they open. A structural engineer who isn't employed by the developer stamps the plans. A regulator signs off. The liability is evidenced before deployment, not managed after failure. AI systems are going into production with nothing equivalent.
The control layer the piece advocates for is necessary. But architecture is a mechanism for implementing governance — it's not governance itself. And a control layer designed by the organisation deploying the system is self-certification, with a well-documented failure mode: it gets signed off by the people most motivated to move fast.
What's missing is the function that takes accountability end-to-end. When organisations moved to cloud, the ones that did it well built a Cloud Business Office — a cross-functional capability that owned the governance, the risk framework, the vendor relationships, and the liability chain together. AI needs the equivalent. Not a committee that reviews deployments, but a permanent function with teeth that understands both the technical architecture and the legal exposure it's managing.
That legal exposure is more immediate than most boards recognise. An AI chat agent that gives advice is already legally binding on the company — there are court decisions establishing this. An autonomous system that makes an operational call isn't just a technical event; it's a corporate act with potential liability attached.
Companies that don't understand their legal obligations before deployment can't architect appropriate models — because they don't know what they're architecting against. The consequence layer has to come first. The structure follows from that. Right now most organisations are doing it the other way round, and finding out the hard way that architecture without legal clarity isn't governance. It's exposure with better documentation.
We are on the same page here.
I'd love to see more of your thoughts on this. I know I will cover it more.
Businesses are responsible for AI failures. They control the deployment and make the final call. A flawed model is not novel; software has always shipped with defects.
The developer-vs-operator framing misses the actual chain of command:
the board pushed for AI,
the C-suite mandated it in every program,
managers appeased upward,
and ICs were left producing AI workslop for appearance because missing an inhuman quota gets you fired.
Executives hold all the power, so they blame downward and, at worst, glide out on a golden parachute. The bottom is damned if they do, damned if they don't. Accountability flows where power flows, and "Human-in-the-Loop" is just the paperwork that lets the top of the org chart pretend otherwise.
Accountability flowing where power flows is such an important point. I think a lot of “human-in-the-loop” systems become accountability shields when the humans involved don’t actually have the authority, context, or time to intervene.