Why "human in the loop" is too weak a guarantee for legal work. How review states, externally-effective work boundaries, and promotion discipline can be expressed as system properties rather than policy hopes.
Governance·Approval-gated execution·Architectural property
FlowCounsel Research · 2026-05-30
Abstract
Most legal AI vendors describe their safety posture as "human in the loop" or "attorney is always in control." Those phrases describe an expectation, not an architectural property. They fail at scale because they depend on user behavior rather than data-model enforcement, and they fall short of the explicit supervisory obligations under ABA Model Rule 5.3 and ABA Formal Opinion 512 (2024). This paper presents approval-gated execution as an architectural alternative: review state expressed as a typed transition on every artifact, externally-effective work gated behind reviewer-identity-bound state changes, promotion-of-pattern distinguished from approval-of-artifact, and audit as a queryable product surface. Four common failure modes when approval is implemented as UI policy rather than architecture are catalogued. The framing draws on the human-feedback alignment work of Christiano et al. (2017), Ouyang et al. (2022), and Bai et al. (2022) to distinguish artifact-level approval from pattern-level promotion.
Most legal AI vendors describe their safety posture with some version of "the human remains in the loop" or "the attorney is always in control." These phrases describe an expectation about user behavior, not a property of the system. A firm running a tool whose only safety mechanism is user discipline is one bypass, one workflow shortcut, or one badly worded prompt away from work leaving the firm without review (FlowCounsel™, "Why Sanctions Keep Rising as AI Spreads Through Legal Work," April 2026; FlowCounsel, "When Policies Are Not Enough: Sullivan & Cromwell, AI Architecture, and the Tool-Selection Problem," April 2026).
The expectation framing also falls short of the supervisory obligations expressed in ABA Model Rule 5.3 and clarified in ABA Formal Opinion 512 (2024), both of which apply to nonlawyer assistance (including AI tools) and require lawyers to ensure the assistance operates compatibly with the lawyer's professional obligations. A safety posture that depends on whether the user happens to click "review" is not the structural assurance the rule contemplates.
The architectural alternative is approval-gated execution. Review states are part of the data model. Externally-effective work cannot transition out of the system without an explicit, reviewer-identity-bound state change. The protection holds at scale because it is a system property, not a user behavior. The remainder of this paper specifies the boundary, the review-state machine, the distinction between approval-of-artifact and promotion-of-pattern, audit as product, and four failure modes that appear when approval is implemented as policy.
"Human in the loop" is an expectation about user behavior. Approval-gated execution is a property of the data model.
Not every system action carries the same supervisory weight. The architectural question is whether the system distinguishes between internal scratch work and externally-effective work, and whether the externally-effective work runs under a different transition discipline. The boundary is the line that ABA Formal Op. 512 (2024) implicitly treats as the supervisory checkpoint: the system action that becomes the lawyer's output to the world.
Externally-effective work is anything that leaves the firm: a demand letter sent to opposing counsel, a draft filed with a court, an email sent to a client, a document signed and returned, a regulatory submission delivered. Internal scratch work includes draft generation, chronology assembly, research summaries, and any artifact staged for review but not yet released. The two classes carry different transition rules. Internal work moves freely between agents and specialists. Externally-effective work cannot transition to "sent" or "filed" without an explicit reviewer-identity-bound approval. The distinction is enforced in the data model, not assumed in the UI (FlowCounsel, "Sandboxing Is Not the Control Layer," May 2026).
The Singapore Court of Appeal decision in Quoine v B2C2 ([2020] SGCA(I) 02) is the canonical algorithmic-contracting precedent for this boundary, addressing the question of how responsibility allocates when automated systems produce contracts the deploying party would not have approved manually. The architectural implication is consistent regardless of how a specific jurisdiction resolves the underlying contract-formation doctrine: a system whose architecture allows externally-effective output without an approval transition pushes the resulting responsibility onto the firm whose name attaches to the output (FlowCounsel, "When an Algorithm Executes a Contract No Human Would Have Approved," April 2026).
Diagram · Externally-effective work boundary
Internal scratch flows freely. Externally-effective work cannot cross without an approval transition. The boundary is enforced in the data model, not in the UI.
A governed approval architecture expresses review state as a typed transition on the artifact. Every work product carries a state: draft, review-ready, approved, rejected, superseded, recalled. Transitions between states are events with reviewer identity, timestamp, evidence, and rationale. The audit obligation under ABA Op. 512 (2024), the lawyer's ability to demonstrate competent supervision after the fact, is satisfied by recording the transitions, not by recording the button clicks.
The negative case looks identical from the user side. An approve/reject button changes some UI flag. Underneath, the data model does not enforce the transition; the next step in the workflow could fire regardless of the flag's value. The protection is theatrical, not structural. The pattern is what FlowCounsel's blog catalogues as the wrapper-vs-product distinction ("How to Tell If Your Legal AI Vendor Built a Product or Prompted One," April 2026).
Diagram · Review state machine
Review state is a typed transition on the artifact. Every change is an event with reviewer identity, timestamp, and rationale.
A related but distinct discipline applies to learning. When an attorney approves a draft after edits, the system does not silently treat the approved version as a firm rule for future drafting. That would conflate approval-of-this-artifact with promotion-of-this-pattern. The two are different decisions and require different governance.
The distinction maps onto a long-standing tension in human-feedback alignment research: a single approval signal can train a model preference (Christiano et al., 2017; Ouyang et al., 2022), but a single approval is rarely sufficient evidence that the preference reflects firm doctrine rather than per-matter discretion. Constitutional AI (Bai et al., 2022) addresses a related problem by encoding principles as system constraints rather than as accumulated preferences; the architectural form here borrows that posture for firm rules.
Promotion discipline runs its own state machine: candidate → under review → approved → rejected → superseded → recalled. Promotion sources are restricted to validated outcomes, repeated approved edits across multiple matters, or explicit operator authorship. A single approval does not promote.
Approval-of-this-document is not promotion-of-this-pattern. They are governed separately.
A common pattern in legal AI is to ship an "audit log" as a list of timestamps and event names. Enough to claim the feature, not enough to satisfy ABA Op. 512's expectation of supervised-AI work product. A real audit architecture surfaces what was loaded, what was generated, who approved, what was changed, and what verifier results existed at the time. The trail is queryable, replayable, and exportable.
The discipline is to treat audit as a product surface rather than as compliance theater. A firm can ask "what happened on this matter last Tuesday?" and receive a structured answer that includes context manifests, model calls, verifier results, and approval transitions. Not a CSV of event names. The framing parallels the "audit-as-product" treatment in FlowCounsel's prior work on governed execution ("The Next Category in Legal AI Is Governed Execution," May 2026; "Agentic Theater and the Architecture That Replaces It," April 2026).
When approval is implemented as UI policy rather than as an architectural property, four distinct failure modes appear and scale with the firm's adoption of the system. Each is a structural risk, not a user-discipline problem.
A firm using legal AI on one matter can review every output manually. The architectural question matters at scale. Once the firm is running thousands of matters through the system, the policy version of "review before sending" reverts to a hope. Either the data model enforces the boundary or the work leaks past it. The boundary that fails quietly is the boundary that produces the next sanctions case.
Approval-gated execution as architecture is the property that makes legal AI deployment legible to a firm's general counsel, defensible under ABA Model Rules 1.1, 1.6, and 5.3, and durable under the kind of usage that produces business outcomes. It is also the property that platforms built on single-model chat wrappers cannot retrofit without rebuilding the underlying operating layer.
A vendor whose safety story is policy and a vendor whose safety story is architecture sound similar on a sales call. They produce different outcomes once a firm trusts them with the work.
A firm cannot defer architectural risk to a vendor that cannot describe its architecture.
References
How to cite this paper
FlowCounsel Research (2026). Approval-gated execution as an architectural property. FlowCounsel™. https://flowcounsel.com/research/approval-gated-architecture
The related papers in the FlowCounsel research program and the architectural thesis they translate into product.