Building a Dynamic Agent Loop with Temporal: Durable Orchestration for Tools, Workflows, and Workspaces

Q: What happens when a tool call needs the user to reconnect an account?

executeMcpTool first tries an OAuth refresh through the auth provider. If that isn't enough, it returns a structured "auth required" payload as the tool result instead of failing the run, so the next iteration sees it like any other result. That's the failure from the opening, where an auth refresh broke a PM tool call mid-run, turned into something the loop can reason about.

How Fabric's orchestrator became a Temporal agent loop with on-demand tool discovery, MCP, workspace retrieval and human approval steps.

March 1, 2026 · 4 min read · by Preetham Reddy

The breaking point wasn't a model hallucination.

It was a perfectly reasonable workflow:

read project context from a workspace
draft a rollout plan
create tickets in the PM tool
post a summary to Slack

The agent did the first two steps well, but then the PM tool call failed on auth refresh and the user sent a follow-up while it was retrying. By the time we recovered, the original plan was stale, the user intent had changed, and the system had no clean way to adapt.

Nothing was "wrong" with the LLM. Our orchestration model was wrong.

That was the shift. We stopped treating execution like a static script and started treating it like a durable control loop.

The Reality of Production Agent Work

Single-agent demos look linear. Production systems aren't.

In practice, every run has moving parts:

tool availability changes mid-run
OAuth states change mid-run
user intent changes mid-run
the best next action is often unknown until you inspect previous results

So "plan once, execute once" becomes brittle fast.

The design we moved to is simple conceptually:

Observe state.
Decide next action.
Execute safely.
Incorporate result.
Repeat until done.

A user request initializes context, then the loop decides, executes, observes and repeats until the work is done and the final response returns.

That loop is where Temporal changed everything for us.

Why Temporal Changed the Architecture

We run the orchestrator asorchestratorExecutionWorkflow onfabric-orchestrator.

Temporal gives us the properties this loop needs:

durability: workflow state survives restarts and deploys
control: live signals forapproval,followUp,cancel
observability: queryable workflow state (progress,status,plan,pendingApproval)
determinism: predictable behavior under retries and replay

The UI starts a stream through the orchestrator API, then sends approvals and follow-ups as signals and reads progress by querying the workflow.

That control model sounds subtle, but it changes behavior radically. The workflow is no longer a black box behind an API poller, because it's a live, addressable process.

The Workflow We Actually Run

For current Fabric orchestrator runs, iterative execution is the default for all modes exceptsave_reuse (which keeps upfront planning for trajectory replay).

High-level shape:

Initialization
Iterative execution loop
Completion child workflow (fire-and-forget)

After initialization the orchestrator iterates through agent responses and gated tool calls, and the final response returns to the user while a completion child workflow runs separately.

The important detail is that every loop turn carries updated context, updated risk state, and updated user intent.

A Concrete Loop Walkthrough

Take this request:

"Create a launch checklist from workspace docs, open tasks in our PM system, then post update in Slack."

What happens in a real run:

Init loads orchestrator memory and preloads metadata for enabled MCP configs, integrations, and agents.
First iteration may callsearch_tools to discover exact capabilities needed now.
Loop executes workspace RAG query (workspace_rag_query) for grounding.
Loop executes PM action tool or integration operation.
If risky action is detected, workflow pauses on approval signal.
User sends follow-up: "Only for P0 and P1 items." Signal updates live state.
Next iteration adapts immediately and completes with updated constraints.

No restart. No context loss. No side channel state machine outside Temporal.

Dynamic Capability Discovery Instead of Tool Bloat

A major issue in earlier versions was large static tool payloads in context.

We now start with a minimal capability surface and expand only when needed via a meta-tool pattern (search_tools).

Under the hood this combines:

always-available capability shortcuts
explicit server mention detection
keyword matching
semantic capability search
agent discovery for delegation

A user need triggers the search tools meta tool, which merges keyword, vector and agent capability results and injects the discovered tools for the next iteration.

This is the core loop behavior: discover first, execute second.

MCP Execution Path: Cached, Typed, and OAuth-Aware

When the loop selects an MCP tool, execution goes throughexecuteMcpTool.

Key behaviors:

cached MCP client retrieval per config
cached tool list retrieval
argument deserialization and schema coercion
OAuth refresh support through auth provider
structured "auth required" output when reconnect is needed

The iterative loop sends a tool call to the MCP executor, which gets a cached client, connects to the server and returns either a result or an auth required payload.

One implementation rule matters a lot here. Credentials are resolved in activities, not embedded in workflow input, and that keeps secrets out of Temporal history.

Integrations and Workspaces Are Loop-Native

The dynamic loop isn't MCP-only.

Workflow integrations

We route directly through integration handlers for providers like Slack, GitHub, Linear, Resend, and Microsoft Graph using tenant-scoped stored credentials.

A step request resolves its provider and operation, fetches tenant scoped credentials and executes the integration operation to produce structured output.

Workspace RAG

Workspace retrieval is on-demand viaworkspace_rag_query, executed during the loop when needed.

An iteration sends a workspace query through the RAG handler, which retrieves relevant chunks from the vector store and returns them as a tool result.

This keeps startup lean and retrieval relevant.

Human-in-the-Loop as a First-Class Runtime Path

Risk checks happen before execution of each tool call in iterative mode.

High-risk operations create approval tasks and pause workflow progress until anapproval signal arrives.

A proposed tool call is risk assessed and either runs directly or waits for an approval signal, and a rejection is recorded before the loop continues.

This model is why we can keep execution adaptive without giving up safety.

Budget Guardrails for Long-Running Loops

Iterative loops can drift in cost if unmanaged.

We enforce:

per-modemaxIterations
per-modemaxTotalTokens
pruning of older tool results
large output summarization
synthesis response when budget limits are hit

Each iteration checks its budget and either executes the next action or runs a final synthesis iteration with no tools.

This keeps the loop bounded while still producing useful completion output.

Completion Is Decoupled from User Latency

After we have user-facing output, we launch a completion child workflow withParentClosePolicy.ABANDON.

That child handles:

reflection on final output
execution pattern recording
trajectory embedding in semantic memory
episodic memory creation
pattern extraction for future runs
optional trajectory save insave_reuse mode

The user gets a response quickly. The system still gets smarter afterward.

The Multi-Tenant Constraint That Shapes Everything

Fabric is multi-tenant by default, so every loop stage carries explicit tenant context:

workflow input includesuserId andorganizationId
MCP lookup and execution use tenant-scoped filters
integration credential resolution is tenant-scoped
workspace retrieval is tenant-scoped
memory writes are tenant-scoped

In dynamic loops, missing tenant propagation is one of the easiest ways to introduce subtle data isolation bugs. We treat propagation as a hard invariant.

The Shift in Engineering Mindset

The biggest change was not "better prompts."

It was this architectural shift:

from static plans to iterative control loops
from client-managed state to workflow-managed state
from tool catalogs to on-demand capability discovery
from binary success/failure to durable, controllable progression

That is the path from an impressive agent demo to a production orchestration system.

And for us, Temporal is the layer that made that path practical.

Frequently asked questions

What happens to a run that's waiting for approval when we deploy?

It keeps waiting. The pause is workflow state, and Temporal keeps that state through restarts and deploys, so the approval signal still reaches the same run when it arrives. Nothing secret has to be replayed to pick it up again either, because the loop resolves credentials in activities and they never enter workflow history.

What happens when a tool call needs the user to reconnect an account?

executeMcpTool first tries an OAuth refresh through the auth provider. If that isn't enough, it returns a structured "auth required" payload as the tool result instead of failing the run, so the next iteration sees it like any other result. That's the failure from the opening, where an auth refresh broke a PM tool call mid-run, turned into something the loop can reason about.

Why does `save_reuse` mode still plan up front?

That mode is for replaying a saved trajectory, and replay needs the plan fixed before execution starts. Every other mode runs the iterative loop by default. The saving happens afterwards: the completion child workflow stores the trajectory only in save_reuse mode.

What does the user get when a run hits its iteration or token budget?

They still get an answer. The loop runs one last synthesis iteration with no tools and returns a bounded response from what it has gathered so far. Because that final turn can't call tools, hitting the budget never triggers a new action on the way out.

What happens when a user rejects an approval?

The loop records the rejection and carries on rather than stopping the run. The next iteration starts with that rejection in its context, alongside the updated risk state and any follow-up the user sent, so it can choose a different next action or finish without the risky step.

Written by

Preetham Reddy

Founder, TechFabric