pg_durable: Durable Execution Inside PostgreSQL

You already use PostgreSQL to store the state your jobs operate on. Durable execution asks one question: why is the state of the job itself stored somewhere else?

Source

pg_durable — Durable Execution inside PostgreSQL

By: Microsoft
Published: Jun 19 2026
Type: Engineering article

A PostgreSQL extension (2.8k GitHub stars, currently in preview) that brings durable workflow execution inside the database process itself. Microsoft ships it as a first-class feature inside Azure HorizonDB.

Durable Execution as a PostgreSQL Extension

The standard architecture for background jobs against a relational database involves at least four moving parts: a scheduler or queue, worker processes, status columns in the target table, and retry logic. None of these are expensive individually. Together they create a coherent failure mode — after a crash at step three of a five-step pipeline, you have to reconstruct which rows were partially processed, which API calls already landed, and which rollbacks are safe. The code that handles this tends to accumulate until it is larger than the job it wraps.

pg_durable proposes a different structure. A workflow is defined as a graph of SQL steps linked by composable operators: ~> chains steps sequentially, |=> names an intermediate result that later steps can reference. Calling df.start() returns an instance ID and hands execution to a background worker that lives inside the PostgreSQL process. The worker executes each step and persists a checkpoint to PostgreSQL before advancing. If the server crashes, execution resumes from the last durable checkpoint on restart. The job’s state and the data the job operates on live in the same database, under the same backup policy, queryable with the same credentials.

The architecture has two internal layers below the SQL DSL. duroxide is a durable task framework that handles orchestration: deterministic replay, checkpointing, sub-orchestrations, and timers. duroxide-pg is a PostgreSQL-backed state provider for duroxide that persists runtime state — instances, history, and work queues — in a dedicated duroxide.* schema. The SQL DSL and background worker sit on top of this pair as a third layer exposed to users through the df.* schema. Users see df.instances and df.nodes; the runtime internals stay in duroxide.*.

Multi-user access is handled through row-level security and an explicit grant function. df.grant_usage('role') gives a role the right to submit and query its own instances. The background worker role must be a superuser to bypass RLS when managing all users’ instances, but application roles get column-level restricted access — they can update status and updated_at on their own instances via df.cancel() but cannot touch the submitted_by identity column. This matters in multi-tenant deployments where different services share the same Postgres cluster.

The HTTP egress model is notable. Steps can call external APIs with df.http(), but the released binary is compiled with http-allow-azure-domains — an HTTP egress policy that restricts outbound calls to Azure domains by default. This is a deliberate production constraint, not a limitation of the core model. The source build accepts EXTRA_FEATURES on the build command to configure a different policy.

The cases where pg_durable changes the architecture meaningfully are those where today’s solution is a pg_cron job plus a status table plus retry counters plus a polling worker. Vector embedding pipelines, ingest pipelines with deduplication and fan-out, scheduled maintenance workflows that pause for human approval — these are workloads where external orchestrators like Temporal or Step Functions add infrastructure that mostly just wraps calls back into Postgres anyway.

The cases where it does not help: a job that is a single INSERT ... SELECT, any workflow that needs sub-millisecond synchronous latency, environments where extensions cannot be installed, or pipelines that span heterogeneous systems where the majority of state does not live in Postgres.

What I want to remember

Remember: Durable execution complexity scales with how far the job state lives from the data it operates on — if both live in Postgres, the gap is zero.