Spatial IaC Terraform & Pulumi for GIS

State Backend Selection for Spatial Infrastructure as Code

Choosing where the deployment ledger lives is the decision that quietly determines whether a geospatial platform is reproducible or fragile. Within the broader discipline of Spatial IaC Architecture & Fundamentals, the state backend is the authoritative record of every provisioned PostGIS instance, tile-serving origin, raster bucket, and geocoding endpoint — and a misconfigured backend produces silent drift, corrupted resource graphs, and broken delivery pipelines rather than clean errors. The right backend depends heavily on the orchestration engine you adopt, a trade-off examined in Terraform vs Pulumi for GIS, and on how reusable your infrastructure is, a discipline governed by Module Design Patterns. This guide establishes a production-grade backend selection framework for spatial workloads, prioritizing strong consistency, distributed locking, environment isolation, and least-privilege access.

Serialized State Write Against a Locked Versioned Backend CI Runner A assumes a short-lived OIDC workload-identity role and acquires the state lock, which serializes writers so only one holds it at a time; CI Runner B is queued and waits. Inside the lock-held critical section the holder reads current state with strong read-after-write consistency, computes a plan diff against live resources, and writes a new versioned object with an atomic PUT, then releases the lock. The backend keeps every prior version, so a lifecycle policy can expire old versions while recent versions remain available for point-in-time rollback. The long-run locked fraction approximates the run arrival rate times the mean apply duration, rho equals lambda times T. CI Runner A active writer CI Runner B concurrent run Assume role (OIDC) short-lived creds State Lock one writer at a time serializes applies held fraction ρ = λT acquire wait — blocked while ρ → 1 Critical section — lock held by Runner A Read current state strong read-after-write Compute plan diff vs live graph Write versioned object atomic conditional PUT release lock Versioned state backend — encrypted at rest v(n-2) v(n-1) v(n) — new PUT v(n) rollback to prior version lifecycle expires non-current versions

Environment Parity and Configuration Drift Mitigation

State backends are the primary defence against configuration drift across the promotion cycle. The failure mode that erodes parity is shared state: when development, staging, and production read or write the same state object, an ephemeral test apply can mutate a production routing table or delete a tile-cache bucket. Every environment must therefore consume a dedicated, physically isolated state path — a separate object-storage prefix or a separate bucket per tier — rather than relying on a single bucket discriminated only by key name. Isolation at the prefix level keeps blast radius bounded; isolation at the bucket level additionally lets you apply distinct retention, replication, and access policies per environment, which matters when production carries data-residency obligations that staging does not.

Geospatial state amplifies the cost of drift because spatial resource graphs are deep and densely interdependent. A single change to a subnet CIDR can cascade through database subnet groups, security-group rules referenced by tile-server target groups, and CDN origins, and the state file is the only structure that records that web of references. If two environments diverge — staging pinned to one PostGIS extension matrix, production to another — the plan diff stops being a reliable description of change. The mitigation is to treat the backend configuration itself as a versioned, reviewed artifact: backend bucket names, key layouts, and lock-table identifiers belong in the same repository as the resources they track, never typed ad hoc into a terminal.

Workspaces are a tempting shortcut for parity, but for spatial estates they are usually the wrong tool. Terraform workspaces multiplex many environments into a single backend with a shared lock scope, so a long-running production raster import can block a staging plan, and an over-broad IAM grant exposes every workspace at once. Separate state backends per environment cost a little more configuration but give independent locking, independent versioning, and independent access boundaries — the parity properties that actually protect a production geospatial platform. Whichever layout you choose, pin it: the backend block, the provider versions, and the pyramid-depth inputs that drive storage sizing should all be reproducible from a commit hash months later.

CI/CD Validation and Operational Guardrails

Backend selection only pays off when the pipeline enforces it. Pipeline runners should authenticate to the backend using short-lived, workload-identity credentials — OIDC federation on AWS, workload identity federation on GCP, federated credentials on Azure — never static access keys checked into a secret store. State access is then scoped exclusively to the CI/CD service principal; human engineers receive read-only audit roles, so the only writer of record is the pipeline itself. This single constraint eliminates the most common source of geospatial drift: an operator hand-editing a bucket policy or resizing a database in the console.

The pull-request stage is where drift is intercepted before it reaches the state file. A terraform plan or pulumi preview runs against the isolated backend for the target environment, and a non-empty diff on an unchanged branch is treated as a drift alarm rather than a routine result. For spatial workloads, the plan gate should be paired with integrity checks specific to the resources being touched: validating that a PostGIS extension version in the plan matches the PostGIS Cluster Provisioning baseline, confirming that object-storage lifecycle rules survive the change, and confirming that tile-server origins still resolve. Pre-apply cost gates from a Cost Estimation Frameworks integration catch the case where a raster pyramid depth change silently inflates PUT volume and egress.

Locking is the guardrail that makes concurrent pipelines safe. In a multi-runner organization, two merges to main can trigger two applies against the same backend within seconds; without serialization they interleave writes and corrupt the resource graph. The probability that an incoming run is blocked rises with both how often runs start and how long each holds the lock. If runs arrive at rate $\lambda$ (runs per hour) and each holds the lock for a mean apply duration $T$ (hours), the long-run fraction of time the backend is locked — and thus the approximate probability a new run must wait — is:

$$\rho = \lambda , T$$

Long geospatial applies (a multi-gigabyte raster catalog migration can hold a lock for tens of minutes) push $T$ up, so high-traffic backends should isolate slow spatial operations into their own state and keep $\rho$ well below 1. Detailed timeout tuning, deadlock resolution, and orphaned-lock recovery are covered in Managing Terraform State Locks for Spatial Data.

Resource Architecture and Service Integration

The backend sits upstream of every other tier in the estate, so its design has to anticipate how spatial services consume it. The data tier is the most demanding consumer: modules that provision PostGIS extensions, partitioned raster catalogs, and tuned parameter groups generate large, interdependent state payloads, and the backend must guarantee strong read-after-write consistency so that a follow-on apply never reads a stale object during a migration. Object-storage backends on AWS satisfy this because S3 has provided strong read-after-write consistency for all operations since December 2020; on other clouds, confirm the consistency model before trusting it with spatial state.

Service integration also flows through the backend’s output surface. Downstream tiers read upstream state to wire themselves together: GeoServer Deployment Patterns consume database connection details produced by the data tier, Compute Node Orchestration reads VPC and subnet identifiers, and tile-server routing defined in VPC Routing for Tile Servers depends on security-group identifiers exported from the network tier. Whether these cross-tier reads use Terraform remote-state data sources or Pulumi stack references, the backend’s access policy must permit the consuming stack to read — and only read — the producing stack’s state. The buckets that hold raster and vector payloads, provisioned under Object Storage for Raster/Vector, are distinct from the buckets that hold state; conflating the two is a common and dangerous shortcut, because it grants data-plane identities visibility into the entire infrastructure ledger.

The orchestration engine determines the integration surface. Declarative Terraform serializes state to a JSON object in external storage and coordinates writes through an explicit lock; the backend is infrastructure you own and harden. Programmatic Pulumi can either use the same self-managed object-storage backend or delegate to its managed service layer, which provides built-in audit trails, encrypted secrets, and a state API at the cost of moving the ledger off your own infrastructure. For regulated geospatial estates with residency constraints, the self-managed object-storage backend usually wins because it keeps the state — including embedded connection strings and endpoint topology — inside a region you control.

Split here, and not there Rate of change, blast radius and ownership are the three properties that argue for a split, and each corresponds to one of the couplings a shared state file creates. Genuine coupling and operational overhead argue against, because components that must change together are more fragile when split across files, and every additional state file carries its own backend configuration, pipeline and review path. The heuristic that follows is to split where lifecycle differs and resist splitting where it does not, which for most platforms yields a handful of state files rather than one or dozens. Rate of change differs split Blast radius must be bounded split Different teams own them split They must change together do not split Overhead per file is real do not split

Runnable Configuration

The configurations below show hardened backends for both engines. Both pin provider versions so a plan run today reproduces months from now, both enforce encryption and versioning, and both isolate environment state by path.

Terraform (S3 backend with native locking)

terraform {
  required_version = ">= 1.10.0"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.60" # pin the provider so plans are reproducible
    }
  }

  backend "s3" {
    bucket       = "spatial-iac-state-prod"            # dedicated per-environment bucket
    key          = "networking/gis-routing/terraform.tfstate"
    region       = "us-east-1"
    encrypt      = true                                 # server-side encryption for state at rest
    use_lockfile = true                                 # native S3 conditional-write locking (Terraform >= 1.10)
    # S3 provides strong read-after-write consistency for all operations (AWS, Dec 2020),
    # so a follow-on apply never reads a stale spatial-state object mid-migration.
    # CI runners authenticate via OIDC workload-identity federation, not static keys.
  }
}

Pulumi (self-managed S3 backend with version-pinned provider)

# Provider versions are pinned in the project manifest, not on the CLI:
#   dependencies: { "@pulumi/aws": "6.40.0" }   # exact pin for reproducible previews
# The AWS profile is supplied through the standard credential chain (OIDC on CI).
export AWS_PROFILE=ci-runner

# Quote the URL so the shell does not treat "?" / "&" as globbing or backgrounding.
pulumi login "s3://spatial-iac-state-prod?region=us-east-1"

# One stack per environment isolates state, locking, and secrets.
# Stack settings live in Pulumi.prod-gis-platform.yaml alongside the code.
pulumi stack init prod-gis-platform

# Stack-level secrets (DB passwords, geocoding API keys) are encrypted at rest
# by the stack's secrets provider before they ever touch the state object.
pulumi config set --secret pgPassword "$DB_PASSWORD"

For full backend parameter semantics and provider-specific limits, consult the official HashiCorp S3 Backend reference.

Guardrails Embedded in Configuration

State locking is the first guardrail, and it belongs in the backend declaration rather than in operator habit. The use_lockfile = true option above replaces the legacy DynamoDB lock table with an S3 conditional write, removing a whole resource and its IAM surface while still serializing concurrent applies. Whatever the mechanism, the contract is the same: one writer at a time, with a timeout aligned to the slowest legitimate spatial apply so that a real raster migration is not mistaken for a hung lock.

Secret management is the second guardrail. State files routinely embed sensitive material — database connection strings, geocoding API keys, internal routing configuration — so encryption at rest is mandatory and non-sensitive secrets must never be written in plaintext. Terraform’s encrypt = true covers the object at rest; Pulumi’s secrets provider encrypts individual config values before serialization. Neither substitutes for keeping the human blast radius small, so audit-only roles for engineers and a single writer principal for CI remain the backbone of the design.

Network isolation is the third guardrail. State buckets should be reachable from the pipeline’s network path without traversing the public internet where the cloud permits it — a gateway or interface VPC endpoint scoped by an endpoint policy that names only the state buckets. This keeps state traffic on the provider backbone and lets the Security Group Hardening and IAM Role Mapping for GIS patterns from the network layer apply to the backend as cleanly as they apply to the tile servers.

Storage-class and lifecycle settings round out the guardrails. Enable object versioning on the state bucket so every apply leaves a recoverable predecessor, giving point-in-time rollback without external backup scripts. Because each version is a full copy of the state object, versioned state grows linearly with apply frequency; a lifecycle rule that expires non-current versions after a bounded retention window keeps that growth — and its cost — predictable. Cross-region replication should stay disabled for state buckets unless a documented disaster-recovery requirement justifies the added latency and spend.

Sizing note: state-bucket parameters are object-storage settings — bucket names, prefixes, versioning, lifecycle days — not database memory parameters, so no RDS parameter-group integer units apply here. When the same pipeline provisions the PostGIS data tier, set its memory parameters (for example shared_buffers) in AWS 8 kB block units, never as percentage strings; see PostGIS Cluster Provisioning.

Troubleshooting and Failure Modes

State lock held by a dead runner. A CI runner that is killed mid-apply (spot reclamation, OOM during a large raster diff) can leave a lock in place, and every subsequent run fails with a lock-acquisition error naming a stale LockID. Confirm the holding run is genuinely dead before force-unlocking; releasing a lock held by a live apply is what actually corrupts state. The recovery procedure and the automated lock-age alerting that catches this are detailed in Managing Terraform State Locks for Spatial Data.

S3 prefix or key collision across environments. When two environments share a bucket and differ only by key, a copy-pasted backend block can point staging at the production key, so a staging apply plans to destroy production tile origins. The symptom is a plan that proposes deleting resources the current environment never created. The fix is structural: separate buckets per tier, or at minimum a key layout that encodes the environment as the leading prefix, validated by a pre-apply check that asserts the key matches the target workspace.

Stale read after a backend migration. Migrating a backend (changing bucket, region, or moving from DynamoDB locks to use_lockfile) without terraform init -migrate-state can leave the new backend empty while the old one still holds the truth; the next plan shows every spatial resource as “to be created.” Always migrate state explicitly, verify the object exists and is non-empty in the new location, and only then decommission the old backend.

VPC endpoint policy gap. If the state bucket is reached through an interface or gateway endpoint whose policy omits the bucket ARN, applies hang and then fail with access-denied or timeout errors even though the IAM role is correct — the request never leaves the VPC. Confirm the endpoint policy names both the bucket and its objects, and that the route table associates the endpoint with the runner’s subnet.

Over-broad state read grant. Granting a downstream stack s3:GetObject on the entire state bucket so it can read one upstream output exposes every environment’s ledger, including embedded secrets. Scope cross-stack remote-state reads to the specific producing key, and keep data-plane identities — the ones that read raster and vector buckets — entirely separate from state-bucket identities.

What belongs in one state file

The backend decision is usually framed as where state lives, and the more consequential question is how much of the estate a single state file describes. A state file is a blast-radius boundary: everything in it can be planned together, destroyed together, and locked together, which is convenient until one of those is not what you wanted.

Three properties argue for splitting. Rate of change — a serving tier redeployed twice a day and a database changed twice a year do not belong in one file, because the frequent changes repeatedly re-plan the infrequent ones. Blast radius — if a careless plan should not be able to propose destroying two things at once, those two things need separate files. Ownership — if two teams change two components, sharing a state file means sharing a lock, and one team’s long apply is the other team’s outage in delivery terms.

Two properties argue against splitting. Genuine coupling — components that must change together produce coordinated applies across files, which is more fragile than one plan. And operational overhead — every state file needs its own backend configuration, its own pipeline and its own review path, and an estate with forty of them for a platform with four teams has traded one problem for another.

The workable heuristic is to split where lifecycle differs and to resist splitting where it does not. In practice that yields a handful of state files per platform rather than one or dozens, aligned with the tiers described in Refactoring a Monolithic GIS Stack into Composable Modules. Cross-file references should be explicit and few, because each one is a coupling that a future split will have to honour.

What a shared state file couples together Four consequences follow from putting two components in one state file. They plan together, so a frequent change re-plans an infrequent one every time. They lock together, so one team's long apply blocks the other's routine change. They can be destroyed together, which is the property that matters most when one of them holds authoritative geometry. And they are reviewed together, so a small change arrives in a plan large enough that nobody reads it closely. Each of those is acceptable for tightly coupled components and unacceptable across a lifecycle boundary. They plan together a frequent change re-plans an infrequent one They lock together one long apply blocks the other team They can be destroyed together the property that matters when one holds geometry They are reviewed together a small change in a plan too large to read