Spatial IaC Terraform & Pulumi for GIS

Testing and Validation for Spatial IaC

Infrastructure code for a geospatial platform fails in ways that generic infrastructure testing does not look for. A plan can be syntactically valid, policy-compliant and cost-approved, and still ship a database without the PostGIS extension, a bucket whose lifecycle rule archives the tile pyramid a renderer reads hourly, or a parameter group whose memory settings are expressed in the wrong unit. None of those are provider errors — every one applies cleanly and then produces a spatial failure at runtime. This topic sits within CI/CD Automation and Governance and defines the layered test strategy that catches those failures before they reach an environment, complementing the enforcement work in Policy as Code for Spatial Resources.

The layering matters more than any individual tool. Static analysis is fast and finds shape errors; contract tests on the plan find semantic errors; integration tests against real provisioned resources find behavioural errors; and post-deploy probes find the errors that only exist when traffic arrives. Each layer is progressively slower and more expensive, so the discipline is to push every check as far left as it will go and to reserve the expensive layers for the questions the cheap ones genuinely cannot answer.

Environment parity and configuration drift mitigation

A test suite is itself configuration, and it drifts exactly like the infrastructure it validates. The characteristic failure is a suite that was written against the development environment’s assumptions and then quietly stops being meaningful for production: it asserts a single-node database because that is what development runs, so the multi-availability-zone production topology is never exercised by anything until it is exercised by an incident.

The fix is to parameterise the test suite from the same module inputs that parameterise the infrastructure, so that a change to production’s shape changes what the tests assert. If the module takes a replica count, the integration test asserts that replica count rather than a hardcoded number. If the module takes a list of published schemas, the tile probe iterates that list rather than a copy of it. A test that restates a value is a test that will disagree with reality the first time the value changes.

Test-data parity is the second axis, and for spatial workloads it is unusually demanding. A PostGIS integration test against an empty table proves the extension loaded and proves nothing about whether the index is used, because the planner will sequentially scan a small table regardless of what indexes exist. Seed fixtures must therefore be large enough and skewed enough to produce a realistic plan — a few thousand features with a realistic spatial distribution, not ten rows in a square. Fixtures also need a stable coordinate reference system and known bounding boxes, because assertions such as “this tile contains features” are only reproducible if the data’s extent is fixed.

Four layers of validation for spatial infrastructure code Four stacked layers ordered from fastest to slowest. Static analysis runs on source in seconds and catches shape errors such as a missing version pin or a public bucket. Contract assertions run on the rendered plan in under a minute and catch semantic errors such as a missing PostGIS extension or a lifecycle rule that would archive live tiles. Integration tests provision real resources in minutes and catch behavioural errors such as an index the planner refuses to use. Post-deploy probes run against live endpoints and catch errors that only appear under traffic, such as an empty tile returned with a success status. 1 · Static analysis seconds, on source — missing version pins, public buckets, unpinned images shape errors 2 · Contract assertions on the plan under a minute — PostGIS extension present, lifecycle rule spares live tiles semantic errors 3 · Integration against real resources minutes — the planner actually uses the GiST index on realistic data behavioural errors 4 · Post-deploy probes against live endpoints — a tile that returns 200 with no features traffic-only errors

Drift between the suite and the estate is best surfaced by the same nightly mechanism that surfaces infrastructure drift. A scheduled run of the full suite against production’s actual configuration — not against a plan, against what is deployed — turns “our tests passed at merge time” into “our tests pass now”, which are different claims and only the second one is useful during an incident.

CI/CD validation and operational guardrails

The pull-request pipeline should be ordered so that the cheapest signal fails first. Formatting and validation, then static analysis, then a plan, then contract assertions on that plan, then cost and policy gates, and only then the integration suite against ephemeral resources. Ordering this way is not merely about wall-clock time; it is about the quality of the failure message. A missing version pin reported by a static analyser in eight seconds is a clear, local fix. The same missing pin discovered when an integration test provisions an unexpected instance class forty minutes later is a debugging session.

Contract assertions on the plan are the layer most teams skip and the one with the best return for spatial estates, because the plan is a machine-readable description of the intended end state and almost every spatial-specific mistake is visible in it. Exporting the plan as JSON and asserting over it catches a database created without the extension list, a parameter group whose shared_buffers is expressed as a percentage string rather than in 8 kB blocks, a bucket lifecycle transition that would move tiles a renderer reads into a retrieval-latency storage class, and a security group opening a data-plane port to the world. None of these require provisioning anything.

Integration tests need an isolation strategy or they will interfere with each other and with the estate. Each run should provision into its own namespace — a distinct workspace, a distinct state key, a distinct resource-name prefix — and each should be responsible for its own teardown even on failure. The most common operational failure in this layer is not a flaky test but an orphaned resource: a run that failed before its cleanup step and left a database cluster running, discovered a month later on a bill. A scheduled sweep that destroys anything carrying the test prefix beyond a maximum age is the guardrail that makes the layer affordable.

Ordering the validation pipeline by cost of failure Stages run cheapest first: format and validate, static analysis, plan, contract assertions over the plan JSON, then policy and cost gates, then the integration suite. The integration suite provisions into an isolated namespace with its own state key and resource prefix and always runs teardown, including on failure. A separate scheduled sweep destroys any resource carrying the test prefix that has outlived its maximum age, which is the guardrail that keeps orphaned test infrastructure from accumulating on the bill. fmt + validate seconds static analysis checkov · tfsec plan export to JSON contract assertions spatial semantics policy + cost gates Integration suite own workspace, state key, prefix teardown runs even on failure Scheduled sweep destroys anything with the prefix older than the maximum age Cheapest signal first — the same defect costs eight seconds here and forty minutes three stages later. The characteristic failure of the integration layer is not flakiness but orphaned resources on next month's bill.

What to assert in a spatial plan

Contract assertions repay a concrete list, because the useful ones are not obvious and the obvious ones are already covered by generic scanners. Five families cover most of what goes wrong in a geospatial estate, and all five are visible in the plan JSON before anything is provisioned.

Extension and version contracts. A managed PostgreSQL instance is not a spatial database until PostGIS is installed, and the plan should show the extension being created at a pinned version rather than left to whatever the engine’s default happens to be. The same applies to the companion extensions a platform actually depends on — postgis_raster, postgis_topology, pg_stat_statements — each of which is routinely assumed present and occasionally absent. Assert the list, not merely the base extension.

Memory parameters in the provider’s units. Managed-database parameter groups express memory in 8 kB blocks, and a value written as a percentage string or as a byte count is accepted by the API and then means something entirely different from what the author intended. This is the single most common spatial-tuning defect, because raster and geometry workloads are memory-sensitive enough that the mistake produces a real regression rather than an invisible one. Assert that memory parameters resolve to integers in the expected range.

Lifecycle rules that spare live objects. A transition rule that moves objects older than thirty days into an archival storage class is correct for source imagery and catastrophic for a tile pyramid that is read continuously and rewritten rarely — the tiles are old by age and hot by access. Assert that lifecycle transitions are scoped by prefix, and that the prefixes a renderer reads are excluded. The reasoning behind those prefixes is developed in Configuring S3 Lifecycle Rules for GIS Tiles.

Projection and coordinate-system inputs. Where a module takes an SRID, a tile extent or a bounding box as input, assert the value rather than trusting the default. A module defaulting to 4326 that is used for a Web Mercator tile pipeline will produce correct-looking infrastructure and geometrically wrong output, and no scanner will ever flag it because nothing about it is insecure.

Destructive changes to stateful spatial resources. The plan is the only place where a replacement is visible before it happens. A change that forces replacement of a database cluster, a bucket holding an authoritative archive, or a key still referenced by live resources should fail the build unless it carries an explicit approval marker. This is the assertion that turns the worst class of incident — an unrecoverable delete applied at 2am by an automated pipeline — into a red build.

Five assertion families for a spatial plan Five checks worth running against the plan JSON before any resource is created. Extension and version contracts confirm PostGIS and its companion extensions are created at pinned versions. Memory parameter checks confirm database memory settings resolve to integers in the provider's units rather than percentage strings. Lifecycle scope checks confirm archival transitions exclude the prefixes a renderer reads continuously. Projection checks confirm the spatial reference identifier and tile extent are set explicitly rather than defaulted. Replacement checks fail the build when a stateful spatial resource would be destroyed and recreated without an explicit approval marker. Extensions postgis pinned raster, topology present, not assumed Memory units 8 kB blocks integer, in range never a percentage Lifecycle scope scoped by prefix live tiles excluded old by age, hot by use Projection SRID explicit tile extent set no silent default Replacement stateful resource destroy and recreate fails without approval All five are visible in the plan JSON, so all five cost a second and provision nothing. None are security findings, which is why a generic scanner reports every one of these estates as clean. The replacement check is the one that turns an unrecoverable overnight delete into a red build.

Resource architecture and service integration

The suite needs infrastructure of its own, and treating it as a first-class part of the platform rather than as scaffolding is what keeps it maintained. That infrastructure is a dedicated account or project for ephemeral test resources, an isolated state backend so test runs cannot lock or corrupt real state, a seeded fixture dataset stored as a versioned artifact, and a runner identity whose permissions are broad within the test account and absent everywhere else.

The isolation of state is the piece most often compromised for convenience, and it is the one with the worst failure mode. If integration runs share a backend with real environments, a concurrent run can contend on a lock with a production apply, and a mis-scoped destroy can target a real workspace. Use a separate backend entirely, following the separation principles in State Backend Selection, and give the runner no credentials that reach production at all — the strongest guarantee that a test cannot damage production is that it cannot authenticate to it.

The runner identity is worth designing rather than inheriting. Integration tests legitimately need broad permissions — they create databases, buckets, keys and networks — and that breadth is only safe because it is confined to an account that contains nothing of value. The moment a convenience grant lets the test runner reach a shared registry, a shared DNS zone or a shared key in the production account, the confinement is gone and the suite becomes the widest-privileged automation in the estate. Where a shared resource genuinely must be reachable, prefer a narrowly scoped read grant with an explicit condition over adding the production account to the runner’s trust policy, and document the exception where the next reviewer will see it.

Fixture data integrates with the same object storage conventions as production raster and vector assets. Versioning the fixture matters because assertions encode expectations about it: a test asserting that a bounding box returns 412 features is coupled to the fixture version, and an unversioned fixture that someone regenerates turns a real regression and a fixture change into the same red build. Publish fixtures as immutable, versioned objects and pin the version in the suite.

Runnable configuration

The following Terratest case provisions a PostGIS module into an isolated namespace, then asserts the properties that matter spatially: the extension is present at the pinned version, the spatial index exists, and the query planner actually chooses it on realistic data.

package test

import (
    "database/sql"
    "fmt"
    "testing"

    _ "github.com/lib/pq"
    "github.com/gruntwork-io/terratest/modules/random"
    "github.com/gruntwork-io/terratest/modules/terraform"
    "github.com/stretchr/testify/assert"
    "github.com/stretchr/testify/require"
)

func TestPostGISModule(t *testing.T) {
    t.Parallel()

    // Every run gets its own prefix so concurrent runs cannot collide, and so
    // the scheduled sweep can identify orphans by name alone.
    prefix := fmt.Sprintf("tftest-%s", random.UniqueId())

    opts := &terraform.Options{
        TerraformDir: "../modules/postgis-cluster",
        Vars: map[string]interface{}{
            "name_prefix":        prefix,
            "postgis_version":    "3.4",
            "instance_class":     "db.t4g.medium",
            "fixture_object_uri": "s3://gis-test-fixtures/parcels/v7/parcels.sql.gz",
        },
        // A separate backend: a test run must not be able to lock real state.
        BackendConfig: map[string]interface{}{
            "bucket": "gis-terraform-test-state",
            "key":    fmt.Sprintf("integration/%s.tfstate", prefix),
        },
    }

    // Teardown is deferred first, so it runs even when an assertion panics.
    defer terraform.Destroy(t, opts)
    terraform.InitAndApply(t, opts)

    dsn := terraform.Output(t, opts, "connection_string")
    db, err := sql.Open("postgres", dsn+"?sslmode=verify-full")
    require.NoError(t, err)
    defer db.Close()

    // 1. The extension exists at the pinned version. A cluster without PostGIS
    //    applies cleanly and fails at the first spatial query.
    var version string
    require.NoError(t, db.QueryRow(
        "SELECT extversion FROM pg_extension WHERE extname = 'postgis'").Scan(&version))
    assert.Equal(t, "3.4", version)

    // 2. The GiST index exists on the geometry column.
    var indexCount int
    require.NoError(t, db.QueryRow(`
        SELECT count(*) FROM pg_indexes
        WHERE tablename = 'parcels' AND indexdef LIKE '%USING gist%'`).Scan(&indexCount))
    assert.Equal(t, 1, indexCount)

    // 3. The planner actually USES it. An index that exists but is never chosen
    //    is the failure an empty-table test can never detect, which is why the
    //    fixture carries a realistic row count and spatial distribution.
    var plan string
    require.NoError(t, db.QueryRow(`
        EXPLAIN (FORMAT TEXT)
        SELECT count(*) FROM parcels
        WHERE geom && ST_MakeEnvelope(-122.5, 37.7, -122.3, 37.9, 4326)`).Scan(&plan))
    assert.Contains(t, plan, "Index Scan")
}

Guardrails embedded in the configuration

  • Teardown is deferred before apply. Registering the destroy first means it runs even when a later assertion panics, which is the difference between a failed test and a failed test plus a running database cluster.
  • A unique prefix on every run. Concurrent runs cannot collide, and orphans are identifiable by name, which is what makes an automated sweep possible without a resource inventory.
  • A separate state backend with its own credentials. The suite cannot contend on a production lock because it cannot reach the production backend at all.
  • The fixture is pinned by version. Assertions that encode expectations about the data are coupled to a specific fixture, and pinning keeps a fixture regeneration from masquerading as a regression.
  • The connection asserts verify-full. Testing over an unverified connection would validate the cluster while leaving the property most likely to be misconfigured — server certificate verification — unexercised.

A last note on ownership. A validation suite with no named owner decays faster than any other part of an estate, because a red build that nobody is responsible for is resolved by whoever is most inconvenienced, and the cheapest resolution is always to skip the test. Name an owning team for the suite in the same place the modules name theirs, and treat a disabled test as a change that requires review rather than as housekeeping.

Troubleshooting and failure modes

1. A green suite that misses a real regression. Almost always a fixture too small for the planner to behave realistically. An index-usage assertion against ten rows passes or fails on planner heuristics rather than on the index, so it tests nothing. Grow the fixture until the plan is stable and representative.

2. Orphaned test resources. A run interrupted between apply and destroy leaves infrastructure behind. Deferred teardown covers panics but not a cancelled job or a runner that dies, which is why the scheduled prefix sweep is a required companion rather than an optional extra.

3. Lock contention with a real environment. Symptom is an integration run that blocks for the length of a production apply, or worse, a production apply that blocks behind a test. The cause is a shared backend, and the fix is separation, not longer timeouts.

4. Assertions that restate module inputs. A test asserting replica_count == 2 while the module takes a variable will pass forever and stop meaning anything the moment production moves to three. Read the value from the module output and assert the relationship, not the literal.

5. Slow suites that get disabled. A forty-minute integration stage on every pull request will eventually be marked optional, at which point it protects nothing. Keep the per-request path to static analysis and plan assertions, and run the full integration matrix on merge to the default branch and on a schedule.

6. Flaky probes against eventually-consistent resources. Certificate validation, DNS propagation and replica promotion are not instantaneous, and a probe that runs immediately after apply will intermittently fail. Retry with a bounded backoff against a specific expected condition rather than sleeping a fixed interval and hoping.