Spatial IaC Terraform & Pulumi for GIS

VPC Routing for Tile Servers: Production-Grade IaC Patterns

A tile server has a split personality that ordinary web workloads do not: its public face must accept thousands of high-frequency map requests from the open internet, while its rendering tier — Mapnik, GeoServer, or a custom vector pipeline — must reach PostGIS and object storage over strictly private paths that never touch a public IP. Getting the route tables wrong does not fail loudly; it manifests as intermittent dropped tiles, mysterious cross-AZ data-transfer charges, and renderers that can reach the internet when they should be sealed off. Within the broader Network Security & Access Control framework, VPC routing is the layer that encodes this split as version-controlled, reviewable infrastructure rather than console-clicked route entries that drift. It has to stay aligned with the credential boundaries set by IAM Role Mapping for Geospatial Workloads and the ingress rules from Security Group Hardening, so that network reachability and identity permissions enforce the same least-privilege posture from two directions.

This page covers the routing decisions that keep tile delivery deterministic: subnet segmentation and route-table hygiene, the CI/CD gates that catch a leaking private route before it ships, how routing integrates with the upstream spatial services it serves, a runnable Terraform module, the guardrails worth embedding in that module, and the failure modes that recur in production map platforms.

Environment parity and configuration drift mitigation

Routing parity is non-negotiable because a tile platform that behaves differently in staging and production is untestable. The structure of the network — which subnets are public, which route tables carry a default route, which endpoints attach where — must be byte-for-byte identical across development, staging, and production, differing only in CIDR allocations, account identifiers, and NAT-gateway capacity. The discipline that makes this hold is the same one used for PostGIS Cluster Provisioning: abstract route tables, subnet associations, and endpoint attachments into reusable modules that accept environment-specific variables, and never hand-edit a route in the console.

The most common drift source is the route table itself. Public subnets should carry exactly one default route — 0.0.0.0/0 to an Internet Gateway (IGW) — and nothing more. Private subnets must never inherit that route; their default traffic egresses through a highly available NAT gateway, and traffic to AWS services such as S3, Secrets Manager, and CloudWatch should bypass the internet entirely through VPC endpoints. When a renderer in a private subnet can suddenly reach 0.0.0.0/0 via an IGW, the segmentation contract is already broken even if every tile still renders. Codifying the route tables means this regression appears as a red line in a terraform plan diff instead of a quiet change someone made under incident pressure.

A second, subtler drift source is dynamic route propagation. Static routes for VPC peering or Transit Gateway attachments should be declared explicitly rather than left to BGP propagation, which can introduce transient asymmetric paths — return traffic taking a different route than the request — that produce TCP state mismatches and dropped tile fetches under load. Pinning routes statically keeps the data plane reproducible, which is the precondition for trusting that staging predicts production.

CI/CD validation and operational guardrails

Routing defects are cheapest to catch before apply, so the pipeline enforces policy-as-code on the plan rather than auditing live infrastructure after the fact. Static analysis with Checkov, Conftest, or tfsec should fail the build on a fixed set of routing invariants:

  • No private route table contains a direct IGW route.
  • Every 0.0.0.0/0 route resolves to a NAT gateway or a VPC endpoint, never to an IGW from a private subnet.
  • Peering and Transit Gateway routes do not overlap with the local VPC CIDR, which would create blackhole or loop conditions.
  • Every route table carries the mandatory ownership, environment, and compliance tags used for cost allocation and audit.

Beyond static checks, gate every change on a full plan diff: terraform plan -detailed-exitcode (or pulumi preview --diff) in the PR pipeline surfaces routing drift before merge, and exit code 2 — changes present — can block auto-merge until a human reviews the route delta. Because route-table mutations briefly interrupt connectivity, sequence them with create_before_destroy lifecycle blocks on route-table associations so a new association exists before the old one is torn down, and schedule larger topology changes inside a maintenance window to avoid dropping active tile-generation jobs. State isolation underpins all of this: each environment uses a dedicated remote backend with locking, exactly as covered in State Backend Selection, so two concurrent applies cannot race on the same route table.

Resource architecture and service integration

Routing is never an end in itself — it exists to connect the tile-serving tiers to the spatial services behind them, and each integration point shapes the route tables. At the edge, an application load balancer or reverse proxy in a public subnet terminates client traffic; it is the only component reachable from the internet, and its ingress is constrained by Security Group Hardening so that the route to the IGW is matched by an equally tight allow-list. Behind it, rendering nodes in private subnets pull geometries from PostGIS and read pre-rendered tiles or raster sources from object storage.

The object-storage path is where routing and cost intersect most sharply. Routing renderer-to-S3 traffic through a Gateway VPC endpoint keeps tile-cache hydration on the AWS backbone, avoids NAT data-processing charges, and lets an endpoint policy scope which buckets the data plane can reach — the network-layer complement to the prefix scoping described in Object Storage for Raster & Vector. For cross-region or cross-account topologies — a distributed renderer fleet sharing a central PostGIS catalog — the routing layer is implemented as peering or Transit Gateway attachments, and Terraform VPC Peering for Distributed GeoServer walks through the declarative route-table associations and static route entries that guarantee private-IP resolution across account boundaries. Prefer Transit Gateway over a mesh of point-to-point peerings once more than two VPCs are involved: it centralizes route propagation and collapses the N-squared peering problem into a single attachment per VPC.

Cross-AZ traffic is the routing decision with the largest cost signature. Every cache miss that pulls a geometry from a PostGIS instance in another availability zone incurs inter-AZ transfer, so the volume scales with the size of the tile pyramid being served. For a pyramid of maximum zoom $Z$, the number of distinct tiles a renderer may need to source is

$$N_{\text{tiles}} = \sum_{z=0}^{Z} 4^{z} = \frac{4^{,Z+1}-1}{3}$$

which grows by a factor of four per zoom level — a reminder that routing renderers and their data stores into the same AZ (with cross-AZ paths reserved for failover) is a cost decision as much as a latency one. This is the kind of egress modeling that the Cost Estimation Frameworks patterns turn into a pre-apply budget gate.

Finally, distributed nodes should resolve one another through private DNS or Route 53 Resolver endpoints so that inter-node traffic never traverses public IP space, keeping the routing contract consistent end to end.

Runnable configuration

The module below provisions a private route table for the rendering tier, points its default route at a NAT gateway, attaches a Gateway endpoint so S3 tile-cache traffic stays on the backbone, and binds the route table to every private subnet. Provider versions are pinned so the routing semantics are reproducible across environments.

# modules/tile_vpc_routing/main.tf
terraform {
  required_version = ">= 1.6.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.40"
    }
  }
}

variable "vpc_id"             { type = string }
variable "env"               { type = string }
variable "region"            { type = string }
variable "nat_gateway_id"     { type = string }
variable "private_subnet_ids" { type = list(string) }

# Private route table for the tile-rendering tier — no IGW route by design.
resource "aws_route_table" "private_tile" {
  vpc_id = var.vpc_id
  tags = {
    Name        = "${var.env}-private-tile-rt"
    Environment = var.env
    CostCenter  = "gis-platform"
    Owner       = "spatial-platform"
  }
}

# Default egress through a NAT gateway, never an Internet Gateway.
resource "aws_route" "nat_default" {
  route_table_id         = aws_route_table.private_tile.id
  destination_cidr_block = "0.0.0.0/0"
  nat_gateway_id         = var.nat_gateway_id
}

# S3 tile-cache traffic stays on the AWS backbone (no NAT data charge).
resource "aws_vpc_endpoint" "s3_tile_cache" {
  vpc_id            = var.vpc_id
  service_name      = "com.amazonaws.${var.region}.s3"
  vpc_endpoint_type = "Gateway"
  route_table_ids   = [aws_route_table.private_tile.id]
}

# Bind the private route table to every rendering subnet; replace before destroy
# so an association always exists and active tile jobs are not dropped.
resource "aws_route_table_association" "private_subnet_bind" {
  count          = length(var.private_subnet_ids)
  subnet_id      = var.private_subnet_ids[count.index]
  route_table_id = aws_route_table.private_tile.id

  lifecycle {
    create_before_destroy = true
  }
}

The route tables encode a strict split: only the public subnet reaches the internet gateway, while private subnets egress through a NAT gateway or stay on-network via VPC endpoints.

The Public / Private Route-Table Split for a Tile Platform Client tile requests enter through the internet gateway. The public subnet uses a public route table whose only default route, 0.0.0.0/0, points at the internet gateway; it hosts the ALB (the sole public ingress) and the NAT gateway (billed per GB of egress). The ALB forwards requests to tile rendering nodes — Mapnik, GeoServer, a vector pipeline — in the private subnet. The private route table sends 0.0.0.0/0 to the NAT gateway for egress and sends the S3 prefix list to a gateway endpoint, and it deliberately has no route to the internet gateway, which is the segmentation guardrail. Renderers pull geometries from PostGIS over a same-AZ private path and hydrate the S3 tile cache over the AWS backbone through the gateway endpoint, avoiding NAT data-processing charges. Internet map / tile clients public VPC Internet gateway Public subnet · public route table 0.0.0.0/0 → internet gateway ALB / reverse proxy only public ingress NAT gateway egress · billed per GB ingress egress Private subnet · private route table — no IGW route Tile rendering nodes Mapnik · GeoServer vector pipeline Private route table 0.0.0.0/0 → NAT s3 prefix → GW endpoint ✗ no 0.0.0.0/0 → IGW S3 gateway endpoint com.amazonaws.<region>.s3 request 0.0.0.0/0 PostGIS same-AZ private path S3 tile cache AWS backbone geometry fetch no NAT charge

Guardrails embedded in configuration

Several protections live in the module above rather than in operator discipline. The absence of an IGW route on private_tile is the guardrail — there is no resource that could route the rendering tier to the internet directly, so the segmentation contract cannot be violated by a single forgotten line; egress is forced through the NAT gateway or the endpoint. The Gateway endpoint carries an implicit isolation benefit: attaching s3_tile_cache to the private route table means renderer-to-S3 traffic resolves to a prefix-list route on the backbone, and a policy argument on that endpoint can be added to restrict which buckets the data plane reaches — the network mirror of IAM prefix scoping.

State locking matters disproportionately here because route tables are edited under incident pressure; a remote backend with locking, per State Backend Selection, prevents two responders from applying conflicting route changes and leaving a half-written table live. No secrets belong in routing state — route tables and endpoints carry no credentials, but any token-validating gateway they front must keep its secrets in a secret manager, never inlined.

Sizing note — NAT gateway capacity, not a percentage. NAT throughput is provisioned in concrete AWS units (a single NAT gateway scales to 100 Gbps and 5 Gbps per single flow), not as a percentage of VPC capacity. Size the count of NAT gateways to one per AZ for availability, and remember that every byte a private renderer pulls from the public internet through NAT is billed per GB processed — which is the cost argument for routing S3 traffic through the Gateway endpoint instead of NAT in the first place.

Where peering or Transit Gateway routes are added, declare them statically and validate CIDR non-overlap in CI, so a new attachment cannot silently blackhole an existing route.

Troubleshooting and failure modes

  • Asymmetric routing across a peering link. Tile fetches succeed intermittently and PostGIS connections time out under load while light traffic works. Return traffic is taking a different path than the request — usually because reciprocal route-table entries are missing on one side of the peering. Fix by declaring symmetric static routes on both route tables and verifying with a traceroute across the link; this is the core scenario in Terraform VPC Peering for Distributed GeoServer.
  • VPC endpoint policy gap. Renderers can reach S3 but receive AccessDenied only for certain buckets, or fall back to slow NAT egress. The Gateway endpoint route is attached but its endpoint policy omits the tile bucket, or the route table association was applied to the wrong subnet. Confirm the endpoint is listed in the private route table and that its policy allows the tile and raster bucket ARNs.
  • Private route table inherits a public route. A renderer unexpectedly reaches the internet directly, defeating isolation, after a console hotfix or a copied route table. The 0.0.0.0/0 entry points to an IGW instead of the NAT gateway. The policy-as-code gate exists precisely to catch this in plan; remediate by reverting to the NAT route and re-running the Checkov check.
  • CIDR overlap creating a blackhole. Cross-account traffic to a peered VPC silently disappears with no error. An overlapping CIDR allocation means the most-specific route wins locally and the peering route is never consulted. Re-IP one VPC or carve non-overlapping ranges, and add a CI assertion that peering destinations never intersect the local CIDR.
  • Connectivity drop during a route-table swap. Active tile-generation jobs fail for a few seconds during apply. A route-table association was destroyed before its replacement existed. Add create_before_destroy to the association (as in the module above) and run topology changes in a maintenance window.