Spatial IaC Terraform & Pulumi for GIS

Scaling Vector Tile Services for Peak Map Traffic

Map traffic does not scale like web traffic. A news story about a wildfire, a storm warning, an election night results map — each produces a step change in load concentrated on a small geographic extent and a narrow band of zoom levels, which is the worst possible shape for a tile platform. The requests that arrive are exactly the ones the cache does not hold, because nobody was looking at that area an hour ago, and they resolve to the most expensive queries in the system, because dense urban geometry at zoom 14 is where the cost lives. Scaling for this is a different exercise from scaling for growth. This guide extends Vector Tile Service Provisioning within Geospatial Resource Provisioning.

Where the ceiling actually is

Adding renderer replicas is the instinctive response and is usually not the constraint. Four ceilings exist and they bind in a predictable order.

Cache hit ratio. The dominant term. At a 95 per cent hit ratio the origin sees five per cent of requests; at 80 per cent it sees four times as many for identical user traffic. A traffic event pushes the ratio down precisely because the requested tiles are novel, so origin load rises far faster than user load. Improving cache behaviour is worth more than any amount of compute.

Database connections. The hard ceiling. Pool size multiplied by replica count is bounded by max_connections, so beyond a certain replica count adding renderers does nothing except bring the database closer to refusing connections.

Query cost at high zoom. Dense geometry at zoom 13 to 16 costs an order of magnitude more per tile than sparse geometry at zoom 8, and a viewport over a city is entirely composed of the expensive kind.

Renderer CPU. Usually the last to bind, and the one everyone scales first.

The four ceilings, in the order they bind Cache hit ratio binds first and dominates everything else: a fall from ninety-five to eighty per cent quadruples origin request volume without any change in user traffic, and a sudden traffic event drives exactly that fall because the requested tiles are novel. Database connections bind second and form a hard ceiling, since pool size multiplied by replica count cannot exceed the connection limit, so beyond that point adding renderers only brings the database closer to refusing connections. Query cost at high zoom binds third, because dense urban geometry costs an order of magnitude more per tile than sparse rural geometry. Renderer CPU binds last, and is the ceiling teams usually try to raise first. 1 · Cache hit ratio 95% → 80% quadruples origin load for identical user traffic; an event drives exactly that fall 2 · Database connections a hard ceiling — past it, more replicas only bring the database closer to refusing 3 · Query cost at high zoom dense city geometry at z14 costs an order of magnitude more per tile than sparse z8 4 · Renderer CPU binds last — and is the one almost everyone scales first

Prerequisites and environment assumptions

Terraform 1.6 or later with hashicorp/aws pinned at ~> 5.60. A tile service already provisioned per Provisioning pg_tileserv on ECS Fargate. A CDN in front of the origin. Metrics broken down by zoom band and cache result, because none of the reasoning below is possible against aggregate numbers — this is the concrete payoff of the dimension discipline in Observability for Spatial Infrastructure.

Know your current numbers before changing anything: cache hit ratio, origin requests per second at peak, p99 origin latency by zoom band, and peak pool utilisation. Scaling decisions made without these are guesses that happen to be expensive.

Step-by-step scaling

  1. Normalise the cache key first. This is the cheapest large win. A tile URL carrying a client-generated cache-buster, a session parameter or an inconsistent parameter order produces distinct cache entries for identical tiles, and a platform can be running at a 60 per cent hit ratio purely because three clients spell the same request differently. Strip every query parameter that does not change the bytes.

  2. Raise the cache time-to-live to match how the data actually changes. Tiles from a dataset that updates nightly do not need a five-minute TTL. Use a long TTL with a versioned path or a purge on publish, so freshness comes from invalidation rather than from expiry. This is the difference between a cache that absorbs an event and one that re-fetches everything every few minutes during it.

  3. Serve stale while revalidating. During an origin slowdown, serving a slightly old tile is nearly always better than serving an error, and for a basemap it is invisible. This single directive converts an origin incident into a latency footnote for most users.

  4. Pre-warm the extent when you can predict it. Some events are predictable — a scheduled data publication, an election, an announced release. Requesting the tiles for the expected extent and zoom band before the traffic arrives converts the worst case into a warm cache.

  5. Scale renderers on concurrency with a cooldown longer than cache fill. Scaling on raw request count against a cache-mediated origin produces oscillation: the fleet scales up, the cache warms, load drops, the fleet scales down, and the next wave finds it small again. A cooldown longer than the fill time damps it.

  6. Shed load deliberately at the edge rather than failing everywhere. A rate limit per client at the CDN keeps a scraper or a runaway client from consuming the capacity that interactive users need. Failing a few abusive callers is a better outcome than degrading for everyone.

terraform {
  required_version = ">= 1.6.0"
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 5.60" }
  }
}

resource "aws_cloudfront_cache_policy" "tiles" {
  name        = "vector-tiles"
  default_ttl = 86400
  max_ttl     = 604800
  min_ttl     = 3600

  parameters_in_cache_key_and_forwarded_to_origin {
    # Strip everything that does not change the bytes. A client-generated
    # cache-buster or an inconsistent parameter order otherwise creates a
    # distinct cache entry for an identical tile.
    query_strings_config {
      query_string_behavior = "none"
    }
    headers_config {
      header_behavior = "none"
    }
    cookies_config {
      cookie_behavior = "none"
    }
    enable_accept_encoding_gzip   = true
    enable_accept_encoding_brotli = true
  }
}

resource "aws_cloudfront_response_headers_policy" "tiles" {
  name = "vector-tiles-headers"
  custom_headers_config {
    items {
      header = "Cache-Control"
      # Serving a slightly old tile beats serving an error, and for a basemap
      # the difference is invisible to the user.
      value    = "public, max-age=86400, stale-while-revalidate=604800, stale-if-error=604800"
      override = true
    }
  }
}

# Scale on concurrency, with a cooldown longer than the cache fill time so the
# fleet does not oscillate as the cache warms and load falls away.
resource "aws_appautoscaling_policy" "tileserv_concurrency" {
  name               = "tileserv-concurrency"
  policy_type        = "TargetTrackingScaling"
  resource_id        = aws_appautoscaling_target.tileserv.resource_id
  scalable_dimension = aws_appautoscaling_target.tileserv.scalable_dimension
  service_namespace  = aws_appautoscaling_target.tileserv.service_namespace

  target_tracking_scaling_policy_configuration {
    target_value = 60.0
    customized_metric_specification {
      metrics {
        id    = "utilisation"
        label = "pool utilisation percent"
        return_data = true
        expression  = "100 * active / capacity"
      }
      metrics {
        id = "active"
        metric_stat {
          metric {
            namespace   = "SpatialPlatform/Tiles"
            metric_name = "ActiveConnections"
          }
          stat = "Average"
        }
        return_data = false
      }
      metrics {
        id = "capacity"
        metric_stat {
          metric {
            namespace   = "SpatialPlatform/Tiles"
            metric_name = "PoolCapacity"
          }
          stat = "Average"
        }
        return_data = false
      }
    }
    scale_in_cooldown  = 600 # slow to shrink: the next wave finds the fleet ready
    scale_out_cooldown = 120
  }
}
How a traffic event amplifies into origin load A traffic event begins with a rise in user requests concentrated on a small geographic extent. Because those particular tiles were not being requested an hour earlier, the cache does not hold them and the hit ratio falls. Origin request volume therefore rises much faster than user traffic — a fall from ninety-five to eighty per cent hit ratio multiplies origin load fourfold on its own. The requests that arrive are concentrated in high zoom bands over dense geometry, which are the most expensive queries the platform runs. Two interventions blunt the chain before any scaling is needed: a normalised cache key raises the baseline hit ratio, and serving stale content while revalidating keeps users served during the origin slowdown. Event narrow extent Hit ratio falls the tiles are novel Origin load ×4 user traffic barely doubled On the dear queries high zoom, dense geometry Normalise the cache key raises the baseline hit ratio before any event cheapest large win available stale-while-revalidate users stay served through the origin slowdown for a basemap the staleness is invisible

Verification

Measure rather than assume. After normalising the cache key, the hit ratio should move within an hour and the change should be visible as a drop in origin requests per second at constant user traffic — if it does not move, something is still varying the key, and the CDN’s cache statistics by URL will show which.

Then run a load test with a realistic shape. A test that requests random tile coordinates across the whole world tests nothing useful, because that traffic caches beautifully and costs almost nothing. Replay a real access log, or synthesise a viewport-shaped pattern over dense geometry at zoom 13 to 16, and watch pool utilisation and p99 by zoom band. Confirm the autoscaling policy reacts and, importantly, that it settles — a fleet that oscillates in the test will oscillate under an event.

Finally test the failure path: block the origin and confirm the CDN continues serving stale tiles rather than errors. stale-if-error untested is stale-if-error unproven.

Two load tests, only one of which resembles real traffic A test requesting random tile coordinates across the whole world spreads its load evenly, misses the cache in a uniform way and touches mostly empty ocean and sparse rural extents, so it costs almost nothing and tells you nothing about capacity. A test that replays a real access log, or synthesises a viewport-shaped pattern over dense urban geometry between zoom thirteen and sixteen, reproduces both the concentration and the query cost that a genuine traffic event produces — which is what makes pool utilisation and per-zoom latency readings from it meaningful. Random global coordinates evenly spread, mostly ocean cheap queries, uniform misses measures nothing you will face Viewport-shaped, dense, z13–z16 concentrated on one extent the expensive queries, novel tiles pool and latency readings mean something And watch that the fleet settles: one that oscillates in the test will oscillate under a real event.

Preventing recurrence

  • Alarm on cache hit ratio, not only on origin latency. A falling ratio precedes an origin problem by minutes, and it is the earliest actionable signal a tile platform has.
  • Keep the URL contract stable and parameter-free. Every parameter a client can vary is a cache fragmentation risk; document the canonical tile URL and treat additions as breaking changes.
  • Rehearse the predictable events. For a scheduled publication or an announced launch, pre-warming should be a pipeline step, not a heroic act.
  • Review the zoom-band cost distribution quarterly. As data densifies, the expensive band moves, and a generalisation strategy tuned two years ago may be leaving full-resolution geometry in a band that is now hot.

Frequently Asked Questions

Should I scale renderers or the database first?

Neither, first. Fix the cache. Only once the hit ratio is high and stable does the origin’s capacity become the limiting factor, and at that point the answer is usually read replicas rather than renderers, because the pool ceiling binds before renderer CPU does.

Why does my fleet oscillate during a traffic spike?

Because it is scaling on a signal the scaling itself removes: more capacity warms the cache, load falls, the fleet shrinks, and the next wave finds it small. Lengthen the scale-in cooldown well past the cache fill time and scale on a smoothed concurrency signal rather than raw request count.

Is pre-generating tiles a better answer than scaling?

For any layer stable enough to pre-generate, yes and by a wide margin — a precomputed archive served from object storage removes the database from the request path entirely. The trade-off between live and precomputed is set out in Vector Tile Service Provisioning, and most platforms should be running both.

How do I stop one client consuming the whole platform?

Rate-limit per client at the edge. It is uncomfortable to fail requests deliberately, but the alternative during an event is that a single misbehaving client degrades the service for every interactive user, which is strictly worse.