Budget Alerts and Anomaly Detection for Geospatial Spend

A pre-merge cost estimate tells you what a change should cost. It cannot tell you that a crawler discovered your tile endpoint on Saturday, that a reprocessing job has been retrying a corrupt scene for nine hours, or that a lifecycle rule stopped matching after a prefix rename and six months of imagery is still sitting in the most expensive storage class. Those are runtime cost events, and they need runtime detection. This guide is the operational half of Cost Estimation Frameworks, within Spatial IaC Architecture and Fundamentals, and it covers the alerting that catches what the estimate structurally cannot.

Symptom identification and triage

Geospatial cost incidents have a characteristic shape that distinguishes them from ordinary cloud overspend: the driver is almost always volume rather than rate, and the volume is usually egress or compute-seconds rather than storage. Four patterns account for most of them.

Tile egress spike. A sudden climb in data transfer with flat request counts at the origin, meaning the CDN is serving more but the platform is not working harder. Usually a scraper, an embedded map on a newly popular page, or a client that disabled its cache. Distinguishable from legitimate growth by its shape: real growth is gradual and diurnal, a scraper is a step function that ignores time of day.

Retry storms in a raster pipeline. Compute-seconds climb with no increase in successfully processed scenes. One malformed input that fails after twenty minutes and retries indefinitely can consume more compute in a weekend than a month of normal ingestion. The signature is a rising ratio of invocations to completions.

Storage-class drift. Storage cost rises steadily while object count is flat. Something stopped transitioning — most often a lifecycle rule whose prefix no longer matches after a naming change, which is silent because a rule that matches nothing is not an error.

Cross-zone or NAT traffic from a topology change. Data-transfer charges appear on an internal path that used to be free, typically after a renderer was rescheduled into a different availability zone from its database, or after a gateway endpoint was removed and object reads began routing through NAT.

Prerequisites and environment assumptions

Terraform 1.6 or later with hashicorp/aws pinned at ~> 5.60. A tag contract already enforced, because every alert here is scoped by tag and an untagged resource is invisible to all of them — the labels described in FinOps Tagging Strategies for Geospatial Resources are a hard dependency, not a nice-to-have. Cost allocation tags activated in the billing account, and at least one full billing cycle of history so anomaly detection has a baseline to compare against.

Decide before you build who receives these alerts. A cost alert routed to a finance mailbox arrives days after the engineer who could stop the bleeding has gone home. Route the anomaly alerts to the same on-call channel that receives operational alarms, and reserve the monthly budget summaries for the finance recipients.

Step-by-step implementation

1. Set a budget per workload, not per account. An account-level budget on a platform running ingestion, serving and analytics tells you that something is expensive, which you already knew. Budgets scoped by the WorkloadType tag tell you which thing, and that is the difference between an alert and a diagnosis.

2. Alert on forecast, not only on actual. An actual-spend threshold at 80 per cent fires on the twenty-fourth day of a month for a workload that will overshoot by 40 per cent. A forecast threshold fires on the fifth, while the month is still recoverable. Configure both: forecast for early warning, actual for the hard limit.

3. Enable anomaly detection on the dimensions that move. Monitors on data transfer and on compute for the spatial services catch the volume-driven incidents that a fixed threshold misses, because they compare against the workload’s own learned pattern rather than a number someone chose a quarter ago.

4. Add a ratio alarm for retry storms. Cost tooling will not catch this cheaply, because the spend rises smoothly. An operational alarm on invocations divided by successful completions catches it in minutes rather than at the next billing refresh, which is often a day later.

5. Close the loop with an automated response for the worst case. A budget action that attaches a restrictive policy when a threshold is breached is appropriate for non-production environments. In production, page instead — an automated throttle on a live tile platform converts a cost incident into an availability incident.

terraform {
  required_version = ">= 1.6.0"
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.60"
    }
  }
}

variable "oncall_topic_arn" { type = string }
variable "finance_email" { type = string }

resource "aws_budgets_budget" "tile_serving" {
  name         = "tile-serving-monthly"
  budget_type  = "COST"
  limit_amount = "4000"
  limit_unit   = "USD"
  time_unit    = "MONTHLY"

  # Scoped by tag: an account-level budget says something is expensive,
  # a workload-scoped one says which thing.
  cost_filter {
    name   = "TagKeyValue"
    values = ["user:WorkloadType$tile-serving"]
  }

  # Forecast first — this fires on day five, while the month is recoverable.
  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 100
    threshold_type             = "PERCENTAGE"
    notification_type          = "FORECASTED"
    subscriber_sns_topic_arns  = [var.oncall_topic_arn]
  }

  # Actual as the hard limit, to finance as well as on-call.
  notification {
    comparison_operator        = "GREATER_THAN"
    threshold                  = 90
    threshold_type             = "PERCENTAGE"
    notification_type          = "ACTUAL"
    subscriber_sns_topic_arns  = [var.oncall_topic_arn]
    subscriber_email_addresses = [var.finance_email]
  }
}

resource "aws_ce_anomaly_monitor" "spatial_transfer" {
  name              = "spatial-data-transfer"
  monitor_type      = "DIMENSIONAL"
  monitor_dimension = "SERVICE"
}

resource "aws_ce_anomaly_subscription" "spatial_transfer" {
  name      = "spatial-transfer-anomalies"
  frequency = "IMMEDIATE"
  monitor_arn_list = [aws_ce_anomaly_monitor.spatial_transfer.arn]

  subscriber {
    type    = "SNS"
    address = var.oncall_topic_arn
  }

  # An absolute floor stops routine noise; the detector still compares against
  # the workload's own learned pattern rather than a number chosen last quarter.
  threshold_expression {
    dimension {
      key           = "ANOMALY_TOTAL_IMPACT_ABSOLUTE"
      match_options = ["GREATER_THAN_OR_EQUAL"]
      values        = ["150"]
    }
  }
}

# Retry storms rise smoothly in cost and are invisible to a budget until the
# next billing refresh. The ratio catches them in minutes.
resource "aws_cloudwatch_metric_alarm" "raster_retry_ratio" {
  alarm_name          = "raster-pipeline-retry-ratio"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 2
  threshold           = 3

  metric_query {
    id          = "ratio"
    expression  = "invocations / IF(completions > 0, completions, 1)"
    label       = "invocations per completion"
    return_data = true
  }
  metric_query {
    id = "invocations"
    metric {
      namespace   = "SpatialPlatform/Raster"
      metric_name = "JobInvocations"
      period      = 300
      stat        = "Sum"
    }
  }
  metric_query {
    id = "completions"
    metric {
      namespace   = "SpatialPlatform/Raster"
      metric_name = "JobCompletions"
      period      = 300
      stat        = "Sum"
    }
  }

  alarm_actions = [var.oncall_topic_arn]
}

Verification

Prove each alert can fire before you rely on it. For the budget notifications, set a temporary limit below current spend in a non-production account and confirm the notification arrives at the intended destination — this validates the tag filter, which is where these silently fail, since a filter matching no resources produces a budget that is permanently at zero per cent and looks healthy.

For the ratio alarm, publish synthetic values to the two metrics and confirm the expression evaluates and the alarm transitions. For anomaly detection, confirm the monitor has ingested at least a full cycle of history and shows a baseline; a monitor created yesterday will not fire meaningfully today, and believing otherwise leaves a gap nobody is watching. The general technique is the alarm-firing test described in Observability for Spatial Infrastructure.

Preventing recurrence

• Make the tag contract a merge gate. Every alert here is tag-scoped, so an untagged resource is an unmonitored resource. Enforce the tags in the policy stage rather than auditing for them later.
• Attach a cost owner to each budget. A budget alert with no named owner is a notification, not an action.
• Re-baseline after deliberate growth. A launch that legitimately triples traffic should be followed by an explicit threshold update, or every subsequent week produces a false alarm and the alerts get muted.
• Review the four signatures quarterly. Each of the incident patterns above corresponds to a specific control — cache policy, retry limits, lifecycle prefixes, endpoint routing — and reviewing them is cheaper than detecting their failure.

Frequently Asked Questions

Related

• Cost Estimation Frameworks — the parent topic covering pre-merge estimation
• FinOps Tagging Strategies for Geospatial Resources — the tag contract every alert here depends on
• Modeling Egress Costs for Raster Tile Workloads — the model an egress spike departs from
• Observability for Spatial Infrastructure — the alarm-testing technique used to verify these alerts