Spatial IaC Terraform & Pulumi for GIS

Rotating KMS Keys Without Breaking Tile Pipelines

There are two operations both called key rotation, and confusing them is how an organisation makes a decade of imagery unreadable. Automatic rotation replaces the key material behind a key while retaining every previous version, so everything encrypted under the old material stays readable and nothing needs to change. Manual rotation creates a new key and retires the old one, and every object still encrypted under the old key remains readable only while that key is enabled — so the retirement step is a data-availability event dressed up as housekeeping. This guide extends Encryption and Key Management for Spatial Data within Network Security and Access Control.

Symptom identification and triage

You are in one of three situations, and they have different urgency.

Routine automatic rotation. Nothing to do. The key identifier is unchanged, prior material is retained, and no ciphertext needs re-encrypting. If someone has raised a ticket for this, the correct outcome is closing it.

A required migration to a new key. Compliance requires a fresh key, or a key’s grant list has grown untrustworthy, or a key was exposed to an account that should no longer have access. This needs the full procedure below, because objects do not re-encrypt themselves and the old key must stay enabled until the last one has moved.

A key already disabled with data still encrypted under it. An incident. Objects encrypted under a disabled key return an access-denied or invalid-state error on read, and the tile pipeline that reads them fails. Re-enable the key first and diagnose afterwards; a disabled key is recoverable, a deleted one is not.

The diagnostic that separates the second from the third is a scan of the archive’s object metadata, which records the key identifier per object. A prefix reporting a key that is no longer enabled is unreadable data waiting to be discovered.

Automatic rotation and key migration are different operations Under automatic rotation, one key identifier remains constant while its underlying material is replaced periodically. Every previous version of the material is retained, so objects encrypted years ago decrypt without any action, and no ciphertext ever needs rewriting. Under a migration, a new key is created and new writes use it, but every existing object remains encrypted under the old key until it is explicitly re-encrypted by copying it in place. The old key must stay enabled for the entire migration, and disabling or deleting it before the last object has moved makes that data unreadable — permanently, in the case of deletion. Automatic rotation — nothing to do one key identifier, unchanged forever material v1 · v2 · v3 — all previous versions retained, so old ciphertext still decrypts no object is rewritten Migration to a new key — every object must move Old key — stays enabled every existing object still needs it disable it early and that data is gone New key — new writes only existing objects move by copy-in-place until then both keys are load-bearing

Prerequisites and environment assumptions

Terraform 1.6 or later with hashicorp/aws pinned at ~> 5.60. An inventory of what the key protects — a bucket inventory report is the practical instrument, because it lists every object with its encryption key identifier and is what turns “probably everything in this prefix” into a number. The compute capacity to re-encrypt, which for a large raster archive is a real job: re-encryption is a server-side copy of every object, and a multi-terabyte archive takes hours and costs request charges.

Confirm before starting that both keys grant decrypt to every workload that reads the data. During the migration the archive contains objects under both keys, so a reader with access to only one of them fails on a subset of objects — which presents as an intermittent, prefix-correlated failure and is confusing precisely because most reads succeed.

Step-by-step migration

  1. Create the new key and grant it to every existing reader. Same policy shape as the old one. Do this first and verify it before any object moves, because a grant missed here produces failures that look random.

  2. Point new writes at the new key. Update the bucket’s default encryption configuration. From this moment the archive is mixed, and both keys are load-bearing.

  3. Re-encrypt existing objects by copying in place. A server-side copy with the new key identifier rewrites the object’s encryption without transferring data out. Work prefix by prefix, oldest first, and keep a record of which prefixes have completed so the job is resumable — it will be interrupted.

  4. Verify the archive is homogeneous. Run an inventory report and confirm no object still names the old key. This is the gate before step five, and skipping it is how the next step becomes an incident.

  5. Disable the old key and wait. Disable rather than delete, and leave it disabled for at least a full business cycle — long enough for a quarterly job to run. If something breaks, re-enabling is instant.

  6. Schedule deletion only after the waiting period. With a long deletion window, so that even this step is reversible for a month.

terraform {
  required_version = ">= 1.6.0"
  required_providers {
    aws = { source = "hashicorp/aws", version = "~> 5.60" }
  }
}

resource "aws_kms_key" "raster_v2" {
  description             = "Domain key: raster archive (v2, migrated from v1)"
  enable_key_rotation     = true
  deletion_window_in_days = 30
  tags = {
    Domain        = "raster-archive"
    Supersedes    = aws_kms_key.raster_v1.key_id
    RotationModel = "automatic-annual"
  }
}

# Step 2: new writes use v2. From here the archive is mixed and BOTH keys are
# load-bearing until the last object has been copied.
resource "aws_s3_bucket_server_side_encryption_configuration" "raster" {
  bucket = aws_s3_bucket.raster.id
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.raster_v2.arn
    }
    bucket_key_enabled = true
  }
}

# Readers need decrypt on BOTH keys for the duration. A reader granted only v2
# fails on the not-yet-migrated subset, which presents as intermittent and
# prefix-correlated rather than as a clean permissions error.
data "aws_iam_policy_document" "reader_dual_key" {
  statement {
    effect  = "Allow"
    actions = ["kms:Decrypt", "kms:DescribeKey"]
    resources = [
      aws_kms_key.raster_v1.arn,
      aws_kms_key.raster_v2.arn,
    ]
  }
}

# Step 5 is deliberately NOT expressed here. Disabling the old key is a manual,
# verified action taken after an inventory report proves no object still
# references it — not something an apply should be able to do by accident.
#!/usr/bin/env bash
# Step 3: re-encrypt by server-side copy. No data leaves the service; the
# object's encryption is rewritten in place.
set -euo pipefail

BUCKET="gis-raster-archive"
NEW_KEY_ARN="$1"
PREFIX="$2"
STATE="/var/lib/keymig/${PREFIX//\//_}.done"

# Resumable by design: this job WILL be interrupted on a multi-terabyte archive.
touch "$STATE"

aws s3api list-objects-v2 --bucket "$BUCKET" --prefix "$PREFIX" \
  --query 'Contents[].Key' --output text | tr '\t' '\n' | while read -r key; do
  grep -qxF "$key" "$STATE" && continue

  aws s3api copy-object \
    --bucket "$BUCKET" --key "$key" \
    --copy-source "${BUCKET}/${key}" \
    --server-side-encryption aws:kms \
    --ssekms-key-id "$NEW_KEY_ARN" \
    --bucket-key-enabled \
    --metadata-directive COPY >/dev/null

  echo "$key" >> "$STATE"
done

# Step 4 gate: nothing may still reference the old key before anything is
# disabled. Run this from the inventory report, not from a sample.
echo "Now verify with an inventory report that no object names the old key."
Where the old key is still load-bearing A timeline across six steps. From creating the new key, through pointing new writes at it, through re-encrypting every existing object prefix by prefix, and through verifying with an inventory report that nothing still references the old key, the old key must remain enabled — every unmigrated object depends on it. Only after verification succeeds is the old key disabled, and it stays disabled for a full business cycle so that infrequent jobs have a chance to fail visibly while recovery is instant. Deletion is scheduled only after that, and with a long window so even the final step remains reversible for a month. 1 · New key grant all readers 2 · New writes archive now mixed 3 · Re-encrypt resumable, by prefix 4 · Verify inventory report 5 · Disable not delete Old key must stay ENABLED across all of this then a cycle Disable is instantly reversible. Deletion is not — schedule it only after a full business cycle has passed quietly.

Verification

The gate before disabling anything is an inventory report showing zero objects referencing the old key. Do not sample; a sample that misses one prefix is exactly the failure this step exists to prevent, and prefixes are where unmigrated objects hide because migration proceeds prefix by prefix.

After disabling, exercise the pipelines rather than waiting to hear about them: run a tile render over the migrated archive, run the raster ingestion path end to end, and restore a snapshot if the key also protects backups. Check that the key-usage metrics for the old key drop to zero and stay there — a non-zero count after disabling means something still tried, and knowing what that was is more valuable before deletion than after.

Sample the wrong way and the gate reports clean Re-encryption proceeds prefix by prefix, so objects still under the old key are not scattered randomly through the archive — they are concentrated in whole prefixes that have not been reached yet. A sample drawn across the archive can easily miss such a prefix entirely and report a clean result, which is precisely the failure the pre-disable gate exists to prevent. Only a full inventory report, listing every object with its encryption key identifier, can establish that nothing still references the old key. A sample unmigrated objects sit in whole prefixes a sample can miss a prefix entirely and report clean when it is not A full inventory report every object, with its key identifier zero rows naming the old key the only gate worth disabling a key on Migration proceeds prefix by prefix — so prefixes are exactly where a sample goes wrong.

Preventing recurrence

  • Prefer automatic rotation everywhere it is supported. It removes this entire procedure from the calendar and eliminates the failure mode where a retired key takes historical data with it.
  • Record the rotation model on every key. The operator asking “is this safe to disable” needs a fact, not a reconstruction.
  • Run an inventory report on a schedule, not only during a migration. It is the only instrument that shows a mixed archive, and a mixed archive can also arise from a misconfigured writer.
  • Keep the deletion step out of automation. An apply that can schedule a key deletion is an apply that can make an archive unreadable, and the assertion set in Testing and Validation for Spatial IaC should fail any plan that does so without an explicit approval marker.

Frequently Asked Questions

Does automatic rotation require me to re-encrypt anything?

No. Previous key material is retained and selected automatically by the ciphertext’s key version, so objects encrypted under earlier material decrypt without any action. Re-encryption is only needed when moving to a genuinely different key.

What happens to objects still encrypted under a disabled key?

They become unreadable until the key is re-enabled. This is recoverable and instant. Deletion is not recoverable, which is why disable-and-wait sits between verification and deletion rather than being skipped.

How long does re-encrypting a large raster archive take?

Hours to days, bounded by request throughput rather than by data transfer, since a server-side copy does not move data out of the service. Plan for the job to be interrupted and make it resumable from the start rather than after the first interruption.

Can I migrate keys without a window where both are needed?

No. Between the first re-encrypted object and the last, the archive contains both, so both keys must decrypt. Attempting to avoid that window is what produces the intermittent, prefix-correlated failures that make this migration look mysterious.