CloudFront invalidations stuck in progress
An AWS CloudFront cache invalidation can remain in the In progress state longer than expected during a busy release or broad purge. The label describes AWS processing, not a guarantee that every edge has removed the object.
Most invalidations finish within minutes, though large path sets or an AWS delay can extend the wait. Treat a delay beyond current service guidance as an incident. An old object after completion points to a separate origin or downstream-cache problem.
A sound runbook moves from the distribution and API record to path scope, service health, and the request chain. Retrying adds invalidations without proving the first failed, so gather evidence first.
Australian teams should normalise timestamps. CloudFront is global, while many workloads originate in Sydney (ap-southeast-2) or Melbourne (ap-southeast-4); record UTC alongside Australian Eastern Time to avoid daylight-saving changes making a healthy sequence appear broken.
Compare symptoms before changing anything
Separate the visible symptom from the console badge. It can represent delayed AWS processing, another cache-key variant, or an object never cached at the requested URL.
| Observed behaviour | Likely cause | First check |
|---|---|---|
| In progress persists beyond normal completion | Processing delay or excessive path scope | API status, path count, AWS Health |
| Completed but old content remains | Origin or downstream cache; stale URL or variant | Exact URL, headers, checksum |
| Request never appears | Wrong distribution, permissions, or caller reference | Distribution ID, IAM policy, CloudTrail |
| API times out during submission | Request may have succeeded | List invalidations before retrying |
If the API reports Completed, trace the object rather than the invalidation. If it remains In progress, record the invalidation ID and creation time before testing another path or escalating.
Verify the distribution and API record
Treat the API as the operational source of truth. Confirm the distribution ID, account, Deployed status, and permissions for CreateInvalidation, GetInvalidation, and ListInvalidations.
aws cloudfront get-distribution \
--id E1EXAMPLE \
--query 'Distribution.{Status:Status,Domain:DomainName}'
aws cloudfront list-invalidations \
--distribution-id E1EXAMPLE --max-items 20 \
--query 'InvalidationList.Items[].{Id:Id,Status:Status,CreateTime:CreateTime,Paths:InvalidationBatch.Paths.Items}'
Use get-invalidation when the list is ambiguous, and compare CloudTrail management events with CLI output. Console refreshes can lag behind the API.
Check for an active deployment, disabled distribution, or role-policy boundary that blocks programmatic calls. Record the distribution ID and invalidation ID separately because they identify different resources.
Check path scope and duplicate requests
An invalidation path must begin with /. Query strings need care because a cache policy can create separate cached variants. Map the cache policy, cache behaviour, and exact production URL before choosing paths.
Keep purges narrow enough to understand and price. A wildcard such as /* is one path but can affect many cached objects; repeated variants consume the monthly free allowance and then incur charges under current AWS pricing. Prefer versioned asset names or a short changed-path list.
A timed-out create call is ambiguous, not proof of failure. List invalidations and inspect CloudTrail before resubmitting, because a duplicate adds cost and processing work. Retain a unique caller reference in the change record.
Rule out an AWS processing delay
Check CloudFront service status and AWS Health, including historical incidents. The service is global, so a healthy workload region does not rule out an edge or control-plane problem. A broad path set can lengthen processing even without a published platform incident.
When escalating, include the distribution ID, invalidation ID, caller reference, exact paths, UTC timestamps, and API responses. State the elapsed time against AWS's current guidance. Teams in Sydney and Melbourne should also record Australian Eastern Time, including AEDT, so daylight-saving boundaries do not distort the sequence.
Trace content through the full cache chain
After completion, request the exact production URL and inspect Age, X-Cache, ETag, Last-Modified, Vary, and Cache-Control. A different ETag or Age suggests another variant; an identical stale ETag points toward the origin or a downstream cache.
Use a cache-busting query only for diagnosis because it creates another cache key. Compare checksums, then check DNS, origin, middleware, and cache policies. Confirm the URL resolves to the intended distribution rather than an old alias.
Australian users may retain copies through NBN connections, mobile carriers, fixed-wireless providers, browsers, or service workers. Compare headers from controlled networks and use a private window before declaring an edge purge ineffective.
Correlate security events and deployment activity
Overlay the invalidation time with CloudTrail, CI/CD, WAF, DNS, and load-balancer logs. Deployments commonly trigger purges, but traffic spikes or blocked-request bursts can too. Separate planned releases from security events before purging again.
Where Radware protects the origin, correlating DefensePro and SIEM events with change records can reveal whether an attack burst coincided with the purge; this DefensePro threat intelligence walkthrough shows how to preserve that context. Treat it as correlation, not proof of a CloudFront delay.
For Australian organisations covered by the Privacy Act 1988 and the Notifiable Data Breaches scheme, CloudFront, WAF, and SIEM logs may contain IP addresses and application identifiers. Restrict access and apply retention rules. Australian buyers using AWS Marketplace security tools should confirm local support and escalation entitlements.
Recovery runbook and evidence checklist
Use this sequence during an active incident and retain the outputs with the deployment record:
- Capture the distribution ID, invalidation ID, caller reference, paths, and UTC timestamps before retrying.
- Compare API status with deployment state, AWS Health, CloudTrail, and the exact object response.
- Test one narrow path and verify the resulting miss, revalidated response, or fresh origin checksum before expanding the purge.
- Escalate with reproducible commands and response headers when processing exceeds AWS guidance.
The final hand-over should state whether the delay was AWS-side, caused by request scope, or only appeared as stale content downstream. A verified invalidation ID, UTC timeline, and checksum give the next operator a defensible starting point without creating another blind purge.