Purge a single domain and it's content from a Fastly service with multiple domains

How we configure Fastly at work is typically to use a single Fastly service and add multiple domains to each service. This works well for a number of reasons:

• Configuration - each service has it's own configuration and VCL snippets. It is common to have the same configuration extent to multiple sites.
• Cost - number of services is a factor in Fastly costs

One of the current limitations out of the box, is that you cannot purge a single domain (and its contents) from a service, without dropping the entire cache. This post will explore a solution to this.

Altering Surrogate-Key in Fastly VCL

It turns out you can add a custom snippet in vcl_fetch to which will adjust the response from origin, to create (or append to) the current domain to HTTP header Surrogate-Key.

# We want to append the host of the request to the response Surrogate-Key
# header, or create it, if it does not already exist. We only want to do this
# once, and this includes if shielding is in use.
#
# This is so we can purge the domain, of all requests, including static files,
# without touching other domains on the service.
if ( !beresp.http.Surrogate-Key ) {
  set beresp.http.Surrogate-Key = req.http.host;
}
else {
  if ( !std.strstr(beresp.http.Surrogate-Key, req.http.host) ){
    set beresp.http.Surrogate-Key = beresp.http.Surrogate-Key " " req.http.host;
  }
}

N.B. This new logic will only apply to new objects in cache only, existing cached objects will be unaffected. It pays to think ahead for this logic. You might also consider dropping the entire service cache after you have the VCL in place (if this is an option).

How to test if it is working

You can use cURL to test this snippet is working, you need to pass a debug header Fastly-Debug in order to see this:

curl -sLIXGET https://www.example.com/ -H 'Fastly-Debug:1' | grep -i surrogate-key
surrogate-key: www.example.com

If you have existing cache tags in the response, then you should see the custom cache tag appended at the end:

curl -sLIXGET https://www.exampletwo.com/ -H 'Fastly-Debug:1' | grep -i surrogate-key
surrogate-key: aZw7 f57P eygW 45lo gq5V Czwh cvlF /+/A 7KwZ eth0 www.exampletwo.com

How to purge an entire domain using the Fastly API

Now armed with the knowledge that every single cached object in Fastly has a custom cache tag attached to it (being the domain) we can easily purge a single domain using the Fastly API:

curl -sXPOST https://api.fastly.com/service/SERVICE_ID/purge/www.example.com -H 'Fastly-Soft-Purge:1' -H "Fastly-Key:$FASTLY_TOKEN" | jq
{
  "status": "ok",
  "id": "10427-1615460322-6"
}

N.B. this is executing a 'Soft Purge', so the object is still in Fastly cache, it is just marked as stale. This is useful if you have configured Serve While Stale (SWS), as stale objects can be served in the event the origin is down or otherwise not returning happy responses. So this is a much safer purge. If you want to read more about this, see my blog post on Soft Purge vs Instant Purge.

Further thoughts

I did ask Fastly if they would consider adding another option to the purge dropdown (my attempt at a mockup below), without having to muck around with custom VCL snippets.

Apparently this is not a popular feature request. If this blog post was helpful for you, I would encourage you to contact Fastly to let them know this would make your life easier.

Stock Varnish can BAN by domain, so this is not a limitation of Varnish.

Comments

If this has helped you, or you have an improvement to the VCL, please let me know in the comments.