Fastly Soft Purge vs Instant Purge - what is the difference?

This topic came up a while ago when Acquia was looking to introduce Soft Purge as an option for their Acquia purge module in Drupal (N.B. not to be confused by the Fastly Drupal module).

There appears to be some confusion about what Fastly Soft Purge is, when it is useful, and how to use it. This post will help clear this topic up.

What is Fastly Soft Purge?

In Varnish, to force a miss for an object in cache you have a couple of options

• evict it completely (e.g. PURGE, or BAN with a lurker). The Varnish documentation cover this. This is the same as Fastly Instant Purge.
• or simply mark the object as expired. I imagine Fastly achieve this by tinkering with the TTL of the cached object. This is what Fastly Soft Purge is.

So the main difference with Fastly Soft Purge is that the object persists in cache, and can be used in a stale context. This can prove to be extremely useful.

When is Fastly Soft Purge useful?

I would say Fastly Soft Purge should be used in 99.99% of all purges. There are extremely limited situations in which I would recommend Fastly Instant Purge:

• You accidentally published embargoed content, or there is some legal issue with the content
• You accidentally published incorrect or incomplete content
• You accidentally published test content (we have all been there)
• You need to destroy the entire cache for a given Fastly service 😱 (which I never really recommend)

It boils down to making a whoopsie really, and I would imagine the vast majority of your purges should (hopefully) not fall into these categories.

Having a stale object is cache is useful as it:

• soft purges to high traffic pages (e.g. the homepage) will not result in slow downs for new users, this is because "Stale while revalidate" will perform a background (asynchronous) refresh of the cached object, and will serve the cached stale object in the meantime.
• if your origin is completely down (or in maintenance) stale objects can be served to users (which is often better than a generic error page). This is the "Stale if error" feature.

How to use Fastly Soft Purge with the Drupal module

Just because you execute a purge to Fastly does not mean you will instantly get any benefit, you also need to combo this with some "Stale Content Options" to which I will go through.

If you are using the Fastly Drupal module, Soft Purge is configured like so:

And as for Stale Content Options, you need to ensure that both "Stale while revalidate" and "Stale if error" are enabled. These are some basic tuning for the values as well:

If you are not using Drupal (or using the Acquia purge module), then you can always set the above HTTP headers manually. See the Fastly documentation on this.

How to use Fastly Soft Purge with the API only

If you don't run Drupal, or just want to execute a soft purge via the Fastly API, then this is simple as well, just pass in the Fastly-Soft-Purge HTTP header:

curl -sXPOST https://api.fastly.com/service/SERVICE_ID/purge/TAG \
 -H 'Fastly-Soft-Purge:1' \
 -H "Fastly-Key:$FASTLY_TOKEN" | jq
{
  "status": "ok",
  "id": "10427-1615460322-6"
}

Wait, I want to learn more

If you want to go deeper in Varnish (what powers Fastly), and how objects, TTL, stale and grace (stale-while-revalidate) are involved, please see their documentation.

There is also this excellent graphic to which explains how an object goes from fresh to stale

Gotchas

If you use Fastly shielding (which you probably should), then there is the potential for soft purged content on the edge PoP to end up being re-cached as fresh 😱. See the Fastly documentation on this. There is a VCL snippet to mitigate this:

if (fastly.ff.visits_this_service > 0) {
  set req.max_stale_while_revalidate = 0s;
}

If you are not already using Fastly shielding, and you care about caching, and cache hit rates, you should really start to look at this. The benefits far outweigh the tiny edge cases.

P.S. if you are not already using the Fastly purge auth snippet, get this installed ASAP. By not doing this, a bad actor is able to purge any single URL on your site, in an instant fashion. This could create a DoS.

# Fastly's URL purge feature allows you to purge individual URLs on your
# website. By default, authentication is not required to purge a URL with the
# Fastly API, but you can enable API token authentication with this snippet.
#
# @see https://docs.fastly.com/en/guides/authenticating-api-purge-requests
if ( req.request == "FASTLYPURGE" ) {
    set req.http.Fastly-Purge-Requires-Auth = "1";
}

Comments

If you have any other pro tips when it comes to soft purging in Fastly, please let me know.