No. And that question is the entire problem with this one.

This post is a reference for the PHP2Shell vulnerability: what actually broke inside PHP, which servers are exposed, how to patch each common setup, how to prove the fix is really running, and which quick fixes give you false comfort. Keep it open while you work through your servers.

What happened: the PHP2Shell vulnerability in plain terms

PHP2Shell is the nickname researchers gave to two flaws in the PHP engine itself, CVE-2026-109618 and CVE-2026-109619. PHP rates both 9.2 on CVSS v4.0, and both can let an unauthenticated attacker run code on a web server. The name is the researchers’, not the PHP project’s.

The dates matter here, because they decide if you were exposed and for how long:

• 20 October 2026: PHP announced an emergency release for two critical issues. Details leaked publicly the same day.
• 21 October 2026: fixed releases shipped: PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35, with a full advisory.
• 23 October 2026: PHP published an informational advisory for a third, much narrower issue, CVE-2026-115560, with a one-line mitigation and no emergency release.

The flaws were reported by Wiz Research, Calif and OpenAI’s Codex Security team, who used AI models to turn long-standing weaknesses into working exploits. The affected code goes back to PHP 5.4, so effectively every PHP version from the last decade is in scope.

The two bugs, without the jargon

CVE-2026-109618 (out-of-memory corruption). PHP enforces memory_limit by aborting a request the instant it crosses the limit. If that abort lands while PHP is still building internal request data, for example while parsing a multipart file upload, the data is left half-initialized. Request cleanup then treats it as valid and works with memory that can still hold attacker-controlled leftovers.

CVE-2026-109619 (stale global state). Some state owned by one request can survive after its memory has been freed, for example after a fatal error. The next request handled by the same worker can use that stale state before normal setup resets it.

Here is why this is worse than the usual plugin CVE. The bug lives in the runtime, not in your code. A site whose own code is flawless is still exposed, and updating WordPress, Laravel or Drupal changes nothing, because the PHP binary underneath comes from your OS packages, your host or your container image. The researchers also demonstrated reading memory from the PHP worker process, which can contain passwords, tokens and keys belonging to other people’s requests, without ever getting a shell.

Who is exposed, and why mod_php is the worst case

PHP’s advisory covers every installation that serves web requests: PHP-FPM, Apache with mod_php, php-cgi, FrankenPHP and LiteSpeed. The risk is not equal across them, though.

mod_php is the highest risk. Apache with mod_php runs request shutdown even when request startup already failed. That means simply running out of memory while PHP parses the request body is enough, and one unauthenticated request can trigger the attack before a single line of your application runs. PHP demonstrated this with default memory and input limits.

PHP-FPM is harder to hit, not safe. FPM shuts the worker down when a request fails early in setup, which blocks the cleanest path. Public reporting says one of the two bugs was still shown to reach code execution under FPM, through a path that depends on something the application does. Treat “we run FPM” as “we have slightly more time,” nothing more.

If you are not sure which mode you run, these two checks answer it on most Linux servers. The first lists Apache modules and shows a PHP module only if mod_php is loaded. The second lists running FPM services.

apachectl -M | grep -i php
systemctl list-units --type=service | grep -i fpm

How to fix PHP2Shell: it depends on where your PHP comes from

The fix is always the same in principle: run a patched PHP build. Getting there depends entirely on who builds your PHP. And there is a trap that runs in both directions. Linux distributions and long-term-support vendors received the patch early and backported it, so a patched package can carry an old-looking version like 8.3.6. Meanwhile, an unpatched process can keep running after the package updated. The version string answers neither question reliably.

Debian and Ubuntu packages

Debian, Ubuntu and Ondřej Surý’s widely used repository (packages.sury.org, also published as ppa:ondrej/php) all shipped fixed packages, including builds for some end-of-life branches. Update normally, then confirm the CVE is in the package changelog instead of reading the version number. Replace 8.3 with your branch.

sudo apt update && sudo apt upgrade
zgrep CVE-2026-109618 /usr/share/doc/php8.3-common/changelog.Debian.gz
sudo systemctl restart php8.3-fpm
sudo systemctl restart apache2

If zgrep prints a line, the fix is in the installed package. If it prints nothing, you are either on an older distro release that needs a paid extended-support channel, or on a repository that has not shipped the fix.

RHEL, AlmaLinux and Rocky Linux

Same idea with RPM. Update, then search the package changelog for the CVE. If you use Remi’s parallel packages, the package name follows its own pattern (for example php83-php-common), so query that name instead.

sudo dnf upgrade 'php*'
rpm -q --changelog php-common | grep CVE-2026-109618
sudo systemctl restart php-fpm httpd

One warning from PHP’s own advisory: the volunteer-run rpms.remirepo.net repository was not expected to have updates on release day, and PHP suggested switching to distribution packages or building from source if you depend on it. Check before assuming the update landed.

cPanel, CloudLinux and LiteSpeed

cPanel ships its own ea-php packages through EasyApache 4, and CloudLinux ships alt-php. CloudLinux backported the fix into supported and end-of-life branches, so most of its fixed builds keep version numbers below the php.net release. CloudLinux’s published procedure is to update both package families, then restart the services that run PHP so no worker keeps the old binary loaded.

yum clean all && yum update 'alt-php*' 'ea-php*' -y
/scripts/restartsrv_httpd

If your domains run PHP-FPM through cPanel, restart the FPM service as well. On LiteSpeed, restart the web server so every lsphp process respawns from the new build. Public analysis says LiteSpeed’s LSAPI behaves more like FPM than mod_php, which buys time but does not remove the need to patch.

DirectAdmin with CustomBuild

CustomBuild compiles PHP from source, so you get the fix when CustomBuild offers the patched release and you rebuild. Refresh the version list, confirm the fixed release appears for each PHP version you run, then build.

cd /usr/local/directadmin/custombuild
./build update
./build versions
./build update_versions

Read the ./build versions output before running the last command. If a branch you use still shows an unpatched release, apply the mitigations below for that branch until it updates. Compiling takes a while on small servers, so plan for it rather than starting it at peak traffic.

Docker and container images

A container never patches itself. If your image is built FROM an official php tag, rebuild with a fresh pull of the base image and redeploy. If your Dockerfile pins the base by digest, the digest has to change or nothing happens.

docker compose build --pull
docker compose up -d

Then run php -v inside the running container, not on the host, to confirm the version the app actually uses.

Managed hosting versus unmanaged VPS

On managed hosting, the host owns the PHP runtime, and several large hosts announced they had patched their fleets. Your job there is to get written confirmation, not to assume. On an unmanaged VPS from Contabo, Interserver or any similar provider, nobody patches PHP but you. That is the trade you accepted for the lower price and full root access, and this week is when it gets collected.

End-of-life PHP: 8.1 and older

The PHP project released no fixed builds below 8.2. Because of the severity, its security team published minimal source backports for PHP 7.4, 8.0 and 8.1 for people who build from source. Beyond that, your options are a distribution or vendor that backports (Ubuntu’s own PHP 8.1 on 22.04, the Surý repository, Freexian ELTS for older Debian, CloudLinux and TuxCare, HeroDevs), or upgrading. Note that PHP 8.2 itself reaches end of life on 31 December 2026, so if you are touching an 8.2 server now, plan the next upgrade at the same time.

Prove the patch is actually running

This is the failure mode that will bite most people. The package manager says “upgraded,” the ticket gets closed, and the old PHP code keeps serving traffic because the processes that loaded it were never restarted. Linux happily keeps running a binary whose file has been replaced on disk.

1. Confirm the fix is in the package with the changelog check above, not the version string.
2. Restart everything that runs PHP: every FPM pool, Apache (for mod_php), LiteSpeed, and any long-running PHP workers such as queue consumers managed by Supervisor or systemd.
3. Check the restart time is after the upgrade. systemctl status php8.3-fpm shows when the service started. If that is earlier than your package install, it is still running old code.
4. Ask the system what still needs a restart. On Debian and Ubuntu, sudo needrestart -r l lists services still using outdated files, without restarting anything.
5. Check the right binary. php -v on the command line reports the CLI build, which can differ from the FPM build serving your site. On Debian and Ubuntu, php-fpm8.3 -v asks the FPM binary directly.

Mitigations if you cannot patch today, and what they don’t do

PHP is blunt about this: only an update is a complete fix. The settings below reduce exposure to the specific attack paths it has confirmed. They do not make a server safe, and some of them have real costs. Use them to buy hours, not weeks.

Set them where applications cannot override them: php_admin_value or php_admin_flag in the FPM pool, or the Apache configuration for mod_php. Do not rely on ini_set() or an FPM .user.ini file, because those apply too late in the request. Here is the FPM pool version on a Debian-style path (adjust the file and branch to yours):

; /etc/php/8.3/fpm/pool.d/www.conf
php_admin_value[memory_limit] = -1
php_admin_value[post_max_size] = 8M
php_admin_value[upload_max_filesize] = 6M
php_admin_flag[session.upload_progress.enabled] = off

And the Apache equivalent for mod_php, inside the virtual host or a Directory block:

php_admin_value memory_limit -1
php_admin_value post_max_size 8M
php_admin_value upload_max_filesize 6M
php_admin_flag session.upload_progress.enabled off

Restart every worker afterwards. Now the reasoning behind each line, because each one protects a different path:

• memory_limit = -1 targets CVE-2026-109618. The confirmed exploit needs a predictable per-request limit to hit, so removing the limit removes the trigger. The cost: one request can now consume all available memory. A genuine allocation failure can also still reach the vulnerable code.
• Low post_max_size shrinks the request bodies an attacker can use. PHP suggests 8M where uploads are not needed and 32M where they are, with upload_max_filesize kept below it.
• session.upload_progress.enabled = off targets CVE-2026-109619. It blocks the confirmed session paths but not other variants. It does nothing at all for CVE-2026-109618.
• On PHP 8.5 only, fatal_error_backtraces = off blocks the confirmed fatal-backtrace path for CVE-2026-109619, and nothing else.

Because removing the memory limit opens a denial-of-service risk, PHP recommends a container or cgroup memory limit as the availability boundary, ideally one that kills only the affected worker. It specifically warns against RLIMIT_AS or RLIMIT_DATA, since those make allocations fail at the limit, which is exactly the condition the bug needs. Enforcing request-size, field-count and multipart-part limits at your reverse proxy helps too.

A web application firewall is a reasonable stopgap, not a fix. Cloudflare deployed rules before the patch shipped and reported attackers changing payloads to get around them. If your site already sits behind Cloudflare, those rules help. They do not replace the update.

The third issue: CVE-2026-115560

Two days after the release, PHP disclosed one more possible code-execution path. It needs an unusual setup: the application sets session.serialize_handler to php_binary or php_serialize in configuration, then switches it to php at runtime with ini_set() before calling session_start(), with session upload progress still enabled. The fixed releases above do not cover it. PHP’s advice is to keep session.upload_progress.enabled=0 until the next regular bugfix release. If you already set it for CVE-2026-109619, leave it in place and you are done.

After patching: assume someone tried

Amazon reported a honeypot hit against these flaws on 17 October, four days before disclosure, and exploitation attempts climbed fast once details were public. Most reports so far describe scanning and failed attempts rather than confirmed break-ins, but “nobody has reported it” is not evidence about your server. If you ran an unpatched, internet-facing PHP site during that window, do three things.

• Rotate secrets a PHP worker could have held in memory: database passwords, API keys, payment provider keys, SMTP credentials and, for WordPress, the salts in wp-config.php (which logs every user out, so warn people first).
• Read your logs for the patterns researchers flagged: unusual multipart requests followed by memory-limit errors or bursts of HTTP 500 responses, oddly large PHPSESSID cookies in responses, and PHP workers restarting with no explanation.
• Look for anything new and executable in your web roots and upload directories, especially PHP files that no deploy put there.

Troubleshooting

“php -v still shows the old version.” Either it is a distro backport (check the changelog for the CVE), or you are looking at the CLI binary while a different build serves the site. Check the FPM binary directly and the container, not the host.

“The site throws 500 errors after the update.” The fix changes how PHP handles running out of memory. Crossing memory_limit now raises a soft error at a safe point, and a second, higher threshold ends the process outright, without running shutdown handlers. A plugin or import job that used to squeak by can now fail differently. Check the error log, then review memory_limit and post_max_size against what the application genuinely needs.

“Cleanup or logging code stopped running on big jobs.” Same cause. If anything important lives in a shutdown function, it will not run when the hard limit is hit. Fix the memory usage rather than raising limits blindly.

“The package manager says there is nothing to update.” You are probably on a repository that has not shipped the fix, an end-of-life distro release, or an end-of-life PHP branch with no vendor backport. That is a migration problem, not a patching problem.

“I set the mitigation but the old value still shows.” Look for overrides in other pool files, per-directory configuration or application code, and make sure you used the admin variants and restarted the workers.

Common mistakes

• Updating WordPress, plugins or the framework and calling it done. None of that touches the PHP runtime.
• Trusting the version number in either direction, and declaring a backported package “vulnerable” or a stale process “patched.”
• Upgrading the package and never restarting FPM, Apache or long-running workers.
• Treating session.upload_progress.enabled=0 as the fix. It was circulated as a workaround before the release and only covers part of one CVE.
• Setting mitigations with ini_set(), .htaccess overrides that apps can change, or .user.ini, which take effect too late.
• Leaving memory_limit=-1 in place long after patching, with no cgroup boundary behind it.
• Forgetting the boxes nobody owns: the staging server, the old client site on PHP 7.4, the forgotten container in a side project.

Best practices

• Keep an inventory of every PHP install, its source (distro, panel, container, source build) and its branch. This week shows why.
• Verify fixes by CVE in the package changelog, and verify the running process by start time.
• Patch mod_php servers first, then public FPM sites, then internal tools.
• Once patched, restore a sane memory_limit, but keep session upload progress off until PHP’s next regular release covers CVE-2026-115560.
• Move servers off mod_php to PHP-FPM where you can. It would not have saved you here, but it narrowed the easiest path.
• Stay on a supported PHP branch, or pay for a vendor that backports. An unsupported branch turns every emergency into a migration.

FAQ

What is the PHP2Shell vulnerability?

It is the nickname for two memory-safety flaws in the PHP engine, CVE-2026-109618 and CVE-2026-109619, both rated 9.2 on CVSS v4.0. They can allow unauthenticated remote code execution on servers that run PHP for web requests.

Which PHP versions fix PHP2Shell?

PHP 8.5.12, 8.4.27, 8.3.36 and 8.2.35 from php.net. Distribution and vendor packages may carry the fix under different, older-looking version numbers, so check the package changelog for the CVE.

Is my WordPress site affected?

If it runs on an unpatched PHP build, yes, regardless of how up to date WordPress and its plugins are. The flaw is in PHP itself, so the fix comes from your server, host or container image.

Is disabling session.upload_progress enough?

No. It blocks the confirmed session paths for one of the two bugs and covers the narrow third issue, CVE-2026-115560. It does nothing for CVE-2026-109618. Only an update is a complete fix.

I’m on PHP 7.4 or 8.1. What now?

php.net has no release for you, but PHP published source backports for 7.4, 8.0 and 8.1, and several vendors and distributions shipped backported packages. Use one of those immediately, then plan the upgrade to a supported branch.

Do I need to restart anything after updating PHP?

Yes. Restart every PHP-FPM pool, Apache if you use mod_php, LiteSpeed, and long-running PHP workers. Until they restart, they keep running the vulnerable code already loaded in memory.

Should I rotate passwords and keys?

If an internet-facing server ran an unpatched build after details went public, it is a sensible precaution. Researchers showed worker memory could be read remotely, and that memory can hold credentials from other requests.

Conclusion

The PHP2Shell vulnerability lives below your application, so the only real fix is a patched PHP build that is actually running. Everything else in this post is either a way to get there or a way to buy time. If you remember one thing, make it this: verify the fix by CVE in the package, and verify the process by its restart time. The version string will mislead you in both directions.

Need help patching PHP2Shell across your servers?

If you have more PHP servers than certainty, I can help you close this out properly:

• Inventory every PHP install across your VPS, cPanel, DirectAdmin and container hosts, with branch and package source.
• Patch each one through the right channel and prove the fix by CVE and process restart time.
• Apply and later roll back the interim mitigations safely, with a memory boundary in place.
• Rebuild and redeploy PHP container images, including pinned-digest pipelines.
• Review logs for the published indicators and plan secret rotation without breaking production.
• Plan the move off end-of-life PHP branches and off mod_php.

Send me the output of php -v, your package changelog check, and how PHP runs on the server (FPM, mod_php or LiteSpeed), and I’ll tell you where you stand.

The post PHP2Shell Vulnerability: What Happened, How to Patch, and How to Prove It Worked appeared first on John Nessime.

Moving the files is the solved part. Which bucket they land in, and what sits in front of that bucket, decides whether you spend almost nothing or spend more than the server you were trying to shrink.

Cloudflare R2 vs Amazon S3 usually gets reduced to one sentence: R2 does not charge for data transfer out, S3 does, therefore R2. That is wrong in both directions, and understanding why is most of the value here. This post covers the failure mode that shows up after the migration rather than during it, an honest profile of each service, the levers that actually move the number, and a procedure you can run against your own traffic in an afternoon.

Free egress is not free serving

R2 charges nothing to move bytes out, on any storage class. It still charges for the read. Every GetObject and every HeadObject is a Class B operation, and a media library made of thousands of small files racks those up fast. Cloudflare’s own asset hosting example in the R2 pricing docs makes the point: storage falls inside the free tier, writes fall inside the free tier, and the whole bill is read operations.

So the meter is not gigabytes. The meter is requests that reach the bucket, which means the variable deciding your bill on either platform is cache hit ratio.

This is where offloading setups go wrong without anyone noticing. The plugin rewrites image URLs to point at the bucket, images render, pages look fine, and every request for every thumbnail travels all the way to the origin and gets billed. Nothing breaks. Nothing alerts. The number just grows with traffic.

There is a sharper version on the R2 side. A new bucket gives you a public URL on an r2.dev subdomain, which is convenient and tempting. Cloudflare documents it as a testing endpoint with a variable rate limit. Push production traffic through it and requests get throttled with 429 Too Many Requests, throughput can be throttled too, and you get no cache, no WAF, no bot management. Those only exist once the bucket sits behind a custom domain you control. That is the invisible failure: not a broken image, a working setup billed on every request and rate limited under load.

Cloudflare R2: where it wins and where it hurts

Where R2 wins

• Egress really is zero. Not zero within a partner network, not zero up to a ratio of stored data. Zero, on both storage classes, through the S3 API and through Workers.
• A custom domain puts it behind the CDN. One managed CNAME and reads start being absorbed at the edge instead of hitting the bucket. No second product to buy and wire up.
• Deletes are free operations, along with aborting a multipart upload. Cleaning up a messy library costs nothing.
• Unauthorized requests are not billed. A caller without permission gets a 401 and you are not charged, which matters once someone starts hammering your bucket path.
• Migration tooling is free to use. Super Slurper copies a bucket across in bulk, Sippy migrates lazily on first miss. You pay only for the operations they perform against R2.

Where R2 hurts

• S3 compatible is not S3 identical. Cloudflare publishes a table of which operations are implemented and which are not. Real tooling has broken on the gaps: when AWS shipped SDKs that enabled CRC32 checksums by default, R2 rejected the header until the mismatch was resolved.
• There is no cold archive tier. Infrequent Access is as cold as it gets, and it carries a minimum storage duration, a retrieval fee, and doubled operation rates. For an active library that is the wrong tier, not a saving.
• Data lives in one primary location. Location hints are best effort rather than a region guarantee, and are honored only the first time a bucket with that name is created. Read performance away from that location comes from the edge cache, not from replicas.
• Plugin support is thinner. WP Offload Media, the long-standing WordPress option, officially lists Amazon S3, DigitalOcean Spaces, and Google Cloud Storage. R2 users generally land on Media Cloud, Advanced Media Offloader, Next3 Offload, or the S3-Uploads route driven from WP-CLI.

Amazon S3: where it wins and where it hurts

Where S3 wins

• Origin transfer to CloudFront is not billed. This quietly demolishes the usual comparison. AWS waives data transfer from an AWS origin to CloudFront, so a properly built S3 media stack never pays the direct-to-internet rate on cached traffic.
• The CloudFront free allowance is permanent, covering a monthly volume of data transfer out and requests rather than expiring after twelve months. Small sites can sit inside it indefinitely.
• Event driven derivatives are mature. Upload fires a Lambda that generates thumbnails, strips metadata, converts formats, writes the result back. Hard to beat if your media pipeline does real work.
• A full storage ladder with real lifecycle transitions, down to deep archive. If a large chunk of your library is genuinely cold, S3 has somewhere cheap to park it.
• Governance depth. Object Lock, versioning, replication, IAM condition keys, CloudTrail data events. When an auditor asks for a control, S3 usually has a named feature for it.

Where S3 hurts

• The default configuration is the expensive one. Point image URLs at the bucket endpoint and every byte leaves at the internet rate with nothing cached in front of it. The cheap path has to be built deliberately.
• Four meters instead of two: storage, bucket requests, CDN requests, CDN data transfer, plus per-feature charges. Forecasting takes real effort.
• Geography changes the rate. Delivery costs more per gigabyte to some regions than others, so an audience shift moves your bill without you changing anything.
• Leaving costs money. Copying a large library out is billed transfer, once, in a lump. Usually small against ongoing savings, but it is a real number somebody has to approve.

How to decide between Cloudflare R2 vs Amazon S3

Four levers move the outcome. Everything else is noise.

Cache hit ratio

The dominant term on both platforms, and the one people skip. At a high hit ratio the origin barely gets touched and both bills collapse toward the storage line. At a low hit ratio, R2 bills read operations and S3 bills origin GETs, and both climb with traffic. Media should be trivially cacheable because the files never change. If yours is not caching, that is a configuration problem worth fixing before you pick a vendor, because fixing it changes the answer.

Object count, not library size

WordPress does not store one file per upload. It stores the original plus every registered image size, and themes and page builders happily register more. Ten thousand uploads can easily be sixty thousand objects, each one a write on migration and a read on a cache miss. Count them before you model anything:

cd /path/to/wordpress

# Objects you are about to create in the bucket
find wp-content/uploads -type f | wc -l

# Bytes
du -sh wp-content/uploads

# How much of that count is derivative sizes rather than originals
find wp-content/uploads -type f -regextype posix-extended 
  -regex '.*-[0-9]+x[0-9]+.(jpe?g|png|webp|avif)$' | wc -l

That last number is usually the surprise. If most of your object count is thumbnails, trimming unused registered sizes before migrating is the cheapest optimization available, and it shrinks writes, reads, and storage at once.

How cold the library actually is

Most libraries are a long tail. A small set of recent files takes nearly all the traffic while the rest sits untouched for years. If that tail is large, S3 has somewhere genuinely cheap to put it and R2 does not. Be careful with R2’s Infrequent Access class here: it lowers the storage rate but raises both operation rates, adds a retrieval fee, and enforces a minimum duration whether you keep the object or not. It fits write-once, read-almost-never data, not a library anything still links to.

What the rest of your stack already is

If your application already runs in AWS and generates derivatives with Lambda on upload, moving to R2 means rebuilding that pipeline against Workers or R2 event notifications. That is real work you do not get paid for. If your site is a VPS from somewhere like InterServer or Contabo with Cloudflare already in front of it, R2 is nearly free work: the DNS is there, the cache is there, and the bucket slots in behind a subdomain you already control.

Setup details that change the answer

A few configuration choices matter more than the vendor choice. Get these wrong and the cheaper platform produces the bigger bill.

Serve from a custom domain, always

On R2 this is the difference between a cached, protected asset host and a throttled test endpoint. Attach one from the dashboard or from Wrangler:

npx wrangler r2 bucket domain add my-media-bucket 
  --domain=cdn.example.com 
  --zone-id=<YOUR_ZONE_ID>

npx wrangler r2 bucket domain list my-media-bucket

Then disable public access on the r2.dev subdomain, or you have left a second uncached, unprotected door into the same objects.

Verify the cache instead of assuming it

Assumptions about caching are where the money leaks. Ask the edge directly: request the same object twice and read the headers.

# Cloudflare in front of R2
curl -sI https://cdn.example.com/2024/07/photo-1024x768.jpg 
  | grep -iE 'cf-cache-status|cache-control|age'

# CloudFront in front of S3
curl -sI https://cdn.example.com/2024/07/photo-1024x768.jpg 
  | grep -iE 'x-cache|cache-control|age'

First request, expect a miss. Second, expect a hit. If the second still reports a miss, or reports a status meaning the response was never eligible for caching, every image view is costing you an origin read. Fix that before you compare anything.

Long cache lifetimes and versioned filenames

Uploaded media is immutable in practice. Nobody edits the bytes of photo-1024x768.jpg, they upload a new file. So cache lifetimes should be long and the object name should change when the content does. If you find yourself purging image caches regularly, the real problem is that your filenames are not versioned.

On the Cloudflare side, Smart Tiered Cache is worth enabling for R2 origins. It routes edge misses through an upper tier data center close to your bucket instead of letting every edge location fetch independently, which cuts the number of requests reaching R2 at all.

Clean up failed uploads

Large uploads use multipart. Interrupted ones leave orphaned parts that occupy billed storage and do not show up in a normal object listing. Both platforms support lifecycle rules for this, and on R2 you can manage them from Wrangler:

npx wrangler r2 bucket lifecycle list my-media-bucket

npx wrangler r2 bucket lifecycle add my-media-bucket 
  --name=expire-temp 
  --prefix=tmp/ 
  --expire-days=30

Configure an abort rule for incomplete multipart uploads on day one, whichever platform you land on. It is the most common source of storage you pay for and cannot see.

A decision procedure you can run this afternoon

1. Measure your current cache hit ratio for image paths, from analytics or a sampled read of access logs. Everything downstream depends on this number.
2. Count objects, not gigabytes. Run the find commands above and separate originals from derivative sizes.
3. Estimate monthly origin reads as total image requests multiplied by the miss rate. That, not your bandwidth, is what you feed into either pricing calculator.
4. Treat writes as a one-off migration spike plus a modest steady rate. Migration is usually the largest write event the bucket ever sees.
5. Price both against current published rates on the same day, using Cloudflare’s R2 calculator and the AWS pricing calculator. Rates move. Do not trust a number copied from a blog post, including this one.
6. Test with a prefix before committing. Offload one year of uploads, point the site at it, watch cache status headers and the operations dashboard for a week. A week of real traffic beats any spreadsheet.

If it comes out close, take R2. Not because it is cheaper in that scenario, but because it has fewer meters to reason about, and a simpler operational model is worth something on the day something breaks.

Arguments that don’t survive contact

“S3 charges egress, R2 doesn’t, so R2 always wins”

Only true against the worst possible S3 setup. Transfer from an S3 bucket to CloudFront is not billed, so a properly built S3 media stack pays CDN delivery rates on cache misses, not the direct-from-bucket rate. Add CloudFront’s permanent free allowance and a small site pays nothing on either platform. The gap opens at scale and on origin-heavy traffic, not on the first terabyte.

“R2 is a CDN”

R2 is object storage that can sit behind a CDN you already have. The caching, the WAF, the bot rules all come from the Cloudflare zone, and none of it applies to the development URL. If the bucket is not behind a custom domain on a zone you control, you have storage without delivery.

“S3 compatible means drop-in”

It means most tools work with a changed endpoint and a region value of auto. It does not mean every operation, header, and checksum behaves identically. Budget an afternoon for fighting an SDK default, and test uploads, multipart uploads, deletes, and signed URLs against the real bucket before cutting over.

“We’ll save money moving cold files to Infrequent Access”

Sometimes. The lower storage rate arrives with higher operation rates, a retrieval fee, and a minimum duration. If files are cold enough to justify that, they are probably cold enough to belong in a real archive tier, which R2 does not have. Model it against your read pattern rather than assuming a cheaper per-gigabyte number is a cheaper bill.

Frequently asked questions

Is Cloudflare R2 cheaper than S3 for a WordPress media library?

Usually, once traffic is meaningful and the library is served publicly. For a small site behind a well configured CDN, both can land near zero. R2’s advantage grows with egress volume and with the number of requests that miss cache, which is why measuring hit ratio comes first.

Can I use the same WordPress plugin for both?

Several plugins speak the S3 API and support both, including Media Cloud, Advanced Media Offloader, and Next3 Offload. WP Offload Media officially lists Amazon S3, DigitalOcean Spaces, and Google Cloud Storage, so check current provider support before assuming a swap is free. Whichever you pick, confirm it rewrites existing URLs and not only new uploads.

How do I migrate an existing S3 bucket to R2 without downtime?

Cloudflare offers two paths. Super Slurper copies everything in bulk. Sippy migrates incrementally, pulling an object from the source the first time it is requested and serving from R2 afterward. Both are free to use, you pay for the operations they perform against R2, and your source bucket may charge you for the reads.

Do I still need a CDN in front of R2?

You need a custom domain on a Cloudflare zone, which is what puts the CDN in front of it. That is not optional for production. The development URL is rate limited by design and gets no cache, WAF, or bot management.

Can I keep S3 and just put Cloudflare in front of it?

You can, and it does cut origin reads. Be aware that cache misses then pull from S3 across the public internet, billed as ordinary S3 egress, unlike an S3 origin sitting behind CloudFront. Reasonable as an interim step, rarely the cheapest end state.

What about Backblaze B2, Wasabi, or DigitalOcean Spaces?

All viable and worth pricing if you are already modelling. B2 competes on raw storage, Wasabi sells predictability, Spaces is convenient when your droplets are already there. The analysis transfers directly: find the meters, find your cache hit ratio, multiply.

How do I monitor the bill after migrating?

Watch operations and cache hit ratio, not storage. Storage grows slowly and predictably. Operations track traffic and configuration mistakes, which is where the unpleasant surprises live. A Grafana Cloud dashboard or a scheduled pull from the provider’s usage API is enough; the point is that somebody looks weekly rather than at invoice time.

Conclusion

If you take one thing from this Cloudflare R2 vs Amazon S3 comparison, take this: the vendor choice is second order. What decides your media offloading bill is how many requests reach the bucket, and that is a function of your cache configuration, your object count, and whether you put a real domain in front of the storage.

Fix the cache first. Count the objects. Then price both, on the same day, against your own numbers. Most of the time R2 comes out ahead for public media, S3 comes out ahead when the library is entangled with AWS services or needs a genuine archive tier, and the difference is smaller than the internet suggests.

Need help moving a media library without breaking it?

Media offloading looks like a plugin install and turns into a URL rewriting, caching, and permissions problem. I work with teams on the parts that are easy to get subtly wrong:

• Modelling R2 against S3 using your real object counts, request volumes, and cache hit ratio instead of a generic calculator
• Planning and running the migration: bulk or incremental copy, URL rewriting, and a rollback path if the cutover misbehaves
• Custom domain, cache rule, and header configuration so origin reads collapse instead of tracking your traffic
• Bucket permissions, CORS, signed URL flows for private downloads, and lifecycle rules for orphaned multipart uploads
• Trimming registered image sizes and derivative sprawl before migration so you stop paying to store thumbnails nothing links to
• Dashboards for operations, cache hit ratio, and storage growth so the bill stops being a monthly surprise

Send me a curl -I of one of your image URLs and a rough object count, and I can usually tell you quickly whether you have a pricing problem or a caching problem.

The post Cloudflare R2 vs Amazon S3 for Media Offloading: Where the Bill Actually Comes From appeared first on John Nessime.

That asymmetry, sending works and receiving doesn’t, is the single most useful clue in the whole problem. It narrows the search enormously. Yet most guides on Zoho Mail not receiving email open by telling you to check your SPF record, which has nothing whatsoever to do with inbound delivery. People lose a full day tuning an SPF string while their MX records quietly point at a data center that has never heard of their domain.

This guide walks the inbound path in the order a sending mail server actually walks it, groups the real causes by failure family, and gives you the exact commands to prove which one you’re looking at. Everything here applies whether your DNS lives at Cloudflare, Namecheap, GoDaddy, Route 53, or a cPanel box at InterServer or Contabo.

Sending and receiving are two separate systems

This is worth internalizing before you touch a single DNS record, because it eliminates most of the internet’s advice on this topic in one move.

When you send from Zoho, your client authenticates against Zoho’s outbound SMTP server with a username and password. Your domain’s DNS plays no part in that handshake. SPF, DKIM and DMARC records exist so that the recipient’s server can decide whether to trust mail claiming to be from you. They are judgements other people make about your outbound mail.

When someone sends to you, none of that matters. Their server looks up your MX records, connects to whatever hostname it finds, and hands the message over. Your SPF record is never consulted. Your DKIM key is never consulted.

So if outbound works and inbound doesn’t, the fault is in the MX lookup, the connection that follows it, or Zoho’s decision about what to do with the message once it arrives. Not SPF. There is one narrow exception, covered later, where Zoho’s own inbound spam checks reject mail because the sender’s SPF fails. That is a different record on a different domain.

The inbound path, in the order it actually happens

Six things happen between someone hitting send and a message appearing in your Zoho inbox:

1. The sending server splits the recipient address and takes the domain part.
2. It asks DNS for the MX records of that domain, following delegation from the root down to whichever nameservers the registry says are authoritative for you.
3. It sorts the results by preference number and takes the lowest one first.
4. It resolves that hostname to an address record.
5. It opens an SMTP connection and offers the message with MAIL FROM and RCPT TO.
6. The receiving server decides whether that recipient exists in its organization and where to file the message.

Steps two through four are pure DNS. Step five is network. Step six is Zoho configuration. Diagnosing this well is mostly a matter of finding out which step you’re stuck on instead of guessing.

Five DNS failure families behind Zoho Mail not receiving email

Nearly every inbound failure I’ve seen falls into one of these. They present almost identically from the user’s chair, which is why guessing goes badly.

1. You’re editing records at a provider that isn’t authoritative

This is the most common one and the most frustrating, because the records look perfect in the control panel you’re staring at. They’re just not the records the internet reads.

It happens whenever the domain is registered in one place and the nameservers point somewhere else. You buy at Namecheap, delegate to Cloudflare, then add MX records back in the Namecheap DNS panel out of habit. Or a previous developer moved DNS to a hosting account you no longer log into. The registrar panel accepts your edits and stores them faithfully. Nobody queries them.

Find out where authority actually sits:

# Which nameservers does the parent zone delegate to?
dig +short NS example.com

# What do those nameservers say about your MX, directly?
dig MX example.com @ns1.yourprovider.com +norecurse

# Watch the full delegation chain from the root
dig +trace MX example.com

The +trace output is the one to read carefully. It shows each referral step, so you can see exactly which nameserver set the world is being pointed at. If those names don’t match the provider where you’ve been editing, you’ve found your problem and the fix takes thirty seconds.

2. Correct-looking MX records for the wrong Zoho data center

This is the failure mode that stays invisible longest, and it’s the reason copying MX values off a random blog post is dangerous.

Zoho runs regional data centers, and your account lives in exactly one of them. The mail hostnames differ by region: the international service uses .com hostnames, and the European and Indian deployments use their own regional equivalents. Your domain’s mailboxes exist only in the data center where your organization was created.

Point your MX at the wrong region and everything looks healthy. The records resolve. The hostnames are genuine Zoho servers. A lookup tool will show green. But mail arrives at infrastructure that has no record of your domain, and senders get a rejection saying the address doesn’t exist, or nothing at all.

For the international service, Zoho’s documentation specifies three records, in this order of preference:

Priority 10   mx.zoho.com
Priority 20   mx2.zoho.com
Priority 50   mx3.zoho.com

Do not take that as gospel for your account. The authoritative source is the DNS Mapping section of your own Zoho Mail Admin Console, which shows the exact hostnames for the data center your organization sits in. Read them there, copy them from there. If you’re unsure which region you’re on, the domain in the URL when you log into the admin console tells you.

Verify what the world sees, not what your panel shows:

dig +short MX example.com

# Confirm the target hostname itself resolves
dig +short mx.zoho.com

3. Leftover MX records from whatever came before

MX preference is not a fallback list in the sense most people assume. A sending server tries the lowest number first, and if that server accepts the message, the transaction is over. It never touches the others.

So a stale record from your old host sitting at priority 0 or 5 doesn’t compete with Zoho. It wins outright, every time. The mail is delivered successfully, to a mailbox nobody checks, on a server that may still be quietly accepting for your domain. The sender sees no error. You see nothing at all.

This is also why partial migrations bite. Two providers both accepting mail for one domain means your messages split between them based on preference and retry timing, which looks like random message loss and is nearly impossible to reason about from the inside.

The rule is simple: after cutover, the only MX records in the zone should be Zoho’s. Delete everything else, including records for subdomains you forgot about. Check the full answer rather than a truncated view:

dig MX example.com +noall +answer

4. A CNAME sitting where the MX records live

Two related problems here, and both are structural rather than typos.

The first: an MX record must point to a hostname with an address record. Pointing an MX at a name that is itself a CNAME is forbidden by the DNS specifications, and Zoho documents this explicitly. Some resolvers cope, others don’t, and the ones that don’t will fail intermittently in ways that look like a network problem.

The second: a CNAME at the zone apex. Classic DNS forbids a CNAME coexisting with any other record at the same name, and MX records at the apex are exactly that. Providers like Cloudflare work around this with CNAME flattening, which synthesizes an address record and does keep your MX records functional. Providers with a naive implementation may shadow or drop them. If someone added an apex CNAME to point the bare domain at a hosting platform or a site builder, that’s worth checking early.

dig CNAME example.com +short
dig A example.com +short

One thing that is not a problem, despite frequent claims otherwise: Cloudflare’s orange-cloud proxy setting. Proxying applies to HTTP traffic on address records. It has no effect on MX records, and toggling it grey will not fix inbound mail.

5. DNSSEC left broken after a nameserver change

Less common, but spectacular when it happens, because it takes the entire domain down rather than just mail.

If DNSSEC was enabled and the DS record at the registry no longer matches the signing keys at your current DNS provider, validating resolvers refuse to return any answer at all. They return SERVFAIL. Non-validating resolvers still work, which is why the site loads fine for you and mail from major providers, who mostly validate, vanishes.

The test is a one-liner. +cd disables validation checking:

# Fails with SERVFAIL if validation is broken
dig MX example.com

# Same query, validation bypassed
dig +cd MX example.com

If the second command returns your records and the first doesn’t, you have a DNSSEC mismatch. The fix is at your registrar: either update the DS record to match your current provider’s keys, or remove DNSSEC entirely, confirm mail flows, and re-enable it properly afterwards.

When DNS is clean and mail still doesn’t arrive

If your MX records resolve correctly to the right data center and nothing else is in the zone, DNS has done its job. The remaining causes sit on either side of it.

Your own server is intercepting the mail

This catches people running a site on cPanel or DirectAdmin, and it’s genuinely counterintuitive. Control panels have an email routing setting per domain, typically offering local, remote, or automatic delivery. If it’s set to local mail exchanger, the mail server on that box delivers messages for your domain into its own accounts and never performs an MX lookup.

The effect is narrow but confusing: external senders reach Zoho normally, while anything sent from a script or a form on that server goes to a local mailbox instead. Contact form submissions disappear while ordinary email works. Set routing to remote mail exchanger on any host where the site lives but the mailboxes don’t.

The address doesn’t exist in your Zoho organization

Zoho will only accept mail for addresses it knows about. A user account, an alias on a user, or a group. If someone hands out an address that was never created, Zoho rejects it and the sender gets an address-not-found bounce that they may not think to forward to you.

Zoho supports a catch-all address for exactly this, and it’s worth setting one during a migration so misaddressed mail lands somewhere recoverable rather than bouncing into the void.

Zoho accepted it and quarantined it

Under Security and Compliance in the Admin Console, Zoho lets you decide what happens to mail that fails SPF, DKIM, DMARC or DNSBL checks. The available actions include delivering it, quarantining it, or rejecting it outright, with or without a bounce.

Set to reject-without-bounce, a sender whose own SPF is slightly wrong gets silently dropped, and neither party learns anything. This is the narrow SPF connection I mentioned earlier: it’s the sender’s SPF being judged, not yours. Check the incoming quarantine before concluding mail never arrived, and check the organization blocked list while you’re there, since a blocked domain or TLD produces exactly the same silence.

The domain lapsed or lost verification

If a domain expires, its DNS stops resolving and every associated record becomes unreachable, which also invalidates the domain verification Zoho performed. Zoho’s documentation notes that after renewal you have to reconfigure MX records as part of reverification. Renewing the domain alone doesn’t restore mail.

A troubleshooting sequence that ends in an answer

Work these in order. Each step either clears a layer or hands you the cause, so you never end up changing three things at once and losing track of which one worked.

1. Confirm delegation. Run dig +short NS example.com and check it matches the provider you’re editing. If not, stop here and fix that first.
2. Read the MX records from an authoritative nameserver. Use dig MX example.com @ns1.yourprovider.com so you’re seeing the source of truth rather than a cached copy.
3. Compare against the Admin Console. Open the DNS Mapping page in Zoho Mail and match hostnames character for character, including the regional suffix.
4. Confirm nothing else is in the answer. Any non-Zoho MX record is a suspect, especially one with a lower preference number.
5. Test resolution from a public resolver. Try dig MX example.com @1.1.1.1 and @8.8.8.8. Differences between them mean you’re mid-propagation and waiting is the fix.
6. Rule out DNSSEC. Compare dig MX example.com against dig +cd MX example.com.
7. Prove the SMTP conversation. Send a real test and watch what the server says.
8. Check quarantine and blocked lists in the Zoho Admin Console before concluding anything was lost.

For step seven, swaks is the cleanest tool. It speaks SMTP for you and prints the full exchange, so you see the actual response code rather than guessing from a bounce message:

# Stop after RCPT TO so no message is actually delivered
swaks --to you@example.com --server mx.zoho.com --quit-after RCPT

A 250 after RCPT TO means Zoho recognises the address and would accept the mail. A 550 means it doesn’t know that recipient, which points straight at a missing mailbox or the wrong data center. A connection timeout usually isn’t Zoho at all: many residential and cloud providers block outbound port 25 by default, so run this from a machine you know has egress on 25. If you only have a laptop on a consumer connection, testing through a VPN endpoint that permits SMTP, or from a small VPS, avoids chasing a phantom.

Common mistakes

• Fixing SPF to solve an inbound problem. Your SPF record has no role in receiving mail. If you can send but not receive, it is not SPF.
• Copying MX values from a tutorial. Regional deployments have different hostnames. Use your own Admin Console every time.
• Adding Zoho’s records without removing the old ones. A lower preference number beats Zoho outright and nothing warns you.
• Trusting the DNS panel over a query. A saved record and a published record are different things. Query it.
• Cutting MX over before creating the mailboxes. Every address that doesn’t exist yet bounces, and those messages are gone.
• Blaming propagation on day three. Propagation is bounded by the TTL that was in effect before the change. If it’s been longer than that, something is genuinely wrong.
• Trailing dot confusion. Some panels want mx.zoho.com. with the dot, others append the zone themselves and turn mx.zoho.com into an unresolvable name. Always confirm with a query afterwards.

Best practices

• Drop the TTL before you migrate. Lower the MX TTL to a few minutes a day ahead of cutover, do the change, then raise it back. This turns a rollback from an overnight wait into a coffee break.
• Create every mailbox, alias and group before touching MX. Zoho’s own guidance is to configure users first and change MX last.
• Set a catch-all during the transition. It converts “we lost an email” into “it’s in the catch-all mailbox”.
• Keep DNS in one place. One authoritative provider, one panel, no ambiguity about where records live.
• Monitor the MX record itself. An uptime or synthetic monitoring service that checks DNS answers, or a scheduled dig comparison in your own pipeline, catches an accidental zone edit before a client does.
• Enable DKIM and set DMARC after inbound is confirmed working. They matter for your outbound reputation. Doing them first just adds variables while you’re debugging.
• Keep a copy of the zone. Export it before major changes. Reconstructing a zone from memory during an outage is miserable.

Frequently asked questions

Why can I send from Zoho Mail but not receive?

Because sending authenticates directly against Zoho’s outbound servers and doesn’t consult your domain’s DNS, while receiving depends entirely on your MX records. Working outbound with broken inbound is a strong signal that the problem is an MX record, a nameserver delegation issue, or a leftover record from a previous provider.

How long should MX changes take to work?

Propagation is governed by the TTL that was set on the old record, not by any fixed schedule. If the previous TTL was one hour, resolvers holding a cached copy will refresh within an hour. If it was set to a day, expect a day. Nothing you do afterwards speeds up a cache that has already been populated.

Do I need all three Zoho MX records?

Mail will flow with only the primary, but the backups exist so that senders have somewhere to deliver if the primary is unreachable. Without them, sending servers queue and retry, which delays delivery and eventually bounces. Add the full set your Admin Console lists.

Does SPF affect whether I receive email?

Your own SPF record does not. It’s evaluated by servers receiving mail from you. The one indirect connection is that Zoho evaluates the sender’s SPF on incoming mail, and if your Admin Console is set to reject or quarantine on SPF failure, legitimate mail from badly configured senders can disappear.

Why do lookup tools say my MX records are fine when mail still fails?

Lookup tools verify that records exist and resolve. They can’t know which Zoho data center your organization lives in, whether the mailbox has been created, or whether a message is sitting in quarantine. A green result rules out one layer, not all of them.

Can I run Zoho Mail alongside another mail provider on one domain?

Not by simply listing both sets of MX records. That splits mail unpredictably. Zoho does offer routing arrangements designed for gradual migration, configured on the Zoho side rather than by stacking MX records in DNS. If you need a genuine split, use the supported mechanism.

My contact form emails vanish but normal email works. Why?

Almost always the control panel email routing setting on the server hosting the site. If it’s on local mail exchanger, mail generated on that machine is delivered into a local account rather than sent out to Zoho. Switch it to remote.

Conclusion

If you take one thing from this: when Zoho Mail is not receiving email but sending works, stop looking at authentication records. Inbound mail is decided by your MX records, and those records have to be published by the nameservers that are actually authoritative for your domain, pointing at the data center your Zoho organization lives in, with nothing else in the answer competing with them.

Query, don’t assume. Read the answer from the authoritative nameserver rather than the panel, compare it against your own Admin Console instead of a tutorial, then work down through SMTP and Zoho’s own quarantine. Almost every case resolves in that order, and the ones that don’t are usually a mailbox that was never created in the first place.

Need someone to untangle your mail DNS?

I work on DNS and mail delivery problems like this regularly, usually for people who have already tried three fixes and made things harder to reason about. Things I can help with:

• Diagnosing inbound mail failures end to end, from delegation through SMTP response codes to Zoho’s quarantine
• Planning and executing a mail migration to Zoho with a TTL schedule and a rollback path that actually works
• Auditing and cleaning up a zone that several people have edited over the years
• Setting up SPF, DKIM and DMARC properly once inbound is stable, including staged DMARC enforcement
• Fixing DNSSEC mismatches after a nameserver move without taking the domain down again
• Adding DNS and mail-flow monitoring so the next zone edit doesn’t go unnoticed for a week

Send me the output of dig +trace MX yourdomain.com and a rejection message from a sender, and I can usually tell you where it’s breaking before we’ve agreed on anything.

The post Zoho Mail Not Receiving Email? A DNS Troubleshooting Guide appeared first on John Nessime.

The fix looked obvious. Drop in a limit_req_zone, apply it, reload, done. Except the reload changed nothing at all, and the reason took an embarrassingly long time to find.

This post covers Nginx rate limiting and bot mitigation the way it actually behaves in front of a real site: how the limiting modules key requests, what burst and nodelay really do to your traffic shape, why a limit that looks correct in the config can be doing nothing (or throttling everyone at once), and how to layer bot handling so you are not playing whack-a-mole with user agent strings forever.

The failure that hides: your limit is keyed on the wrong address

This is the one that bites hardest, because the config parses, nginx -t passes, and the error log stays quiet. Nothing tells you it is wrong.

If anything sits in front of Nginx (Cloudflare, another CDN, a load balancer, or a local Varnish tier), then $remote_addr is the address of that proxy, not the visitor. Key a rate limit on it and you get one of two outcomes, both bad:

• Every visitor arriving through a given edge node shares a single bucket. One aggressive client burns the budget and legitimate users start collecting 503s.
• Or your rate is generous enough that the shared bucket never fills, in which case the limit is decorative and the abusive client sails straight through.

The real IP module fixes this, but only if you trust the right sources. It rewrites $remote_addr and $binary_remote_addr from a header, and only for connections coming from addresses you have explicitly declared trustworthy.

# http context. One entry per CDN range you actually receive traffic from.
set_real_ip_from 203.0.113.0/24;
set_real_ip_from 2001:db8::/32;

real_ip_header CF-Connecting-IP;   # or X-Forwarded-For for a generic proxy
real_ip_recursive on;

Two things matter here more than the syntax.

First, never widen set_real_ip_from to 0.0.0.0/0. That tells Nginx to believe a client-supplied header from anyone, which means an attacker can pick a new “client IP” on every request and your per-IP limit becomes unlimited. It also poisons your logs, your deny rules and anything downstream that trusts the forwarded address.

Second, after the real IP module has run, $binary_remote_addr holds the visitor address and $realip_remote_addr holds the original connecting address (the proxy). Key your limits on $binary_remote_addr. Keying on $realip_remote_addr puts you right back in the shared-bucket failure, silently.

The proxy ranges change over time, so treat the trust list as a file that gets regenerated on a schedule and reloaded, not something you paste in once. On a self-managed VPS from a provider like Contabo or InterServer, a small cron job that fetches the current ranges, rewrites an include file, runs nginx -t, and only reloads on success is about twenty lines of shell and saves you from a very confusing afternoon later.

What limit_req actually does, and what burst really means

Nginx rate limiting for requests uses a leaky bucket. The zone stores, per key, the current number of excessive requests. The rate defines how fast that excess drains. Two directives do the work: limit_req_zone declares the bucket in the http context, limit_req applies it in http, server or location.

http {
    limit_req_zone $binary_remote_addr zone=general:10m rate=10r/s;
    limit_req_zone $binary_remote_addr zone=login:10m   rate=20r/m;

    limit_req_status 429;
    limit_req_log_level warn;
}

The rate is expressed in requests per second (r/s) or requests per minute (r/m). If you want less than one request per second, use r/m. There is no native per-hour or per-day unit in the stock module, which surprises people writing password-reset limits.

burst without nodelay is a queue, not a rejection

This is the second-most common misunderstanding after the keying problem. By default, excessive requests are delayed so that they come out at the configured rate. They are only rejected once the number waiting exceeds burst.

location /search/ {
    limit_req zone=general burst=20;
}

At 10r/s, a client that fires 21 requests at once gets one served immediately, nineteen held and released over roughly two seconds, and the last one rejected. Those nineteen held requests are still open connections on your server. You have not reduced concurrency, you have converted a spike into a slow trickle and paid for it in worker slots and file descriptors.

That is genuinely useful for smoothing traffic to a fragile upstream. It is a bad idea for abuse control, where holding the connection is exactly what the attacker wants.

nodelay, delay, and picking between them

nodelay changes the behaviour to: serve everything in the burst window immediately, reject everything past it. The bucket still drains at the configured rate, so a client that sustains the burst rate will hit the wall and stay there.

location /wp-login.php {
    limit_req zone=login burst=5 nodelay;
    limit_req_status 429;
    # ...fastcgi_pass or proxy_pass here
}

The delay parameter is the middle ground: it sets the point within the burst at which requests start being delayed rather than served immediately. burst=20 delay=10 means the first ten excess requests go straight through, the next ten get queued, and anything beyond is rejected.

How I decide: if a human clicking around your site could plausibly generate the spike, use nodelay or a delay that covers it. A single page load pulling twelve uncached assets through Nginx is one “request” to the user and twelve to the limiter. If the endpoint is machine-facing and expensive (search, export, login, an API write path), tighten the rate, keep the burst small, and use nodelay so rejection is fast and cheap.

Also set limit_req_status 429. The default is 503, which tells clients and monitoring that your server is broken rather than that they are being throttled. 429 is the correct semantic, and well-behaved clients back off on it.

Sizing the zone, and the number most guides get wrong

You will see “10m holds about 160,000 IP addresses” repeated everywhere. That is the 32-bit figure. Nginx documents the stored state for limit_req as 64 bytes on 32-bit platforms and 128 bytes on 64-bit platforms, so one megabyte holds roughly 16 thousand states on 32-bit and roughly 8 thousand on 64-bit. On the 64-bit server you are almost certainly running, a 10m zone is closer to 80,000 states.

This matters because of what happens when the zone fills. Nginx evicts the least recently used state. If it still cannot allocate, the request is terminated with your configured error status. A zone that is constantly thrashing under a distributed crawl will start returning errors for reasons that have nothing to do with the rate you configured. If you are limiting a high-cardinality key, give the zone room.

limit_conn solves a different problem, and you probably need both

limit_req caps how fast requests arrive. It does nothing about a client that opens sixty connections and reads each response one byte at a time. That is a concurrency problem, and it belongs to limit_conn.

http {
    limit_conn_zone $binary_remote_addr zone=perip:10m;
    limit_conn_status 429;
}

server {
    limit_conn perip 20;

    location /downloads/ {
        limit_conn perip 4;
        limit_rate_after 2m;
        limit_rate 512k;
    }
}

Three behaviours worth knowing before you pick a number.

• Not every connection counts. Nginx only counts a connection once the full request header has been read and the request is being processed. A client stalling mid-header is not counted here, which is why client_header_timeout and client_body_timeout still matter.
• Under HTTP/2 and HTTP/3, each concurrent request counts as a separate connection. A limit of 4 that felt generous on HTTP/1.1 can break a modern browser loading a page with many assets. This is a very common self-inflicted outage.
• The limit_conn state is smaller than the limit_req state, so the same zone size holds roughly twice as many entries.

The pairing is the point: limit_req protects against volume, limit_conn protects against occupancy. Rate limiting alone leaves you open to slow-read exhaustion, and connection limiting alone does nothing about a client making one fast request after another on a keepalive connection.

Choosing keys: IP is a starting point, not the answer

The key can be any variable or combination of variables. That flexibility is where most of the value is, because per-IP limits have a real accuracy problem in both directions.

On one side, carrier-grade NAT, corporate egress, university networks and shared VPN exit nodes (NordVPN, Surfshark and every other provider works this way) put many genuine users behind one address. A tight per-IP limit will hit them. On the other side, a distributed scraper renting a few hundred cheap addresses never touches a per-IP limit at all, because no single address is doing much.

So layer keys instead of tuning one harder:

limit_req_zone $binary_remote_addr zone=perip:20m    rate=10r/s;
limit_req_zone $server_name        zone=persite:1m   rate=200r/s;

Applied together, the per-IP zone catches the loud individual and the per-site zone caps total damage regardless of how many addresses are involved. When several limit_req directives match, all of them are evaluated and the most restrictive result wins. If any one of them rejects, the request is rejected.

One inheritance rule catches people out: limit_req directives are inherited from the enclosing level only if the current level defines none. Add a single limit_req inside a location and you have dropped every limit set at server level for that location. If you want both, restate both.

Allowlisting with an empty key

Requests whose key evaluates to an empty string are not accounted at all. That is the clean way to exempt monitoring probes, your office, or a payment webhook source without maintaining a parallel set of locations.

geo $limit {
    default        1;
    10.0.0.0/8     0;
    192.0.2.10/32  0;
}

map $limit $limit_key {
    0 "";                    # empty key, never limited
    1 $binary_remote_addr;
}

limit_req_zone $limit_key zone=perip:20m rate=10r/s;

Nginx evaluates map variables only when they are used, so declaring a large map costs nothing on requests that never reference it.

Roll it out in dry run, not in production

Both modules have a dry run mode. Excessive requests are counted in the shared memory zone and logged exactly as they would be, but nothing is actually rejected or delayed. Combined with the status variables, this gives you a real answer to “would this limit have hurt anyone” before you find out from a customer.

log_format ratelimit '$remote_addr $status "$request" '
                     'req=$limit_req_status conn=$limit_conn_status '
                     'ua="$http_user_agent"';

server {
    limit_req_dry_run  on;
    limit_conn_dry_run on;

    access_log /var/log/nginx/ratelimit.log ratelimit;
}

$limit_req_status reports PASSED, DELAYED, REJECTED, DELAYED_DRY_RUN or REJECTED_DRY_RUN. $limit_conn_status reports PASSED, REJECTED or REJECTED_DRY_RUN. A numbered rollout that has never embarrassed me:

1. Add the zones and the limit_req directives with dry run on, plus the log format above.
2. Run for a full traffic cycle. A week if you have weekly patterns, longer if you have a monthly billing spike.
3. Count the would-be rejections by user agent and by address. If real browsers appear, your rate or burst is wrong, not your users.
4. Turn dry run off on the least critical location first. Watch 429 rates alongside your normal error budget.
5. Only then apply it to login, checkout, or anything that costs you money when it breaks.

Shipping the rate limit log into Loki and graphing it in Grafana makes step three take minutes instead of an evening with awk. Any log platform with decent label cardinality handling works; the point is having rejection counts broken down by zone and user agent, over time, next to your latency graphs.

Bot mitigation in layers, cheapest first

Rate limiting treats all clients the same. Bot mitigation is about deciding that some clients deserve different treatment before the limiter ever sees them. The useful mental model is a set of layers ordered by cost and by how easy each is to defeat.

Layer one: robots.txt, and being honest about what it does

robots.txt is a request, not a control. Crawlers that respect it will honour a Disallow on your search endpoint, your faceted filter URLs and your infinite calendar pages, and that alone removes a surprising amount of load. Crawlers that do not respect it will ignore the file completely, and some will read it specifically to find out where the interesting URLs are.

Use it to shape well-behaved traffic. Never use it as a security boundary. And be careful about blanket disallows on search engines you actually want: deindexing yourself is a much more expensive mistake than a bit of crawler load.

Layer two: user agent classification with map

A map on $http_user_agent is cheap, readable, and easy to revise. It is also trivially spoofable, so treat it as traffic shaping rather than enforcement.

map $http_user_agent $ua_class {
    default          human;
    ""               empty;
    "~*bot|crawler|spider|scrape"  bot;
}

map $ua_class $bot_limit_key {
    bot   $binary_remote_addr;
    empty $binary_remote_addr;
    human "";
}

limit_req_zone $bot_limit_key zone=bots:10m rate=30r/m;

That gives declared bots a much tighter budget than browsers while leaving human traffic on the general limit. An empty user agent is worth its own class: almost nothing legitimate sends one, and it is a reliable signal for cheap scraping tools.

Two cautions. Broad substring matching on bot will catch strings you did not intend, so review a sample of your own logs before deploying the pattern. And blocking rather than throttling is a bigger decision than it looks: if you return 403 to a crawler you actually want, you find out weeks later in your search traffic, not in your error log.

Layer three: verifying bots that claim a name

The interesting case is a request claiming to be a major search crawler. Impersonating one is the standard way to bypass a user agent based allowlist, so a claim needs verification.

The method the search vendors themselves document is a forward-confirmed reverse DNS check: reverse-resolve the client address, confirm the resulting hostname belongs to the vendor’s domain, then forward-resolve that hostname and confirm it returns the original address. An attacker cannot fake this without controlling the vendor’s DNS. Several vendors also publish their crawler IP ranges as machine-readable lists, which avoids the DNS round trip entirely.

Here is the honest limitation: stock Nginx has no directive that does this. There is no built-in reverse DNS verification in the core modules. Your realistic options are a third-party module, doing the verification out of band and feeding the result into a geo block or an included allowlist file, or pushing the decision up to a CDN or WAF layer that already does it. Anyone showing you a pure stock-Nginx config that “verifies Googlebot” is verifying a string, not an identity.

This is also the point where an edge provider earns its keep. Cloudflare, Fastly and similar edge platforms maintain verified bot lists and fingerprinting that no single origin server can reproduce, and they drop the traffic before it costs you a worker. If your bot problem is large and distributed, the edge is the right layer and Nginx is your backstop, not your front line.

Layer four: escalation to the firewall

A client that collects hundreds of 429s and keeps going is not going to stop. Serving it a rejection still costs you a TCP handshake, a TLS negotiation and a log line. At that point the cheapest response is to stop answering at all.

Fail2ban is the usual tool: a filter that matches your rate limit rejections in the Nginx error or access log, an action that inserts a firewall drop, and a ban time long enough to be inconvenient. Two things to get right. Match on a pattern that only fires for real limit rejections, or you will ban yourself during a deploy. And if you are behind a CDN, your firewall sees the CDN address, not the client, so the ban has to be applied at the edge instead. Banning a Cloudflare range at your origin firewall takes your whole site offline, and it is a mistake people make exactly once.

Return code 444 is worth knowing here. It is Nginx-specific and closes the connection without any response, which is cheaper than generating an error page for a client that will never read it.

Troubleshooting Nginx rate limiting

The limit does nothing. Check the key first. Log $binary_remote_addr alongside $realip_remote_addr and confirm they differ in the way you expect. Then check inheritance: a limit_req in the matched location discards the server level ones. Confirm which location actually matched by adding $request_uri and a marker header while testing.

Real users are getting 429 or 503. Turn dry run back on and read the log rather than guessing. Usually it is one of three things: a burst too small for a normal page load’s asset count, a limit_conn value set for HTTP/1.1 assumptions on an HTTP/2 server, or a shared-egress network where many people genuinely share one address.

Nothing appears in the error log. The default log level for rate limit refusals is error, and delays are logged one level below whatever you set. If you set limit_req_log_level warn, delays go to info, which your log level may be filtering out entirely. Grep for the limiting messages directly:

grep 'limiting requests' /var/log/nginx/error.log | tail -50
grep 'limiting connections' /var/log/nginx/error.log | tail -50

Intermittent errors under heavy load with no matching rate. Suspect zone exhaustion. Enlarge the zone and see whether the pattern changes. High-cardinality keys and distributed crawls are the usual cause.

Health checks and uptime probes are being throttled. Add them to the allowlist map. Monitoring hitting a rate limit produces alerts about an outage that is not happening, which is worse than no monitoring.

Common mistakes

• Trusting forwarded headers from every source, which turns a per-IP limit into no limit at all.
• Leaving limit_req_status at the default 503, so throttling looks like an outage in your own monitoring.
• Using burst without nodelay on an abuse path, and quietly filling worker slots with held connections.
• Setting a limit_conn value based on HTTP/1.1 intuition on a server serving HTTP/2 or HTTP/3.
• Adding one limit_req in a location and silently dropping every server-level limit for that path.
• Blocking user agent substrings without checking a sample of real logs first.
• Applying identical limits to reads and writes. A cacheable GET and a POST that writes to the database do not deserve the same budget.
• Banning CDN address ranges at the origin firewall because the logs show them as the client.

Best practices worth the effort

• Verify the key before tuning the rate. Every other decision depends on it being correct.
• Always start in dry run and measure a full traffic cycle before enforcing.
• Use separate zones per traffic class. Search, login, API, static and general traffic have different shapes and should not share a bucket.
• Pair limit_req with limit_conn. The first caps arrival rate, the second caps occupancy, and neither substitutes for the other.
• Layer a per-server zone above your per-IP zone so distributed traffic still hits a ceiling.
• Allowlist with an empty key rather than duplicating location blocks.
• Export $limit_req_status to your log platform and graph rejections next to latency and upstream errors.
• Keep CDN trust ranges in a generated include file that is refreshed on a schedule and validated with nginx -t before reload.
• Throttle before you block. Blocking failures are invisible until they show up in your search traffic.

Frequently asked questions

Does Nginx rate limiting stop a DDoS attack?

It helps with application-layer floods that reach your server, because rejecting a request is far cheaper than processing it. It does nothing about volumetric attacks that saturate your network link, since that traffic never gets to Nginx to be limited. For that you need capacity in front of the origin: a CDN or scrubbing service. Treat rate limiting as protection against abuse and accidental floods, not as DDoS mitigation.

What rate should I actually set?

There is no universal number, and any guide that gives you one has not seen your traffic. Measure instead: run dry run mode, look at your real per-IP request distribution, and set the rate above the busiest legitimate client with headroom. Then set a separate, much tighter zone for the specific endpoints that are expensive or attacked.

Should I use 429 or 503 for rejected requests?

429 in nearly every case. It communicates that the client should slow down, well-behaved clients back off on it, and it keeps your 5xx metrics meaningful. The default is 503 purely for historical reasons.

Can Nginx rate limit per hour or per day?

Not with the stock module. It accepts requests per second and requests per minute only, so the slowest native rate is one request per minute expressed as 1r/m. Longer windows need a third-party module, a shared store such as Redis via a scripting module, or enforcement in the application itself. For something like a password reset limit, the application is usually the better place anyway, because it can key on the account rather than the address.

Do rate limit zones work across multiple Nginx servers?

Not in the open source build. Each instance keeps its own shared memory zone, so a client spread across four load-balanced servers effectively gets four times the budget. The sync parameter on limit_req_zone synchronises state across a cluster, but it is part of the commercial subscription. Without it, either divide your intended rate by your instance count, ensure your balancer pins clients consistently, or move the limit to a shared store.

Will blocking AI crawlers hurt my SEO?

Blocking training crawlers is separate from blocking search indexing crawlers, and the distinction is worth checking per vendor before you write a rule, because some vendors run several crawlers with different purposes under related names. The risk is collateral damage: a broad user agent pattern that also matches a search crawler will remove you from results, and you will not notice quickly. Verify your pattern against real log samples, and prefer throttling over outright blocking while you are unsure.

Where should rate limiting live: the CDN, Nginx, or the application?

All three, doing different jobs. The CDN handles volume and known-bad reputation before it costs you anything. Nginx handles per-client shaping and protects your workers from anything that gets through. The application handles limits that need identity or business context, like per-account quotas, which Nginx cannot see. Skipping the Nginx layer because you have a CDN leaves you exposed the moment someone finds your origin address.

The one thing worth remembering

Nginx rate limiting is not hard to configure. It is easy to configure into a state where it silently does nothing, or silently punishes the wrong people, and both failures look identical from the outside: a config that parses, a clean nginx -t, and no complaints until there are.

So get the key right, prove it with a log line rather than assuming it, and run every new limit in dry run until the data tells you it is safe. Layer bot handling by cost, throttle before you block, and be honest about which layer can actually verify an identity. Everything else is tuning, and tuning is easy once the foundation is not lying to you.

Need a second pair of eyes on your Nginx edge?

Most of the work here is diagnostic rather than clever. Things I regularly help with:

• Auditing an existing limit_req and limit_conn setup to find out whether it is actually limiting anything, including the keying and inheritance traps above.
• Fixing real client IP handling behind Cloudflare or another CDN, with a generated trust list that refreshes safely instead of drifting.
• Designing per-endpoint zones for WordPress, WooCommerce or API origins so login, search and write paths get budgets that match their real cost.
• Running a measured dry run rollout and reporting back what would have been rejected, broken down by user agent and network, before anything is enforced.
• Building bot classification and escalation: user agent maps, allowlists, Fail2ban integration, and knowing which decisions belong at the edge instead.
• Wiring rate limit telemetry into Grafana and Loki so throttling is something you can see on a dashboard rather than discover from a ticket.

If you would like a look at yours, send me the relevant server block, a sample of the access log, and whatever is currently going wrong. That is usually enough to say something useful straight away.

The post Nginx Rate Limiting and Bot Mitigation Without Blocking Real Users appeared first on John Nessime.

Here is the awkward part: nothing was broken. No alert fired, no error rate moved, no dashboard turned red. Third-party API spend is one of the few production signals with no failure mode attached to it. A background job that polls a vendor every thirty seconds and gets back the exact same payload it got last time looks, from the outside, identical to a healthy integration. It just costs money every single time.

This post is about how to reduce SaaS API costs by changing the shape of your calls rather than by renegotiating your contract. Six levers, in the order I’d pull them: read the billing unit, kill pointless polling, collapse N+1 patterns, cache honestly, stop your own retry amplification, and attribute spend so you can tell whether any of it worked.

Start with the billing unit, not the call count

The single most common mistake I see is optimising HTTP requests when the vendor doesn’t bill HTTP requests. Before you touch any code, read the pricing page carefully enough to answer one question: what is the countable thing?

It varies more than people expect:

• Per request. The obvious one. Geocoding, enrichment, most REST endpoints.
• Per session. Google Maps Platform bills Places Autocomplete this way when you pass a session token: the whole keystroke stream plus the terminating Place Details call is grouped and billed as one unit. Omit the token, or reuse one across sessions, and every keystroke request is billed individually.
• Per token issued. Auth0’s machine-to-machine quotas count access tokens obtained through the client credentials flow, not the API calls you make with them. A service that fetches a fresh token on every outbound request is burning the meter for no reason.
• Per input and output token. LLM providers. Output is typically priced higher than input, which makes “be concise” a cost control and not just a style note.
• Per record, event, or message. Twilio, SendGrid, most streaming and messaging platforms. One API call can carry many billable units.
• Not at all. Stripe doesn’t bill you per API call; the constraint there is a rate limit, not a line item. Optimising those calls buys you reliability headroom, not money.

Getting this wrong sends you down the wrong path for weeks. Batching requests against an endpoint that bills per record changes your request count and leaves the invoice exactly where it was. Read the unit first.

Lever one: stop polling for things that rarely change

This is almost always the biggest single line, and it’s invisible because it’s steady. A sync job on a five-minute cron is 288 calls a day per resource. Multiply by resources, by environments, by the replicas you quietly scaled to three, and you get a number with no relationship at all to how often the data changed.

Conditional requests, when the vendor honours them

If the API returns an ETag or Last-Modified header, store it and send it back on the next poll as If-None-Match or If-Modified-Since. If nothing changed you get a 304 Not Modified with an empty body.

# First call: capture the ETag from the response headers
curl -sS -D - -o /dev/null 
  -H "Authorization: Bearer $TOKEN" 
  https://api.github.com/repos/OWNER/REPO/pulls

# Second call: send it back and check the status code only
curl -sS -o /dev/null -w '%{http_code}n' 
  -H "Authorization: Bearer $TOKEN" 
  -H 'If-None-Match: "PASTE_THE_ETAG_HERE"' 
  https://api.github.com/repos/OWNER/REPO/pulls

-D - dumps response headers to stdout so you can see the etag value. -w '%{http_code}' on the second call prints just the status, which is all you care about: 304 means the resource is unchanged and you saved a payload.

Whether that saves money depends on the vendor. GitHub documents that a conditional request returning 304 doesn’t count against your primary rate limit when it’s correctly authorised. Plenty of other vendors return 304 happily and bill it anyway. Test against your own invoice before you assume.

Three things that bite people with ETags:

• They’re per-page on paginated endpoints. A 304 on page one says nothing about pages two through five, so store one ETag per page.
• They can be scoped to the credential. Rotate a token and your stored ETags may stop matching, which quietly turns your conditional requests back into full ones.
• GraphQL endpoints generally don’t do conditional requests at all. You cache those yourself, keyed on a hash of the query and its variables.

Webhooks, when they exist

The proper fix is to stop asking. If the vendor supports webhooks, the call count collapses from “one per interval” to “one per actual change”, which is usually a difference of two orders of magnitude on a slow-moving resource.

Be honest about what it costs, though. Webhooks need a publicly reachable endpoint, signature verification, idempotent handling because vendors redeliver, a dead-letter path, and a reconciliation sweep for deliveries that never arrived. On a resource that changes every few seconds, polling is simpler and the savings are small. On one that changes twice a week, the webhook pays for itself immediately.

A middle option people forget: many APIs expose a cheap “has anything changed” endpoint, a sync cursor, or a delta token. Poll that on a tight interval and only fetch the expensive payload when it moves.

Lever two: collapse the N+1 pattern

You know this one from databases, and it shows up identically against third-party APIs. Fetch a list of 200 items, then fetch each item’s detail individually: that’s 201 billable calls where the vendor probably offers a way to do it in one or two. Look for:

• Expansion or embed parameters. Many REST APIs let you inline related objects in the list response so the detail calls disappear entirely.
• Field selection. Asking for fewer fields is free on per-request billing but matters a lot where the field tier drives the price. Google Places is explicit about this: requesting a single higher-tier field upgrades the whole response to that tier’s pricing.
• Bulk and batch endpoints. Salesforce, Algolia, most CRMs and search platforms have one. They’re usually documented in a corner nobody reads.
• Larger pages. If the endpoint accepts a page size of 200 and you’re defaulting to 25, you’re paying eight times as many calls to fetch identical data.

The page-size one is the cheapest win in this entire post and it’s a one-line change. Check your client library defaults today.

Lever three: cache with a boring, honest TTL

Caching third-party responses is the lever everyone reaches for first and gets subtly wrong. Three failure modes matter more than the cache itself.

Stampede. A popular key expires, fifty concurrent requests all miss, and fifty identical calls go out to the vendor. You just paid fifty times for one refresh, at the worst possible moment.

No negative caching. A lookup that returns “not found” is usually not cached at all, so the same failing lookup hammers the vendor forever. Cache your 404s. Short TTL, but cache them.

Cache key design. If your key doesn’t include the credential or tenant, you will eventually serve one customer’s data to another. This is the one mistake in this post that turns a cost problem into an incident.

You don’t necessarily need application code for this. An Nginx instance in front of the vendor handles all three, and it’s the approach I reach for first when several services hit the same API:

proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:20m
                 max_size=1g inactive=60m use_temp_path=off;

server {
    listen 127.0.0.1:8080;

    location / {
        proxy_pass https://api.vendor.example;
        proxy_set_header Host api.vendor.example;

        proxy_cache api_cache;
        proxy_cache_key "$request_method$request_uri$http_authorization";

        proxy_cache_valid 200 5m;
        proxy_cache_valid 404 30s;

        proxy_cache_lock on;
        proxy_cache_use_stale updating error timeout;
        proxy_cache_background_update on;

        add_header X-Cache-Status $upstream_cache_status;
    }
}

Reading that from the top down, because every directive is there for a reason:

• proxy_cache_key includes $http_authorization, so responses are isolated per credential. That’s the tenant-leak fix.
• proxy_cache_valid 404 30s is the negative caching. Failed lookups stop being free traffic for the vendor.
• proxy_cache_lock on is the stampede fix. Only the first request for a given key goes upstream; the rest wait for it.
• proxy_cache_use_stale updating plus proxy_cache_background_update on serves the stale copy while a single background request refreshes it. Latency stays flat and you don’t pay a thundering herd at every expiry.
• X-Cache-Status gives you HIT, MISS, STALE or UPDATING in the response so you can actually measure your hit rate instead of guessing at it.

The trade-off is worth stating plainly: you’re now serving data that can be five minutes stale, and you own another moving part. Fine for a currency rate or a product catalogue. Not fine for an inventory count on a checkout page. Set the TTL from how wrong the data is allowed to be, never from how much you want to save.

Where the caller is distributed, Redis with a short lock key does the same job in application code, and a small always-on VPS from something like InterServer or DigitalOcean is plenty to host either the proxy or the cache. At the edge, Cloudflare Workers with KV lets you cache vendor responses close to users, which helps when the calls originate from browsers rather than your backend.

Lever four: stop paying for your own retries

This produces the sudden spike rather than the slow creep, and people miss it because retries feel like resilience.

The vendor wobbles. Every client retries. With a fixed delay they retry in lockstep, so the vendor gets a synchronised wave on top of normal load, struggles longer, and generates more retries. AWS’s architecture write-up on exponential backoff and jitter is the canonical reference, and its central finding is worth repeating: backoff alone still leaves visible clusters of calls. Randomising each client’s delay is what decorrelates the herd, and in their simulation it cut total call count substantially.

import random, time

BASE = 0.5      # seconds
CAP  = 30.0     # never sleep longer than this

def sleep_before_retry(attempt):
    # capped exponential backoff, then full jitter
    window = min(CAP, BASE * (2 ** attempt))
    time.sleep(random.uniform(0, window))

The cap stops a long outage turning into hour-long sleeps. The random.uniform(0, window) is the jitter: two processes that failed at the same instant now wake at different times. Note that full jitter can occasionally produce a near-zero delay, which is fine for transient errors but too aggressive for throttling responses that reset on a per-minute window. For a 429, honour the Retry-After header if the vendor sends one, and use a jitter formula with a guaranteed floor if they don’t.

Three more rules that cost nothing to implement:

1. Never retry a 4xx except 429. A 400 or a 422 will fail identically forever. Every one of those retries is a billed call buying you nothing.
2. Cap retries as a fraction of traffic. A retry budget (say, retries may not exceed some small percentage of total requests) is the only control that sees the system-wide picture. Per-client backoff can’t.
3. Check your layers. If the SDK retries three times, your HTTP client retries three times, and your job runner retries three times, one logical operation is up to twenty-seven calls. This stacking is extremely common and almost never intentional.

For writes, retrying safely needs an idempotency key so the vendor deduplicates rather than creating two of whatever you asked for. Stripe’s Idempotency-Key header is the pattern most others copied. Without it, teams disable retries on writes entirely and then eat manual reconciliation work instead.

Lever five: move latency-tolerant work off the synchronous path

Ask a blunt question about each integration: does a human wait on this response?

Nightly enrichment, classification, report generation, bulk address validation, embedding backfills. Nobody is waiting. Several vendors, including the major LLM providers, offer asynchronous batch endpoints at a discount to the synchronous rate in exchange for a looser delivery window. Rates move, so check current terms rather than a number you read somewhere, but the mechanism is stable and the discount is usually substantial.

Even without a batch endpoint, moving work behind a queue lets you coalesce. Ten events touching the same customer record within a minute become one enrichment call instead of ten. That deduplication window is often worth more than the batch discount.

Lever six: attribute the spend or you’re guessing

Everything above is unfalsifiable until you can answer: which feature, which tenant, which job. The invoice gives you the total and never the cause. What I’d put in place, in order:

1. Route outbound calls through one place. A shared HTTP client wrapper or the caching proxy above. If every service instantiates its own client, you have no chokepoint and no measurement.
2. Emit a counter per call with labels for vendor, endpoint, status class, and cache result. Keep the label set small; per-tenant labels will blow up your metrics cardinality and trade one bill for another.
3. Build one unit-cost metric. Vendor spend per active tenant per month. It’s the only number that survives growth, because raw spend going up during a good quarter is not a problem.
4. Alert on call rate, not on the bill. A rate alert fires in minutes. An invoice fires in thirty days. Grafana, Prometheus, Datadog, CloudWatch, whatever you already run is fine; the point is that the alert exists at all.
5. Track cache hit rate as a first-class SLI. When someone changes a cache key format and the hit rate silently drops to zero, this is the graph that tells you before the invoice does.

Troubleshooting: the bill jumped and nothing shipped

Work down this list. In my experience the cause is usually in the first four.

• Replica count changed. Autoscaling or a Kubernetes replica bump multiplies every cron-driven poller. Check whether your scheduled jobs have leader election, or whether all five pods are dutifully polling the same endpoint.
• Cache hit rate collapsed. A changed key format, a Redis eviction policy under memory pressure, or a TTL someone dropped “temporarily” during debugging. Graph the hit rate over the period, not the current value.
• Retry loop against a persistent 4xx. Look for a flat, high rate of identical failing calls. This has the distinctive signature of high call volume with zero successful responses.
• A non-production environment pointed at production credentials. Staging, a CI pipeline, or an integration test suite running on every commit. Check whether calls are arriving from IPs you don’t recognise.
• Session or token handling regressed. A refactor that dropped a session token parameter, or a service that stopped caching its auth token and now mints a new one per request.
• Legitimate growth in an expensive segment. A few large customers using a costly feature move the bill without moving call count much. Not a bug, but worth ruling in before you go hunting for one.

If the vendor exposes per-key usage breakdowns in their dashboard, split your credentials by service before your next incident. Separate API keys per environment and per major consumer turn a whodunnit into a two-minute lookup.

Common mistakes

• Optimising the endpoint that’s easy to change rather than the one that dominates the bill. Rank by spend first.
• Caching without a per-credential key, and finding out the hard way in a multi-tenant system.
• Treating 304 responses as free when your specific vendor bills them anyway.
• Setting TTLs from a savings target rather than a staleness tolerance, and finding out during a customer escalation.
• Leaving retry logic enabled at three separate layers and calling the result “defence in depth”.
• Adding per-tenant labels to outbound-call metrics and moving the cost from the API vendor to the observability vendor.
• Shipping the optimisation and never checking the next invoice against the prediction.

Best practices worth keeping

• One outbound HTTP client wrapper per service, with timeouts, retries, and metrics configured in exactly one place.
• Separate API credentials per environment and per major consumer, so usage dashboards mean something.
• Cache auth tokens until shortly before expiry. Refresh early, not on every call.
• Add jitter to scheduled jobs as well as to retries. Everything set to run at the top of the hour arrives at the vendor as a spike.
• Write down the staleness tolerance for each cached resource next to its TTL, so the next person doesn’t have to reverse-engineer your reasoning.
• Re-check the pricing page after any vendor product update. Billing units change more often than endpoints do.

FAQ

How do I find out which service is driving my API spend?

Start with the vendor’s own usage dashboard broken down by API key. If all your services share one key, that’s the first thing to fix. Failing that, log every outbound call at the client wrapper with the endpoint and calling service, and aggregate for a week. A week of honest logs beats a month of theorising.

Do ETags actually reduce SaaS API costs, or just help with rate limits?

It depends entirely on the vendor. GitHub documents that an authorised conditional request returning 304 doesn’t count against the primary rate limit. Others count every request regardless of status. Run a controlled test: a known number of conditional calls over a billing period, then compare against the usage report. Don’t assume either way.

Is caching third-party API responses allowed?

Usually yes, but not always, and some terms of service restrict how long you may store certain data. Mapping and place data is a common example of restricted caching. Check the terms for the specific vendor before you build a permanent store, particularly if you’re caching anything personally identifiable.

How much can caching realistically cut an API bill?

Nobody can give you a number without knowing your traffic. Estimate it yourself: for a candidate endpoint, count distinct request keys versus total requests over a day. That ratio is roughly your ceiling for hit rate. Multiply by the endpoint’s share of spend. If the answer is a couple of percent, spend your time elsewhere.

Should I always replace polling with webhooks?

No. The comparison is change frequency against poll frequency. If the resource changes almost as often as you poll, you save little and take on signature verification, redelivery handling, and a reconciliation job. Webhooks win decisively on slow-changing resources and on anything where you’re polling more than a handful of objects.

Does rate limiting my own users reduce vendor costs?

Only when user actions map directly to vendor calls, such as a search box that fires an upstream request per keystroke. Debouncing input and enforcing per-plan quotas helps a lot there. For background processing driven by your own schedules, user rate limits change nothing.

Should I disable retries to save money?

No. Fix them instead. Capped exponential backoff with jitter, no retries on non-429 client errors, a retry budget, and retry logic living at exactly one layer. Disabling retries entirely trades a cost problem for a reliability problem, and reliability problems are more expensive.

The one thing worth remembering

Most efforts to reduce SaaS API costs fail because they start with code. They should start with two questions: what is the vendor actually counting, and which of my calls returned information I already had?

Answer those honestly and the work usually collapses to a handful of unglamorous changes. Raise a page size. Cache a token. Put a lock around a cache refresh. Delete a poller that was watching something that changes twice a week. None of it is clever. All of it shows up on the next invoice, which is the only review that counts.

Need help getting your API spend under control?

This is work I take on regularly. Typical engagements look like:

• Auditing outbound API traffic to find which endpoints, jobs and tenants are driving the bill, and ranking them by spend rather than by how obvious they are.
• Building a caching layer in front of chatty vendor APIs, with Nginx, Redis or Cloudflare Workers, including stampede protection and per-tenant key isolation.
• Replacing polling jobs with webhook receivers or delta-token syncs, with signature verification, idempotent handling and a dead-letter path.
• Auditing timeout and retry configuration across services to remove stacked retries and add proper backoff, jitter and retry budgets.
• Moving latency-tolerant workloads onto batch or queue-based paths, with deduplication windows that cut call volume before it reaches the vendor.
• Setting up call-rate and cache-hit-rate dashboards and alerts in Prometheus, Grafana, Datadog or CloudWatch so the invoice is never the first signal.

If you’ve got a cost graph that’s climbing, a job you suspect is over-polling, or a client wrapper you’re not sure about, send it over and I’ll tell you what I’d look at first.

The post Optimizing API Calls to Reduce SaaS Costs: Six Levers That Actually Move the Bill appeared first on John Nessime.

The host had removed it. It was on their disallowed list, a routine scan found it, and the platform did exactly what its documentation said it would do. Everyone had agreed to that when they signed up. Nobody had read it.

That incident is the whole managed WordPress hosting vs VPS argument in miniature, and it has almost nothing to do with page speed. Both models can serve a fast WordPress site. They differ on which failures land on you, which land on someone else, and which ones stay invisible until the worst possible moment.

This post compares the two the way I’d compare them for a client: an honest profile of each, the costs that never appear on the invoice, a procedure for modelling the real number, and the decision rules I actually use. No benchmark charts, because your theme and your plugin count will dominate anything the host does.

The real question: who owns the failure at 2am

Most comparisons frame this as control versus convenience. True, but useless, because it doesn’t tell you what to buy. The framing that predicts regret is: when something breaks, whose problem is it, and how fast can they fix it?

Split the failure surface into layers and the answer gets concrete:

• Hardware and hypervisor. Theirs in both models. Your recovery time is still yours.
• OS, web server, PHP, database. Theirs on managed hosting, yours on an unmanaged VPS. This is the big split.
• Backups and restores. Theirs on managed, and tested, because their business depends on it. Yours on a VPS, and almost nobody tests them.
• Core, themes, plugins. Mostly yours either way. Some hosts auto-update, but the compatibility fallout is still yours.
• Your code and your content. Always yours. No host takes this on.

The bottom two rows never move. A managed platform will not make your bloated page builder fast, and it will not stop a vulnerable plugin from being exploited. It moves the middle three rows off your plate and charges you in money and in flexibility.

Managed WordPress hosting, assessed honestly

Where it genuinely wins

The strongest argument isn’t performance. It’s that the boring, high-consequence work gets done by people whose job it is, on a schedule, whether or not you remember.

• Restores that work. One-click recovery from a backup the platform verifies. That’s the difference between a bad hour and a bad week.
• Server-level page caching, already tuned. WP Engine, Kinsta, Pressable and Flywheel run full-page caching in front of PHP. It’s why they ban plugins that try to do the same thing badly.
• Staging that isn’t a science project. Clone, test, push back. On a VPS that’s yours to build, and a half-built staging setup is worse than none.
• Support that knows WordPress. A generic VPS provider’s ticket ends at “the server is up”. A WordPress host will look at your slow query log.
• Edge protection before PHP loads. WAF and bot filtering at the infrastructure layer, the only layer where they meaningfully help. A plugin firewall runs after the request has already cost you a PHP worker.

If nobody on your team wants to own a Linux box, that list is decisive. The rest of this section is about what you’re trading away.

Visit metering is a billing model, not a traffic report

This is the cost that bites hardest, because it doesn’t look like a technical constraint until the invoice arrives.

Most managed plans meter “visits” rather than bandwidth or CPU, and the definition is narrower than it sounds. WP Engine defines a billable visit as one unique IP address logged per day, with static assets like images, CSS and JavaScript excluded, and known and suspected bot user agents filtered out. Kinsta counts the sum of unique IPs seen within each 24-hour period across the plan, and also sells bandwidth-metered plans as an alternative. Flywheel filters IPs belonging to known bots, spammers and attackers before counting.

Three consequences follow, and they’re the ones people miss:

• Your host’s number will not match Google Analytics. Analytics counts sessions from browsers that ran its JavaScript. Your host counts IPs that hit the server. Arguing about the gap is wasted effort.
• Bot filtering is best-effort. Hosts filter what they can identify. A scraper with a browser-shaped user agent and rotating IPs looks, to the meter, like a crowd of new visitors.
• Overages are charged per block of excess visits. Rates change, so check the current figure rather than trusting any article, including this one. What matters is the shape: a viral post or a bad crawl month produces a bill you didn’t budget for.

A VPS has the opposite failure mode. Bad traffic costs you CPU and RAM, not money, right up until it exhausts them and the site falls over. One model converts load into a bill, the other converts it into an outage. Pick the one you’d rather explain to whoever owns the site.

The plugin policy is a real constraint

Managed hosts publish disallowed plugin lists and they enforce them. WP Engine documents that disallowed plugins are found by periodic scans of the site’s filesystem, the owner is notified, and the plugin is removed. That’s not a threat, it’s the operating model, and it’s why the incident I opened with was nobody’s fault.

The banned categories are consistent across vendors, and the reasoning holds up in each case:

• Caching plugins. They fight the platform’s own page cache and produce stale output that support then has to debug.
• Backup plugins. They write large archives onto the same disk the site runs on, which is a useless backup if the disk is what fails.
• Database optimisation plugins. Long-running write-heavy queries on shared infrastructure affect neighbours.
• Some security plugins, or specific features within them. Filesystem-based rule storage and high-volume traffic logging are the usual triggers. Policies vary a lot here.
• Plugins that send mail directly. Platforms push you to a transactional provider instead, for deliverability reasons that are genuinely in your interest.

None of that is unreasonable. The problem is when a plugin your business depends on lands in one of those buckets, or when a host adds something to the list later. Before migrating, read the current disallowed list for that specific host and diff it against your live active plugins. Ten minutes, and it’s the highest-value thing you can do in a hosting evaluation.

You get one shape of application

A managed WordPress platform runs WordPress. If your project also needs a Node service, a Python worker, a queue, or a self-hosted analytics instance, that’s a second bill somewhere else. On a VPS those things are free in cash terms because you already own the machine. They cost you memory and attention instead. Whether that’s a good trade depends on whether you were going to need them anyway.

Running WordPress on your own VPS, assessed honestly

Where it genuinely wins

• Resources per unit of money. Hetzner, InterServer, DigitalOcean and Vultr will sell you more CPU and RAM than an entry managed plan. Whether you can use it is a separate question.
• Many sites, one bill. This is where the economics genuinely flip. Ten small sites on one adequate VPS is one server to patch. Ten managed plans is ten invoices.
• No metering surprises. Traffic spikes hit your load average, not your credit card.
• You choose the stack. Nginx or Apache, PHP-FPM pool sizing, Redis for the object cache, your own MariaDB tuning and firewall rules. If the platform’s opinions were what limited you, this is the fix.
• No lock-in. A tarball and a database dump move anywhere. Platform-specific caching behaviour, redirect rules and deployment workflows do not.

The failure that stays invisible: the backup you never restored

Here’s the one that actually ends projects. Backups on a self-managed VPS fail silently. The cron entry runs, the exit code goes nowhere, the destination bucket quietly rejects writes after a credential rotation, and nothing tells you. You find out the day you need it, which is also the day you have no other option.

The fix isn’t a better tool. Restic and BorgBackup are both excellent and neither will save you. The fix is a restore drill on a schedule, into a scratch directory, where you verify the data is actually there.

# Restore the newest snapshot into a scratch path, never over the live site
restic -r /srv/backups/wp restore latest --target /tmp/restore-drill

# Confirm the dump is real and not a zero-byte file or a truncated
# write from a run that died halfway through
ls -lh /tmp/restore-drill/db/
head -n 20 /tmp/restore-drill/db/site.sql

# Load into a throwaway database and count what came back.
# If wp_posts is empty, your backups have been failing for weeks.
mysql -u drill -p drill_db < /tmp/restore-drill/db/site.sql
mysql -u drill -p -e "SELECT COUNT(*) FROM drill_db.wp_posts;"

Adjust the repository path and database names to your own. The shape is the point: restore somewhere harmless, then check a row count that would be non-zero on a healthy site. A backup you haven’t restored is a hypothesis, not a backup. Uptime checks from something like UptimeRobot tell you the site is down; they tell you nothing about whether you can get it back.

Patching debt compounds quietly

The second invisible cost is the gap between “updates are available” and “updates are applied and the affected services have actually restarted”. Upgrading a package without restarting the process leaves the old, vulnerable code resident in memory. That’s why security scanners and package managers disagree so often.

# RHEL family (AlmaLinux, Rocky, CentOS Stream)
# What security updates are outstanding?
sudo dnf updateinfo list security

# After applying them: does anything still need a restart?
# Exit 0 means no reboot required, exit 1 means reboot required.
sudo dnf needs-restarting -r

# Debian and Ubuntu: what would unattended-upgrades actually do?
sudo unattended-upgrade --dry-run --debug

Run those on any VPS you inherited from someone else. The output is usually educational. Then there’s the WordPress layer, which stays yours regardless of hosting model:

# Do the core files match the official checksums for this version?
# A mismatch means a modified core file or an active compromise.
wp core verify-checksums

# How many plugins are behind?
wp plugin list --update=available --format=count

# Which tables are eating the disk? Usually wp_options, wp_postmeta,
# or a logging plugin nobody remembers installing.
wp db size --tables --format=table

Model the cost before you argue about it

Sticker price comparisons are how people talk themselves into the wrong answer in both directions. This procedure produces a number you can defend.

1. Count your sites. One tilts toward managed. Five or more tilts toward a VPS, because admin time is close to fixed and plan fees are per-site.
2. Estimate your metered visit count from your own logs. If you’re already on a server you control, you can approximate what a metered host would bill you before signing anything.
3. Price the VPS honestly. Instance plus block storage plus offsite backup destination plus any control panel licence. The advertised instance price is rarely the whole line.
4. Put a rate on your own hours. Whatever you’d bill a client, or pay someone else. Zero is not an honest number.
5. Add the one-off build. Standing up a hardened WordPress VPS with TLS, firewall, object cache, tested backups and log rotation is a solid chunk of focused work the first time. Amortise it over twelve months.
6. Add your recovery cost. Hours to rebuild from scratch, times your rate, times how often you honestly think it’ll happen. This term is what makes managed hosting look cheap for a single revenue-generating site.

Step two is worth doing properly, because it’s the number nobody has. On Nginx with the standard combined log format, this gets you close:

# Unique client IPs in the log. A rough upper bound on metered "visits".
awk '{print $1}' /var/log/nginx/access.log | sort -u | wc -l

# Closer to how hosts count: ignore static assets, since most platforms
# exclude images, CSS and JS from billable visits.
awk '$7 !~ /.(css|js|png|jpg|jpeg|gif|svg|webp|woff2?|ico)$/ {print $1}' 
  /var/log/nginx/access.log | sort -u | wc -l

# What is actually hitting you? User agents by request count.
# If the top entries are crawlers, that is your overage risk.
awk -F'"' '{print $6}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head -20

Field positions assume the combined log format; adjust the column numbers if you’ve customised log_format. The third command is the interesting one. If crawlers dominate your traffic, visit-metered hosting will charge you for whatever the host’s filtering misses, and that’s a risk you can quantify before signing up rather than after.

The middle ground worth looking at first

This gets presented as two options. There are four, and the middle two get skipped far too often.

• Managed VPS. InterServer, Liquid Web and others sell VPS plans where the provider handles OS patching, firewall and backups while you keep root. Read the scope: “managed” often stops at the operating system and leaves the WordPress stack to you.
• Control panel on your own VPS. DirectAdmin, cPanel or CyberPanel gives you WordPress-aware tooling, staging and one-click TLS on infrastructure you still own. You pay a licence and give up some configuration purity.
• Server management platforms. Cloudways, RunCloud, GridPane and SpinupWP provision and manage a stack on a VPS you rent from a provider of your choice. Managed-style workflows without the per-visit meter.
• A CDN in front of anything. Putting Cloudflare or a similar edge ahead of a modest VPS absorbs a large share of the traffic that would otherwise push you onto a bigger plan. Cheapest performance win available to either model, and frequently the actual answer to “we need better hosting”.

If you’re leaning toward a VPS mainly because managed pricing looks steep, look at options two and three before committing to a bare Linux box.

How I’d decide between managed WordPress hosting vs VPS

These are the rules I apply, in order. The first match usually settles it.

• Nobody on the team can or wants to run Linux. Managed, and it isn’t close. An unpatched VPS is worse than the most restrictive managed plan.
• One site, and it generates revenue. Managed. The recovery term dominates everything else in the cost model.
• Five or more sites and someone competent to run them. VPS, or a management platform on top of one.
• You need services WordPress isn’t. VPS. Don’t pay for two platforms to avoid learning one.
• A plugin you can’t remove is on the host’s disallowed list. VPS, or a different managed host. Do not migrate hoping for an exception.
• Traffic is spiky and crawler-heavy. Lean VPS, or pick a bandwidth-metered plan over a visit-metered one. Metering punishes exactly this traffic shape.
• You’re mostly annoyed at the current bill. Neither, yet. Put a CDN in front, cut the plugin count, fix the object cache. Hosting is often not the bottleneck.

Arguments that don’t survive contact

• “A VPS is faster because you get dedicated resources.” Only if it’s configured well. A default LEMP install with no object cache and an untuned PHP-FPM pool loses to a managed platform’s edge cache every time.
• “Managed hosting handles security, so I’m covered.” It handles infrastructure security. The most common route into a WordPress site is a vulnerable plugin, and that’s yours in both models.
• “I’ll just move if I outgrow it.” Migration cost is real and grows with the site. Redirect rules, caching behaviour, deployment workflow and email configuration all need rebuilding.
• “The VPS is only a few dollars a month.” The instance is. Offsite backup storage, monitoring and your own hours are not, and they’re most of the number.
• “Managed hosts back everything up, so I don’t need my own backups.” Retention windows are finite and account access can be lost. Keep an independent copy the host doesn’t control. This applies to both models.

Frequently asked questions

Is a VPS actually cheaper than managed WordPress hosting?

For one site, usually not once you price your own time honestly. For several sites, usually yes, and the gap widens with each site you add. The crossover sits somewhere around three to five sites for most people, depending on what your hours are worth and how much the sites change.

Why does my managed host report more visitors than Google Analytics?

They measure different things. Analytics counts browser sessions that executed its JavaScript, so it misses anything blocking scripts and misses non-browser clients entirely. Your host counts requests reaching the server, deduplicated by IP per day. Bot filtering closes some of the gap but never all of it. The two numbers are not supposed to match.

Can I run other applications alongside WordPress on managed hosting?

Generally no. Managed WordPress platforms run WordPress and nothing else. A background worker, a separate API, a queue or a self-hosted analytics instance needs a VPS or a second platform. Factor that second bill into the comparison.

What happens if I install a disallowed plugin?

It depends on the host, but the documented pattern at WP Engine is that periodic filesystem scans detect it, you get notified, and the plugin is removed. Check the specific host’s current policy and enforcement mechanism before migrating, and check it against your live plugin list rather than what you assume is installed.

Do I still need a caching plugin on a VPS?

You need caching. Whether it’s a plugin is a design choice. The strongest setup is full-page caching at the web server or CDN layer plus a persistent object cache in Redis or Memcached, which handles the repeated database queries page caching can’t touch. A page cache alone leaves logged-in traffic and admin requests hitting the database on every load.

Is managed VPS hosting a real middle ground or just marketing?

It’s real, but the scope varies enormously and the word isn’t standardised. Ask specifically: who applies OS security patches, who restarts services afterwards, who owns backups and where they live, whether restores are tested, and whether the WordPress stack is in scope or the coverage stops at the operating system. Get the answers in writing before comparing prices.

How hard is migrating from managed hosting to a VPS?

Moving files and the database is the easy part. The work is everything the platform did invisibly: redirect rules living in the host’s dashboard, cache purging behaviour, transactional email, TLS renewal, and any deployment workflow tied to the platform. Plan a parallel run with low DNS TTLs rather than a cutover, and keep the old plan alive until the new one has survived a full traffic cycle.

The one thing worth remembering

Managed WordPress hosting vs VPS isn’t a performance question, and it isn’t really a price question either. It’s a question about which failures you’re equipped to own.

Managed hosting converts operational risk into a predictable bill and a set of constraints you have to live inside. A VPS converts that bill into flexibility plus a standing obligation to do the boring work: patch, restart, and prove your restores actually restore. Both are defensible. What isn’t defensible is buying a VPS because it looked cheap and then skipping the work, which is the most common failure in this whole category and the one that ends with a rebuild from a backup nobody ever tested.

Run the restore drill. Whatever you choose.

Need a second opinion on your hosting decision?

Most of the hosting work I get asked for is one of these. If it looks like your situation, I can help:

• Cost modelling before you commit. Working out from your own access logs what a visit-metered plan would actually bill you, against a realistically priced VPS including backup storage and hours.
• Migration in either direction. Managed to VPS or VPS to managed, planned as a parallel run with DNS TTL staging so there’s no blind cutover.
• Building the VPS properly the first time. Nginx or Apache with PHP-FPM sized to the actual RAM, Redis object cache, TLS with automated renewal, firewall and Fail2ban, log rotation that doesn’t fill the disk.
• Backups you’ve actually restored. Offsite destination, retention policy, and a scheduled restore drill that fails loudly instead of silently.
• Plugin policy audits. Checking your live plugin list against a specific host’s disallowed list before migration, and finding replacements for anything that fails.
• Monitoring that answers the right question. Grafana and Prometheus dashboards covering PHP-FPM saturation, database load and cache hit ratio, not just whether the site returns 200.

Send me something concrete and I’ll tell you what I see: an access log sample, your current plan and traffic numbers, your active plugin list, or the output of the commands above. That’s usually enough for a straight answer before either of us talks about scope.

The post Managed WordPress Hosting vs VPS: Who Owns the Failure? appeared first on John Nessime.

Six weeks later the Core Web Vitals report in Search Console still says the same URLs are failing. Nothing moved.

Here is why, and it is the single most important thing to understand before touching anything: the score and the assessment are two different measurements of two different things. The score is Lighthouse running a simulated page load on throttled hardware, once, from wherever you are. The assessment is Chrome telemetry from real visitors on real devices, aggregated over 28 days, graded at the 75th percentile.

Worse: Lighthouse cannot measure one of the three metrics at all. Interaction to Next Paint requires an actual human interacting with the page, so no lab tool produces it. You can score 100 and fail INP for every mobile visitor you have.

This walks through fixing Core Web Vitals on WordPress the way that actually shifts the report: get field data first, then work each metric in the order that pays.

Field data first, or you are guessing

Three facts that determine everything else.

You are graded at the 75th percentile. Three quarters of page views need a good result. Not the median, not your laptop. The fourth-slowest visitor out of four, typically a mid-range Android phone on patchy mobile data, decides whether you pass.

The window is 28 days and it rolls. Deploy a fix today and the report will not reflect it for weeks, and even then it moves gradually as old data ages out. Anyone promising you a same-week improvement in Search Console does not understand the mechanism.

Thin URLs fall back to origin-level data. If a page has too few visits for its own dataset, Google uses a group or your whole origin instead. So one badly performing template can drag URLs that are individually fine.

Practically: start in the Core Web Vitals report in Search Console to see which templates fail and how many URLs are affected. Then run PageSpeed Insights on a representative URL and read only the field data section at the top to decide what is broken. Use the Lighthouse section below it to work out why, which is what lab data is genuinely good for.

If you want faster feedback than 28 days, add real user monitoring. Google’s web-vitals JavaScript library reports all three metrics from your own visitors and you can send them wherever you like. On a site where performance matters commercially, this is worth the afternoon it takes.

LCP: the biggest thing on screen

Largest Contentful Paint is when the largest visible element finishes rendering. Good is under 2.5 seconds. On WordPress it is almost always the hero image or the first image in post content, and the causes are boringly consistent.

The lazy-loaded hero

Still the most common LCP problem on WordPress sites, years after core started handling it. Core skips lazy loading on the first image in content and adds fetchpriority="high" to the likely LCP candidate, but the heuristic only works on images core knows about. A hero rendered by a page builder, a theme header, or a slider often sits outside that, and some optimisation plugins re-add loading="lazy" to everything regardless.

# Look at what the server actually sends. If your hero image has
# loading="lazy" on it, you have found your LCP problem.
curl -s https://example.com/ | grep -o '<img[^>]*>' | head -5

For a theme-rendered hero, be explicit rather than hoping a heuristic catches it:

<!-- No loading="lazy". width and height reserve space, which
     also protects CLS. fetchpriority tells the browser this
     is the one image worth fetching before everything else. -->
<img src="/hero.webp"
     width="1600" height="900"
     fetchpriority="high"
     decoding="async"
     alt="">

Slow server response

LCP cannot be faster than time to first byte. On shared hosting with no page cache, an uncached WordPress response can eat most of your budget before a single byte of image is requested. Full-page caching is the single highest-yield change on most WordPress sites, and a CDN in front of it removes the geographic distance as well.

If TTFB is still poor with caching on, the cache is being bypassed. Cookies from analytics, a cart, or a logged-in session will do it. Check the cache-status header on a real request rather than trusting the plugin’s dashboard.

Render-blocking CSS

The browser will not paint until it has the stylesheets. A theme loading four separate CSS files plus a font stylesheet plus an icon library delays every LCP candidate on the page. Self-host fonts with font-display: swap, drop icon libraries you use six glyphs from, and be sceptical of “combine CSS” plugin features, which on HTTP/2 often make things worse rather than better.

Speculative loading, which is free

Worth knowing because most WordPress performance guides predate it. Since WordPress 6.8, core ships speculative loading using the Speculation Rules API: the browser starts fetching a link’s destination when the visitor interacts with it, so the next page is partly or entirely loaded before they arrive. It is on by default for logged-out visitors, using prefetch with “conservative” eagerness, and it is a progressive enhancement with no downside for browsers that do not support it.

You can push it further, at a cost:

add_filter( 'wp_speculation_rules_configuration', function ( $config ) {
    // Prerender actually renders the next page, including running
    // its JavaScript. Much faster, and analytics or anything with
    // side effects may fire for a visit that never happens.
    return array(
        'mode'      => 'prerender',
        'eagerness' => 'moderate',
    );
} );

Test that carefully on anything transactional. Core also honours no-prefetch and no-prerender CSS classes on individual links, which is the simplest way to keep carts, checkouts and logout links out of it.

INP: the hard one, and it is your JavaScript

Interaction to Next Paint measures the time from a user interacting to the browser painting the response, across every interaction on the page, and reports roughly the worst one. Good is under 200 milliseconds; above 500 is poor.

It replaced First Input Delay in March 2024, and the change matters more than a metric swap sounds. FID measured only the delay before processing the first interaction. INP measures the whole round trip, every time. Plenty of sites that comfortably passed FID fail INP, because the thing that was always sluggish is now the thing being graded.

Since no lab tool reports it, use Total Blocking Time in Lighthouse as a proxy, and Chrome DevTools’ Performance panel to find which handler is blocking. On WordPress the usual suspects:

• Plugin JavaScript loading site-wide. A form builder, a slider and a popup tool each shipping their bundle on every page, including pages that use none of them.
• Third-party tags. Tag managers, chat widgets, heatmaps, ad scripts. These run on the main thread and are usually the largest single contributor.
• Long tasks in event handlers. Anything doing real work synchronously in response to a click.
• jQuery-era code attaching handlers to hundreds of elements on a listing page.

The fixes, in order of yield. First, stop shipping JavaScript to pages that do not use it. Conditionally dequeue plugin assets rather than accepting that a contact form script belongs on every article. This alone often solves INP.

Second, audit the third-party tags. Every one is somebody’s request from two years ago, and at least one of them is for a tool nobody uses any more. Removing a tag is the cheapest performance work available.

Third, break up long tasks so the browser can paint between chunks:

// Yielding lets the browser respond to the click that started this,
// instead of finishing all the work first. INP measures exactly
// that gap, so yielding moves the metric even though total work
// stays the same.
async function processAll(items) {
  for (const item of items) {
    doWork(item);
    await new Promise(resolve => setTimeout(resolve, 0));
  }
}

Newer Chrome versions offer scheduler.yield(), which does the same thing without losing your place in the task queue. Feature-detect it and fall back to the above.

One honest warning: “delay JavaScript until user interaction” features in optimisation plugins look like an INP fix and frequently are not. Delaying scripts until the first interaction means the first interaction now waits for all of them to load and execute, which is the worst possible moment. It improves lab scores and can make INP worse.

CLS: usually an afternoon

Cumulative Layout Shift measures unexpected movement of visible content. Good is under 0.1, and it is the metric most likely to be fixable in one sitting.

• Images without dimensions. Set width and height so the browser reserves space before the file arrives. WordPress does this for media library images; hand-written theme markup often does not.
• Ads and embeds. Reserve a fixed-height container. An ad slot that collapses to nothing and then expands is a large shift on every page view.
• Fonts. A web font swapping in at a different metric reflows everything. font-display: swap plus size-adjust on the fallback, or a system font stack, both work.
• Cookie banners and notices injected by JavaScript that push the page down rather than overlaying it.
• Lazy-loaded content above the fold that appears after paint.

Chrome DevTools’ Performance panel highlights layout shifts with the element that moved, which turns this from detective work into a checklist.

The order I’d work in

1. Anything in the “poor” band, whichever metric it is. Poor hurts more than two mediocre metrics.
2. Caching and TTFB, because it is one change and it moves LCP for every page at once.
3. The LCP image on your highest-traffic template.
4. JavaScript you can delete. Unused plugin assets and dead third-party tags, for INP.
5. CLS, which is quick and visible to users even where the number already passes.
6. Everything else, measured one change at a time.

What I would skip: chasing a green Lighthouse score once field data passes, stacking a second optimisation plugin on top of the first, and micro-optimising a metric that is already comfortably good. There is no bonus for 1.1 seconds when 2.5 is the bar.

When the numbers don’t move

It has been two weeks. That is normal. The window is 28 days and rolling, so give it a full month before concluding anything.

Lab improved, field did not. You optimised something the lab measures and users do not experience, or the problem is INP, which the lab never measured. Go back to the field data section.

Desktop passes, mobile fails. Expected, and mobile is what is graded for most sites. Test with CPU throttling on, because the gap is usually processing rather than network.

One URL is fine but the group fails. Origin-level fallback, or a different template in the same group is dragging it. Check what else Search Console has bundled together.

It regressed after a plugin update. Very common, and the reason to record your field numbers before and after any significant change rather than relying on memory.

Common mistakes

• Treating the PageSpeed Insights score as the thing being measured.
• Optimising INP with a lab tool that cannot measure INP.
• Testing from a fast laptop on fibre and concluding the site is fine.
• Lazy-loading the hero image, or letting a plugin re-add it.
• Running two optimisation plugins that both minify and both defer.
• Enabling “delay JavaScript until interaction” and expecting INP to improve.
• Judging a fix after a week.
• Loading every plugin’s JavaScript on every page.
• Adding a CDN while the origin has no page cache.
• Fixing the homepage when the failing template is the blog archive.

Best practices

• Decide what is broken from field data; work out why from lab data.
• Fix by template, not by URL, since one template covers thousands of pages.
• Full-page caching first, then a CDN, then image work.
• Explicit width and height on every image in theme markup.
• fetchpriority="high" on the LCP image and lazy loading on nothing above the fold.
• Dequeue plugin assets on pages that do not use them.
• Review third-party tags on a schedule and delete the ones nobody owns.
• Self-host fonts with font-display: swap.
• Real user monitoring so you are not waiting a month for feedback.
• Record field numbers before and after every significant change.

FAQ

Why is my PageSpeed score 95 but Search Console says I’m failing?

They measure different things. The score is a simulated load on your connection; the assessment is real Chrome users at the 75th percentile over 28 days. Read the field data section at the top of PageSpeed Insights, not the score.

How long before a fix shows up?

The data window is 28 days and rolls daily, so improvement appears gradually and takes about a month to fully reflect. Real user monitoring gives you same-day feedback if you cannot wait.

Will a caching plugin fix Core Web Vitals?

It will help LCP substantially, because TTFB is a floor under it. It does very little for INP, which is about JavaScript execution, and nothing for CLS. Caching is a strong first move and not a complete answer.

What happened to FID?

INP replaced it in March 2024 and FID is retired. Any guide still listing it predates the change, which is a useful signal about the rest of its advice.

Do I need to switch themes?

Sometimes, and it is a bigger project than it looks. A heavy multipurpose theme loading a page builder, a slider library and an icon set on every request is fighting you. Measure first, because a theme swap on a site with hundreds of pages is rarely the cheapest route to passing.

Does passing actually help rankings?

Core Web Vitals are a ranking signal among many, and content relevance dominates. The reliable win is on the user side: faster pages convert better and get abandoned less. Treat ranking as a bonus rather than the business case.

The one thing to remember

You cannot fix what you are not measuring, and the tool most people measure with cannot see the metric most people are failing. Lighthouse is a debugger. The Chrome User Experience Report is the exam.

So open Search Console before you open PageSpeed Insights, find out which of the three is actually red, and fix that one. Then wait a month before deciding whether it worked, because the measurement takes longer than the work.

Site failing and not sure why?

Most of the WordPress sites I get sent have already had a fortnight thrown at the wrong metric. Work I take on:

• Diagnosing which Core Web Vital is actually failing, for which templates, from field data rather than a score.
• LCP work: caching and TTFB, image priority and formats, render-blocking CSS, CDN configuration.
• INP work, which is mostly JavaScript archaeology: dequeuing plugin assets by page, auditing third-party tags, breaking up long tasks.
• CLS fixes across templates, including ads, embeds, fonts and consent banners.
• Setting up real user monitoring so you get feedback in a day instead of a month.
• Untangling sites running two or three overlapping optimisation plugins.

Send me your Search Console Core Web Vitals report and a URL from a failing template, and I will tell you what is actually wrong.

The post A 100 Score and a Failing Report: Fixing Core Web Vitals on a Real WordPress Site appeared first on John Nessime.

Your server is only sending the leaf certificate. The intermediate that links it to a trusted root is missing. Your browser papers over that: it either has the intermediate cached from another site, or it quietly fetches it using the AIA extension in the certificate. curl, Java, Go, Python and every other non-browser client do no such thing. They see a certificate signed by something they do not trust and stop.

This is the most common of all TLS certificate errors and it is close to invisible from a desktop. The padlock in your browser is not a test. It is a client with a large safety net doing you a favour.

This post covers the four things a client actually checks and how each one fails: the chain, the names, the clock, and the trust store. Then the automation half, which has its own quiet failure: a renewal that succeeds and never reaches the running server. Plus the two things about TLS that have genuinely changed recently and will break config you copied from an older guide.

Look at what the server is actually sending

Before theorising, get the facts. One command tells you almost everything.

# -servername sends SNI. Without it you get whatever the default
# vhost serves, which is a different certificate on most servers
# and the source of a lot of confusing results.
# -showcerts prints every certificate the server sent, in order.
openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null

Read the top of that output. There is a Certificate chain block listing what arrived, numbered from zero, with s: for subject and i: for issuer. Entry 0 is your certificate. Entry 1 should be the intermediate that issued it, and its subject should exactly match entry 0’s issuer.

If the chain block has only entry 0, that is your bug. Right at the bottom, Verify return code gives the verdict: 0 (ok) is what you want, 21 (unable to verify the first certificate) means the chain is incomplete, and 20 (unable to get local issuer certificate) means the issuer is not in the local trust store.

Run it from a machine that is not yours. A clean container is ideal, because your laptop has accumulated intermediates and corporate roots that a customer’s server has not.

Chain problems

Almost all of these come down to one file choice. ACME clients write out several files and two of them look interchangeable:

• cert.pem is the leaf alone. Using this is the mistake.
• chain.pem is the intermediates alone.
• fullchain.pem is leaf plus intermediates, in the right order. This is the one your server should point at.

# nginx
ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

# Apache. On current versions the chain goes in SSLCertificateFile.
# SSLCertificateChainFile is deprecated; guides that still use it
# are old enough that other advice in them is probably stale too.
SSLCertificateFile      /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile   /etc/letsencrypt/live/example.com/privkey.pem

Order matters as well as contents. The leaf must come first, then each intermediate that signs the one above it. A chain assembled in the wrong order will be rejected by strict clients even though every certificate in it is correct. You can check a chain offline before deploying it:

# Does this leaf chain to a trusted root using these intermediates?
# Exits non-zero and explains itself if not.
openssl verify -untrusted chain.pem cert.pem

Do not include the root certificate in the file you serve. The client either has it or does not, and sending it wastes bytes on every handshake without changing any outcome.

Name problems

The second family. A certificate can be perfectly valid and still be the wrong certificate for the name being requested.

# The hostname the client asked for must appear in here.
openssl x509 -in cert.pem -noout -text | grep -A1 "Subject Alternative Name"

Three things people get wrong here:

• The Common Name does not count. Modern clients ignore the CN field entirely for hostname matching. If the name is only in the CN and not in the SAN list, it will fail, and the error message will not explain that clearly.
• Wildcards are exactly one label deep. *.example.com covers www.example.com and api.example.com. It does not cover example.com itself, and it does not cover a.b.example.com. The bare domain has to be listed separately, and it usually is not.
• SNI decides which certificate you get. One IP serving many sites picks the certificate based on the name the client sends during the handshake. A client that does not send SNI, or an openssl command without -servername, gets the default vhost’s certificate and produces a name mismatch that has nothing to do with your actual configuration.

Time and trust problems

# Local file
openssl x509 -in cert.pem -noout -dates

# Live endpoint, subject and dates in one go
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null 
  | openssl x509 -noout -dates -subject

Expired is obvious. notBefore in the future is the interesting one, because it almost never means the certificate is wrong. It means the client’s clock is wrong. Containers with no NTP, a VM restored from a snapshot, an embedded device with a dead battery: all of them produce “certificate not yet valid” against a perfectly good certificate. Check the clock on the failing machine before you touch anything on the server.

Trust store problems look similar but come from a different place. The root that signed your chain has to be in the trust store of the machine making the request, and old systems have old trust stores. A device that has not received an OS update in years may simply not know about a certificate authority that appeared since. There is no server-side fix for that beyond choosing a CA with broader legacy coverage, which is one of the few genuine arguments for a paid certificate.

The renewal that succeeded and changed nothing

Here is the second invisible failure, and it catches experienced people. Your renewal timer runs, the ACME client obtains a new certificate, writes it to disk, logs success. The site goes down anyway.

Nginx read the certificate into memory when it started. It does not watch the file. Neither does Apache, HAProxy, Postfix, or almost anything else. Until the service is reloaded it keeps serving the certificate it loaded at boot, which is the one that just expired. Everything in your logs says the renewal worked, because it did. The delivery step was never wired up.

Certbot handles this with deploy hooks, which run only when a certificate was actually renewed. Put a script in the hooks directory and it applies to every certificate on the box:

#!/usr/bin/env bash
# /etc/letsencrypt/renewal-hooks/deploy/reload-services.sh
# Remember: chmod +x, or it silently never runs.
set -euo pipefail

# Validate the config BEFORE reloading. A reload with a broken
# config can take the site down at the exact moment you were
# trying to keep it up.
nginx -t
systemctl reload nginx

# Anything else that holds a copy of the certificate.
# systemctl reload postfix

Then prove the whole path works without waiting sixty days to find out:

# Exercises the full renewal against the staging environment.
# Installs nothing, counts against no rate limits, and will
# fail loudly if your challenge setup is broken.
certbot renew --dry-run

One caveat worth knowing: --dry-run does not run deploy hooks by default, precisely because nothing was deployed. So it validates the issuance path but not the reload. Test that separately by touching the service and confirming the served certificate changes.

Expiry automation, and why it is no longer optional

This part has a date attached, so it is worth stating plainly rather than dancing around it.

The CA/Browser Forum approved a phased reduction of maximum certificate lifetimes. The cap dropped from 398 days to 200 days on 15 March 2026, falls to 100 days on 15 March 2027, and reaches 47 days on 15 March 2029. Domain validation reuse periods shrink alongside it.

The practical consequence: any process that involves a person remembering to renew something once a year is already broken, and by the third phase you are renewing every six weeks. Whatever you were getting away with manually, stop. ACME with a timer is the baseline. Certbot is the default choice, acme.sh is lighter and easier to script around, and Caddy obtains and renews certificates with no configuration at all, which is a genuinely good reason to consider it for a simple reverse proxy.

Choosing a challenge type

• HTTP-01 proves control by serving a file over port 80. Simple, and it breaks whenever port 80 is closed, redirected wholesale to HTTPS in a way that mangles the challenge path, or sitting behind a proxy that does not forward it.
• DNS-01 proves control by writing a TXT record. More setup, needs an API token for your DNS provider, and it is the only option for wildcard certificates. It also works for hosts with no inbound internet access at all, which is why I reach for it on anything internal.

The trade-off with DNS-01 is a credential: your renewal process now holds a token that can edit your DNS zone. Scope it to the narrowest permission your provider offers, and if they support delegating just the _acme-challenge records to a separate zone, do that instead.

Monitor expiry externally

Do not rely on the renewal system reporting its own health. If it is broken, it is also probably not telling you. Let’s Encrypt has discontinued its expiration reminder emails, so if that was quietly acting as your safety net, it is not one any more.

Check from outside, against the live endpoint, the way a user would:

#!/usr/bin/env bash
set -euo pipefail

HOST="example.com"
DAYS=21

# -checkend takes a number of seconds and exits non-zero if the
# certificate expires within that window. No parsing dates by hand.
if ! openssl s_client -connect "$HOST:443" -servername "$HOST" </dev/null 2>/dev/null 
     | openssl x509 -noout -checkend $(( DAYS * 86400 )); then
  echo "TLS certificate for $HOST expires within $DAYS days" >&2
  exit 1
fi

If you already run Prometheus, the blackbox exporter gives you probe_ssl_earliest_cert_expiry for free, which is a timestamp you can alert on and graph in Grafana across every endpoint you own. That is the better answer at more than a handful of hosts, because it also catches the certificates nobody remembered existed.

Two pieces of config that are now wrong

OCSP stapling with Let’s Encrypt. Let’s Encrypt removed the OCSP responder URL from newly issued certificates in May 2025 and shut down its responders in August 2025. If your config still has this, it does nothing except print a warning on every reload:

# nginx: [warn] "ssl_stapling" ignored, no OCSP responder URL in the certificate
# Comment these out if your certificates come from Let's Encrypt.
# ssl_stapling on;
# ssl_stapling_verify on;

Nothing breaks by leaving them, but a warning that appears on every reload is a warning people learn to ignore, and then they ignore the next one too.

Hand-tuned cipher suite lists. If you pasted a ssl_ciphers line from a guide more than a couple of years old, re-check it against the current Mozilla SSL Configuration Generator. Lists that were considered hardened at the time now sometimes exclude things you want or include things you do not, and the defaults have improved considerably.

When there is a CDN in front

Behind Cloudflare or a similar edge, there are two separate TLS connections: browser to edge, and edge to your origin. They use different certificates and fail independently.

This creates a specific blind spot. Your origin certificate can be expired, self-signed or serving the wrong name, and visitors will see nothing wrong, because the edge presents its own valid certificate and, in the more permissive encryption modes, does not verify the origin. It works until you switch to strict origin validation, or until the edge is bypassed, and then everything fails at once for reasons that look unrelated to anything you changed.

Test the origin directly by name, bypassing DNS. If your origin is a VPS with a provider like InterServer and the public DNS points at the CDN, this is the only way to see what the origin is really serving:

# --resolve forces the hostname to a specific IP without touching
# /etc/hosts, so SNI and Host header stay correct.
curl -v --resolve example.com:443:203.0.113.10 https://example.com/

Troubleshooting by error message

unable to get local issuer certificate

Missing intermediate on the server, or a trust store on the client that lacks the root. Check the server first with -showcerts; if the chain is complete, the problem is on the client side.

certificate has expired, but I just renewed it

The service was never reloaded. Compare the dates on disk with the dates being served. If they differ, that is your answer, and the fix is a deploy hook rather than another manual renewal.

Hostname mismatch on a name that is definitely in the certificate

Either SNI is not reaching the right vhost, or the name is in the Common Name rather than the SAN list, or you are hitting a wildcard at the wrong depth. Check the SAN list and re-run s_client with an explicit -servername.

Works over IPv4, fails over IPv6 (or the reverse)

Two different machines are answering, and only one of them was updated. Force the family with curl -4 and curl -6 and compare. Load balancers with one member out of sync produce the same intermittent pattern.

Renewal fails with a challenge error

For HTTP-01, confirm that http://example.com/.well-known/acme-challenge/test is reachable from outside without being redirected or rewritten. A blanket redirect to HTTPS is the usual culprit. For DNS-01, check that the TXT record actually appears with dig, and give slow-propagating providers more time.

Everything looks right and it still fails

Run an external scan. SSL Labs or a local run of testssl.sh will check things you did not think to, including protocol support and chain issues from the perspective of clients you do not have installed.

Common mistakes

• Pointing the server at cert.pem instead of fullchain.pem.
• Treating a padlock in your own browser as proof the certificate is correctly deployed.
• Testing with openssl s_client without -servername and debugging the wrong certificate.
• Assuming *.example.com covers the bare domain.
• Putting the hostname only in the Common Name.
• Renewing without reloading the service.
• Forgetting chmod +x on a deploy hook, so it silently never runs.
• Relying on the renewal system to tell you when it is broken.
• Including the root certificate in the served chain.
• Leaving OCSP stapling enabled with certificates that no longer support it.
• Ignoring the origin certificate because the CDN hides its state.
• Debugging a “not yet valid” certificate on the server when the client’s clock is wrong.

Best practices

• Serve the full chain, leaf first, without the root.
• Verify from a clean machine or container, never from the laptop you browse on.
• Put every name you serve in the SAN list, including the bare domain alongside any wildcard.
• Automate renewal with ACME and a timer, and run certbot renew --dry-run after any change to the setup.
• Attach a deploy hook that validates the config before reloading, and make it executable.
• Monitor expiry externally against the live endpoint, and alert with weeks of headroom.
• Use DNS-01 for wildcards and internal hosts, with a narrowly scoped API token.
• Keep NTP running everywhere, including inside containers.
• Test the origin directly when there is a CDN in front, not just the public hostname.
• Inventory your certificates, because the one that takes you down is the one nobody remembered.

FAQ

Why does my certificate work in Chrome but fail in curl?

Your server is not sending the intermediate certificate. Browsers work around this by caching intermediates from other sites and by fetching them from the URL in the certificate’s AIA extension. Command-line tools and application HTTP libraries do neither. Switch the server to fullchain.pem and reload.

What is the difference between cert.pem, chain.pem and fullchain.pem?

Leaf only, intermediates only, and both concatenated in the correct order. Web servers want fullchain.pem. The private key stays in privkey.pem and never goes in any of them.

Does a wildcard certificate cover the root domain?

No. *.example.com matches exactly one label, so it covers www.example.com but not example.com and not a.b.example.com. Request the bare domain as an additional SAN, which most ACME clients will do if you pass both names.

How often should certificates renew now?

Whatever your CA issues, renew at roughly two-thirds of the lifetime so a failed attempt has room to retry before anything expires. Since maximum lifetimes are on a published downward schedule, build for automated renewal at any interval rather than tuning a number you will have to change again.

Should I still enable OCSP stapling?

Not with Let’s Encrypt certificates, which no longer carry a responder URL. If your CA still supports it, stapling remains worthwhile. Check whether your certificate actually contains an OCSP URL before enabling the directives, rather than assuming.

Is a paid certificate more secure than a free one?

Cryptographically, no. The encryption is identical. What money buys is broader compatibility with old trust stores, longer validation reuse, warranties, and human support when something goes wrong. For a normal public website, a free automated certificate is the right default.

How do I get a certificate for something with no public DNS or open ports?

DNS-01, on a public domain you control, pointed at whatever internal address you like. The CA only needs to see the TXT record; it never needs to reach the host. This is how internal services get publicly trusted certificates without being exposed.

The one thing to remember

Most TLS certificate errors are not about cryptography. They are about delivery: whether the server sends everything the client needs, whether the name it sends matches the name that was asked for, and whether the certificate sitting in memory is the same one sitting on disk.

So stop checking with your browser. Check with openssl s_client from a machine that owes you no favours, and make sure something outside your server is watching the expiry date. As lifetimes keep shrinking, the gap between a renewal process that works and one that merely appears to work gets narrower every year.

Certificates causing you trouble?

Certificate problems tend to arrive at bad moments and turn out to be one wrong file path. Things I help with:

• Diagnosing a certificate that works for some clients and not others, and telling you exactly which link in the chain is missing.
• Setting up ACME renewal with deploy hooks across nginx, Apache, HAProxy, mail servers and anything else holding a copy.
• Migrating to DNS-01 for wildcards and internal hosts, with properly scoped API credentials.
• Certificate inventory and expiry monitoring in Prometheus and Grafana, so nothing expires without warning.
• Preparing a certificate estate for shorter lifetimes, replacing anything that still depends on a person remembering.
• Origin and edge TLS configuration behind a CDN, including strict origin validation.

Send me the output of openssl s_client -connect yourhost:443 -servername yourhost -showcerts and the error your failing client reports, and I will tell you what is wrong.

The post It Works in My Browser: Debugging TLS Certificate Errors in Chains, SANs and Expiry appeared first on John Nessime.

That is the failure mode worth taking seriously. Not slowness, not downtime — the CDN cached a page that was personalised for one user and served it to everybody else. And you will not notice, because your browser holds a session and looks fine. The people seeing the broken version are the ones who never contact you.

Getting CDN setup right for a dynamic site is not about finding the right toggle in a dashboard. It is about deciding, per route, what is public and what is personal, and then making the origin say so clearly enough that no edge node has to guess.

Sort your routes before touching any config

Every URL on your site falls into one of three buckets. Write the list out — actually write it, this takes ten minutes and prevents the incident above.

• Static assets. CSS, JS, images, fonts. Identical for everyone, content-addressed if your build is sensible. Cache these hard, for a year.
• Public dynamic. Generated by the application but identical for every anonymous visitor. Blog posts, product pages, category listings. This is where the real performance win lives, and where most people leave everything uncached out of nervousness.
• Private dynamic. Anything personalised or authenticated. Carts, dashboards, checkout, admin, most API responses. Never cached at a shared cache, no exceptions.

The middle bucket is the whole game. If you cache only the first bucket you have built an asset CDN and left the expensive work untouched. If you accidentally cache the third, you have a data leak.

The header that separates the buckets

The single most useful thing to understand is the difference between private and no-store, because people use them interchangeably and they mean very different things.

# Static assets with a content hash in the filename
Cache-Control: public, max-age=31536000, immutable

# Public dynamic: edge caches it, browser revalidates
Cache-Control: public, s-maxage=300, max-age=0, must-revalidate

# Personalised: browser may cache, shared caches must not
Cache-Control: private, no-store

# Genuinely sensitive: nobody stores this anywhere
Cache-Control: no-store

private means “a browser cache may keep this, a shared cache may not”. no-store means “do not write this to disk anywhere”. For an authenticated dashboard you want both. For a bank statement you want no-store alone.

The other key pair is s-maxage versus max-age. s-maxage applies only to shared caches — the CDN — and overrides max-age there. That lets you cache a product page at the edge for five minutes while telling browsers to revalidate every time, so a price change propagates in five minutes rather than living in someone’s browser for an hour.

Set caching headers at the origin, per route, and let the CDN follow them. Configuring cache rules only in the CDN dashboard means the policy lives somewhere your application code cannot see and your staging environment does not have.

Vary, and how it quietly ruins your hit rate

Vary tells caches that the response depends on a request header, so a separate copy is stored per distinct value. Used precisely it is essential. Used carelessly it destroys your cache.

# Fine. A handful of encodings, a handful of variants.
Vary: Accept-Encoding

# Also fine if you genuinely serve different content per language
Vary: Accept-Encoding, Accept-Language

Now the one to avoid:

# Effectively disables caching. Every browser build is a distinct
# User-Agent string, so every visitor gets their own cache entry.
Vary: User-Agent

If you need device-specific responses, most CDNs expose a normalised device-type value with two or three possible values. Use that rather than the raw header.

Vary: Cookie deserves its own warning. It looks like a safe way to separate logged-in from anonymous traffic, and it is not — every visitor carries slightly different cookies from analytics and consent tools, so you get a near-unique cache key per person. The correct approach is to make the presence of a session cookie bypass the cache entirely, rather than varying on it.

Cookies are where sites actually break

Two rules, and between them they prevent most CDN incidents.

Rule one: a session cookie means bypass

If a request arrives carrying your session cookie, it should go to origin uncached. Most CDNs let you express this as a cache rule; in nginx, if you are running your own edge, it is a variable:

map $http_cookie $bypass_cache {
    default                 0;
    "~*wordpress_logged_in" 1;
    "~*woocommerce_items"   1;
    "~*comment_author"      1;
    "~*PHPSESSID"           1;
}

location / {
    proxy_cache            edge_cache;
    proxy_cache_bypass     $bypass_cache;   # do not serve from cache
    proxy_no_cache         $bypass_cache;   # do not write to cache
    proxy_cache_valid      200 301 302 5m;
    proxy_cache_use_stale  error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_lock       on;

    add_header X-Cache-Status $upstream_cache_status always;
    proxy_pass http://origin;
}

You need both directives. proxy_cache_bypass stops a logged-in user being served a cached anonymous page. proxy_no_cache stops their personalised response being written into the cache for everyone else. Setting only the first is a common and dangerous mistake — the read path is protected while the write path is wide open.

That X-Cache-Status header is not decoration. It is how you verify any of this actually works.

Rule two: strip cookies you do not need

Analytics and consent tools set cookies on every visitor. If your CDN treats any cookie as a bypass signal, your anonymous traffic stops being cacheable the moment you add a tag manager. Configure the CDN to ignore everything except the specific cookies that matter, and be explicit about which those are.

Query strings, and the cache-key explosion

By default many CDNs include the full query string in the cache key. Then a marketing campaign starts appending utm_source, utm_medium, fbclid and friends, and one page becomes thousands of cache entries that each miss exactly once.

Decide deliberately which parameters affect the response:

• Include parameters that change the content — ?page=2, ?sort=price, ?category=boots.
• Ignore tracking parameters entirely. They never change what the server renders.
• Normalise order where the CDN supports it, so ?a=1&b=2 and ?b=2&a=1 are one entry rather than two.

An allowlist is safer than a blocklist here. New tracking parameters appear constantly; the set of parameters your application actually reads changes rarely.

Keeping dynamic pages cached without going stale

The usual objection to caching public dynamic pages is that content changes. Two mechanisms solve this, and together they let you cache aggressively without serving stale content for long.

Stale-while-revalidate

Cache-Control: public, s-maxage=60, stale-while-revalidate=600, stale-if-error=86400

Read that as three separate promises. Fresh for sixty seconds. For the following ten minutes, serve the stale copy immediately while fetching a fresh one in the background — so nobody waits. And if the origin is throwing errors, keep serving the stale copy for a day rather than showing a 502.

That last directive turns your CDN into a partial outage shield. It is one of the highest-value headers on this page and it costs nothing.

Purge on change

Time-based expiry is a blunt instrument. If your application knows when a product price changed, tell the CDN immediately rather than waiting for a TTL.

Tag-based purging is the version worth building toward — tag responses by the entities they contain, then purge by tag when an entity changes:

# Origin response for a product page
Cache-Tag: product-1234, category-boots, layout-main

Update product 1234 and you purge one tag, invalidating the product page, every category listing it appears on, and the homepage if it is featured — without knowing any of those URLs in advance. Header name and support vary by provider, so check what yours calls it.

Verifying it before you point DNS

Test with curl, not a browser. Your browser has cookies, a warm local cache and an opinion; curl tells you the truth.

# What headers is the origin actually sending?
curl -sI https://example.com/products/boots | grep -i 'cache-control|vary|set-cookie'

# Twice in a row: second request should show a HIT
curl -sI https://example.com/products/boots | grep -i 'x-cache|age'
curl -sI https://example.com/products/boots | grep -i 'x-cache|age'

Now the test that matters most. Request an authenticated page with a real session cookie and confirm it never returns a cache hit:

curl -sI https://example.com/account 
  -H "Cookie: session=<a-real-session-value>" 
  | grep -i 'x-cache|cache-control'

You want BYPASS or MISS, and Cache-Control: private. If that ever shows HIT, stop and fix it before going further.

The leak test, which is the one people skip: fetch a personalised page with a session, then fetch the same URL with no cookies at all and confirm you get the anonymous version.

# With a session
curl -s https://example.com/account -H "Cookie: session=<value>" | grep -i 'welcome back'

# Immediately after, with nothing. Should NOT show the personalised content.
curl -s https://example.com/account | grep -i 'welcome back'

If the second command finds personalised content, the CDN cached a private response. That is the Monday-morning email, and you have just caught it before your customers did.

Test the edge before DNS points at it

curl -sI https://example.com/ --resolve example.com:443:<edge-ip>

That sends a real request to a specific edge node with the correct hostname and TLS SNI, without changing DNS. It is how you find out whether the whole thing works before any user does.

Things that break in ways nobody expects

Client IPs all become the CDN’s

Your logs fill with edge node addresses, rate limiting starts throttling everyone at once, and geolocation breaks. The real address arrives in a forwarded header, and your origin has to be configured to trust it — but only from the CDN’s address ranges. Trusting X-Forwarded-For from anywhere lets any visitor claim any IP they like, which turns a logging annoyance into an authentication bypass.

Redirect loops

The CDN talks to your origin over HTTP, the origin sees a non-HTTPS request and redirects to HTTPS, which comes back through the CDN, forever. Either terminate TLS at the origin too, or have the origin trust X-Forwarded-Proto when deciding whether to redirect.

POST responses getting cached

Shouldn’t happen by default, but a badly written cache rule can do it, and the result is one user’s form submission response served to the next person. Only cache GET and HEAD. Ever.

Websockets and streaming

Websocket upgrades and server-sent events need to pass through untouched. Buffering breaks streaming responses silently — the connection works, the data just arrives in one lump at the end, which looks like an application bug rather than a proxy setting.

Set-Cookie on a cacheable response

If a public page returns Set-Cookie and gets cached, every subsequent visitor receives that cookie — potentially somebody’s session identifier. Most CDNs refuse to cache responses carrying Set-Cookie, but not all, and not always. Check your origin is not setting cookies on pages you intend to cache.

Rolling it out without a bad week

1. Write down the route list. Three buckets, every path accounted for.
2. Set headers at the origin first, with the CDN not yet in front. Verify with curl.
3. Start with everything bypassed except static assets. A CDN that caches nothing still terminates TLS closer to users and is not a failure.
4. Add public dynamic routes one at a time, with short TTLs. Sixty seconds, then extend once you trust it.
5. Run the leak test after every rule change, not just at the end.
6. Watch origin request volume. It should drop noticeably. If it does not, something is bypassing that you think is caching.
7. Keep a purge command to hand and know how long a full purge takes before you need it.

Common mistakes

• Testing only in a logged-in browser, where cached-private-content bugs are invisible.
• Setting proxy_cache_bypass without proxy_no_cache, protecting reads but not writes.
• Vary: Cookie or Vary: User-Agent, which shatters the cache into near-unique entries.
• Leaving tracking parameters in the cache key.
• Configuring caching in the CDN dashboard only, so the policy is invisible to your code and absent from staging.
• Trusting forwarded IP headers from any source rather than only the CDN’s ranges.
• Using private where you needed no-store, or the reverse.
• Caching everything on day one and finding the problems in production.

Best practices

• Default to no caching, then allow specific routes. Opt-in beats opt-out when the failure mode is a data leak.
• Own the policy at the origin so it travels with your code and exists in staging.
• Use s-maxage for the edge and max-age for browsers. They are different problems.
• Always set stale-if-error. It is free outage insurance.
• Expose a cache status header and monitor hit ratio per route, not site-wide.
• Build purge-on-publish into the application rather than relying on TTLs alone.
• Make the leak test part of CI, so a future config change cannot reintroduce it quietly.

Frequently asked questions

Can a CDN cache dynamic pages at all?

Yes, and that is where most of the benefit is. Dynamic does not mean personalised. A blog post rendered from a database is identical for every anonymous visitor and can be cached at the edge for minutes. Only genuinely per-user content must bypass.

What is the difference between private and no-store?

private permits browser caching but forbids shared caches. no-store forbids writing the response to any cache at all. Use both for authenticated pages; use no-store alone for genuinely sensitive responses.

Why did my hit ratio collapse after adding a cookie banner?

Because the consent tool sets a cookie on every visitor and your CDN treats any cookie as a bypass signal, or you are varying on Cookie. Configure the CDN to ignore all cookies except the specific ones that indicate a session.

How do I verify private pages are not being cached?

Request the page with a real session cookie and confirm the cache status shows a bypass or miss, never a hit. Then request the same URL with no cookies and confirm the personalised content is absent. Do this after every cache rule change.

Should I cache API responses?

Public read-only endpoints, yes — with short TTLs and a cache key that includes whatever parameters affect the response. Anything requiring an Authorization header should be treated as private by default.

Do I still need caching at the origin?

Yes. The CDN protects you from repeat requests for the same URL; it does nothing for the first request, for uncacheable routes, or during a purge. Origin-level caching handles those.

Wrapping up

A CDN in front of a dynamic site is a classification problem wearing a performance costume. Sort your routes into public, private and static; express that at the origin with Cache-Control; make session cookies a hard bypass on both the read and write path; and keep tracking parameters out of the cache key.

Then test the way an anonymous visitor arrives, not the way you browse your own site. Two curl commands — one with a session, one without — catch the failure that matters, and they take about ten seconds. Run them every time you change a cache rule and the Monday email never arrives.

Want a CDN set up without the surprises?

I work with teams on edge caching for dynamic sites — route classification, cache headers that survive a redesign, purge-on-publish wired into the application, and the monitoring that tells you when hit ratio quietly drops.

If you would rather have this done carefully the first time, you can hire me on Upwork.

The post CDN Setup Without Breaking Dynamic Content appeared first on John Nessime.

Nearly every TLS certificate error I have had to chase down comes back to one of three things: the chain your server sends is incomplete, the certificate does not actually cover the hostname being requested, or it expired and no alert fired. This article walks through all three — how to identify which one you are looking at, the exact commands to prove it, and how to set up renewal automation so the third category stops being a category at all.

Why TLS certificate errors are so confusing to debug

Two things conspire to make this harder than it should be.

The first is that browsers are forgiving in ways other clients are not. When a server sends an incomplete certificate chain, most modern browsers will quietly fetch the missing intermediate using the Authority Information Access extension embedded in the certificate. They fix your misconfiguration on the fly and show you a padlock. Meanwhile curl, Python’s requests, Java clients, Go services, and older Android devices do no such thing. They fail. This is why “it works in Chrome” is worth almost nothing as a diagnostic signal.

The second is that error messages describe the symptom rather than the cause. “Unable to get local issuer certificate” sounds like a client problem. It usually is not — it usually means your server is not sending the full chain. Learning to translate the message into the actual failure is most of the work.

One command that tells you almost everything

Before touching config files, look at what the server is actually sending on the wire:

openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null

The -servername flag matters more than people expect. It sets SNI, and without it you may be handed the server’s default certificate rather than the one for the domain you care about. Plenty of confusing “the certificate is for the wrong domain” reports come from testing without SNI.

Three parts of the output are worth reading carefully:

• The “Certificate chain” block near the top, listing each certificate the server sent with its subject and issuer.
• The “Verify return code” at the bottom. 0 (ok) is what you want. 20 and 21 both point at chain problems.
• The subject line of the first certificate, which tells you whether you even got the certificate you expected.

If you want the certificate details without the chain noise, pipe it into x509:

echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null 
  | openssl x509 -noout -subject -issuer -dates -ext subjectAltName

That single line answers “who issued it”, “when does it expire”, and “what hostnames does it cover” in one shot. It is the first thing I run.

Family one: certificate chain issues

A publicly trusted certificate is almost never signed directly by a root. There is at least one intermediate CA in between. The client already trusts the root — it ships in the operating system or browser trust store — but it has no idea about the intermediate unless you hand it over. Your server’s job is to send the leaf certificate plus every intermediate needed to walk up to a trusted root.

The classic mistake: serving the leaf without the intermediates

If you use Let’s Encrypt, certbot writes several files into /etc/letsencrypt/live/example.com/. Two of them look interchangeable and are not:

• cert.pem — the leaf certificate on its own.
• fullchain.pem — the leaf plus the intermediates.
• chain.pem — the intermediates on their own.
• privkey.pem — the private key.

Nginx wants fullchain.pem. Point it at cert.pem and browsers will still work, so the mistake survives testing and surfaces weeks later when an API client fails.

server {
    listen 443 ssl;
    server_name example.com www.example.com;

    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
}

On Apache 2.4.8 and later, SSLCertificateFile can hold the leaf and the intermediates concatenated together, and SSLCertificateChainFile is deprecated. If you inherited a config that still uses the old directive, it will work, but it is worth cleaning up:

SSLCertificateFile    /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem

Wrong order, or a root that should not be there

Order is part of the specification, not a suggestion. The leaf comes first, then each issuer in sequence walking upward. Some clients tolerate a shuffled bundle; several do not, and the ones that do not tend to be the ones you find out about from an angry customer.

Including the root certificate at the end is a different case: it is harmless from a trust perspective, since the client ignores it and uses its own copy, but it adds bytes to every single handshake for no benefit. Leave it out.

Verifying a chain offline

You can check a bundle before deploying it, which is far better than discovering the problem in production:

# Does the leaf verify against the intermediates you plan to serve?
openssl verify -untrusted chain.pem cert.pem

# Print the subject and issuer of every certificate in a bundle, in order
openssl crl2pkcs7 -nocrl -certfile fullchain.pem 
  | openssl pkcs7 -print_certs -noout

That second command is the one I reach for most. Read the output top to bottom: the issuer of each certificate should be the subject of the next one down. The moment that pattern breaks, you have found your gap.

The key and the certificate do not match

Less common, but it produces a spectacularly unhelpful error. After a renewal where files were copied around by hand, confirm the pair still belongs together — the two hashes must be identical:

openssl x509 -noout -modulus -in cert.pem | openssl md5
openssl rsa  -noout -modulus -in privkey.pem | openssl md5

Family two: SAN mismatches

The Common Name field is dead. It has been deprecated for hostname matching for years, and Chrome stopped honouring it entirely back in 2017. What matters now is the Subject Alternative Name extension. If the hostname a client requested is not in the SAN list, the connection fails, regardless of what the CN says.

Check what a certificate actually covers:

# From a file
openssl x509 -noout -ext subjectAltName -in cert.pem

# From a live server
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null 
  | openssl x509 -noout -ext subjectAltName

The four SAN mistakes that keep recurring

• Apex without www, or www without apex. These are two distinct names. A certificate for example.com does not cover www.example.com. Both need to be in the SAN list, and both need to be passed to certbot with separate -d flags.
• Assuming a wildcard covers everything. It does not. *.example.com matches exactly one label. It covers api.example.com. It does not cover example.com itself, and it does not cover staging.api.example.com. Multi-level subdomains need their own wildcard or their own entry.
• A new subdomain that never made it into the renewal. You add metrics.example.com to nginx, point DNS at it, and forget that the certificate was issued for a fixed list of names. Adding a name means reissuing, not just reloading.
• Connecting by IP address. Almost no public certificate includes an IP SAN, so hitting the server by IP will always fail hostname verification. That is expected behaviour, not a broken certificate.

One more that catches people out: if you use Cloudflare in proxied mode, the certificate a visitor sees is Cloudflare’s edge certificate, not yours. Debugging the origin means bypassing the proxy and testing the origin IP directly with the Host header set correctly, or temporarily turning the orange cloud grey.

Family three: expiry, and why automation is no longer optional

Expiry used to be a calendar problem. You bought a certificate for a year, set a reminder, and dealt with it once. That era is closing.

In April 2025 the CA/Browser Forum passed ballot SC-081v3, which phases the maximum validity period for publicly trusted TLS certificates down from 398 days to 47 days. The schedule runs in three steps: 200 days from 15 March 2026, 100 days from 15 March 2027, and 47 days from 15 March 2029. The first phase is already in force. Domain validation reuse windows shrink alongside it.

The 47-day target was chosen deliberately: short enough that manual renewal stops being viable at any meaningful scale. Automation is the point of the change, not a side effect of it.

If you run a handful of domains, this is a nuisance. If you run fifty, manual renewal is already finished as a strategy.

Getting ACME renewal right

Certbot installs a systemd timer or cron entry that attempts renewal twice a day and does nothing unless the certificate is close to expiry. Confirm it is actually there:

systemctl list-timers | grep certbot
certbot renew --dry-run

The dry run exercises the whole path against the staging environment without touching rate limits. Run it after any change to your web server config, not just after installing certbot. A renewal that worked last quarter can break because someone added a redirect that intercepts the ACME challenge path.

The trap: renewal succeeds, the site still serves the old certificate

This is the failure I see most often, and it is genuinely maddening because every log says success. Certbot writes the new certificate to disk. Nginx has the old one loaded in memory and keeps serving it until it is told otherwise. The certificate on disk is valid, the certificate on the wire is expired, and everything looks fine from the server’s point of view.

The fix is a deploy hook, which runs only when a certificate was actually renewed:

certbot renew --deploy-hook "systemctl reload nginx"

Better still, make it permanent by adding it to the renewal configuration at /etc/letsencrypt/renewal/example.com.conf, under the [renewalparams] section, so it survives however the renewal happens to be triggered:

renew_hook = systemctl reload nginx

Use reload rather than restart. A reload picks up the new certificate without dropping connections.

Remember that anything else terminating TLS needs the same treatment. A mail server, a database proxy, a Docker container with the certificate baked into its image — each one needs its own reload step, and containers in particular will happily run for months with an expired certificate mounted from the host.

Wildcards need DNS-01

Wildcard certificates cannot be issued through HTTP-01 validation. They require DNS-01, which means certbot needs to create a TXT record on your domain, which means API credentials for your DNS provider. Use a provider plugin rather than the manual mode — manual DNS-01 cannot be automated, and an unautomated wildcard on a 47-day cycle is an outage with a scheduled start time.

If issuance fails outright with a policy error, check your CAA records before assuming the ACME client is broken. A CAA record that names a different CA will block issuance entirely:

dig CAA example.com +short

Monitor from outside, not from the server

Checking the certificate file on disk tells you what should be served. Only an external check tells you what is being served. Given the reload trap above, that difference is the entire point.

A minimal check that works anywhere:

#!/usr/bin/env bash
# Usage: ./cert-days.sh example.com
# Exits non-zero if fewer than 21 days remain.
set -euo pipefail

DOMAIN="$1"
THRESHOLD=21

END=$(echo | openssl s_client -connect "${DOMAIN}:443" -servername "${DOMAIN}" 2>/dev/null 
      | openssl x509 -noout -enddate | cut -d= -f2)

DAYS=$(( ( $(date -d "${END}" +%s) - $(date +%s) ) / 86400 ))
echo "${DOMAIN}: ${DAYS} days remaining (expires ${END})"

[ "${DAYS}" -lt "${THRESHOLD}" ] && exit 1
exit 0

If you already run Prometheus, the blackbox exporter gives you this for free through the probe_ssl_earliest_cert_expiry metric. An alert rule looks like this:

groups:
  - name: tls
    rules:
      - alert: TLSCertificateExpiringSoon
        expr: (probe_ssl_earliest_cert_expiry - time()) / 86400 < 21
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "TLS certificate expiring in under 21 days"

Set the threshold well above your renewal window. With 200-day certificates renewing at 30 days remaining, a 21-day alert gives you nine days of warning that something in the pipeline broke. Once 47-day certificates arrive, those numbers compress hard, and an alert that fires too late is the same as no alert.

A workflow for diagnosing TLS certificate errors

When something breaks, work through it in this order. It takes about a minute and rules out whole families of causes as you go.

1. Reproduce outside the browser. Use curl -v or openssl s_client. If the browser works and curl does not, you are almost certainly looking at a chain problem.
2. Read the verify return code. Code 20 or 21 means a chain issue. A hostname mismatch message means a SAN issue. A date error means expiry — or a wrong system clock on the client.
3. Print the dates. Rules expiry in or out in one command, and tells you whether the certificate on the wire matches the one on disk.
4. Print the SANs. Confirm the exact hostname being requested appears in the list. Watch for apex versus www.
5. Walk the chain. Each certificate’s issuer should be the next one’s subject. Find where the sequence breaks.
6. Compare disk against wire. If the file is newer than what is being served, you have a reload problem, not a certificate problem.
7. Check the client’s trust store. If everything above is clean and one specific client still fails, the problem has moved to their end — an old Android device, a Java service with its own cacerts, or a container image built on a base with a stale CA bundle.

Common mistakes worth avoiding

• Testing only in a browser, and mistaking its silent chain repair for a working configuration.
• Pointing the web server at cert.pem instead of fullchain.pem.
• Renewing without reloading the service that holds the certificate in memory.
• Running certbot renew on a schedule but never running --dry-run after config changes.
• Copying key and certificate files between servers by hand and losing track of which pair belongs together.
• Monitoring the file on disk rather than the certificate actually being served.
• Forgetting that mail servers, load balancers, and containers terminate TLS too, and need their own renewal path.
• Adding a subdomain to the web server config and assuming the existing certificate covers it.

Best practices that hold up over time

• Automate issuance and renewal from day one, even for a single domain. The habit matters more than the scale.
• Always use a deploy hook, and always reload rather than restart.
• Alert on days remaining, measured externally, with a threshold at least twice your renewal window.
• Keep an inventory of every hostname you hold a certificate for. You cannot monitor what you have forgotten about.
• Prefer DNS-01 with a provider plugin if you need wildcards, and never rely on manual DNS validation.
• Set CAA records deliberately so you know which CAs are permitted to issue for your domain.
• Test with a non-browser client before calling a certificate deployment finished.

Frequently asked questions

Why does my site work in Chrome but fail in curl?

Almost always an incomplete chain. Browsers fetch missing intermediate certificates automatically using the Authority Information Access extension; curl does not. Point your web server at the full chain file rather than the leaf certificate and both will work.

What does “unable to get local issuer certificate” actually mean?

The client could not build a path from your certificate to a root it trusts. Nine times out of ten the server is not sending its intermediates. Occasionally the client’s own CA bundle is out of date, which is common in older container base images.

Does a wildcard certificate cover the root domain?

No. *.example.com matches a single label, so it covers www.example.com but not example.com and not a.b.example.com. Include the apex explicitly as an additional SAN entry when you request the certificate.

My certificate renewed but the site still shows it as expired. Why?

The service holding the certificate has not reloaded. The new file is on disk; the old one is still in memory. Add a deploy hook that reloads the service on renewal, then verify from outside with openssl s_client rather than trusting the file timestamp.

How short will certificate lifetimes actually get?

Under the CA/Browser Forum schedule, 200 days applies from March 2026, 100 days from March 2027, and 47 days from March 2029. Private and internal CAs are not bound by these rules, but public certificates are.

Is the Common Name field still used at all?

Not for hostname validation. Modern clients read only the Subject Alternative Name extension. A certificate whose CN matches but whose SAN list does not include the hostname will be rejected.

Wrapping up

TLS certificate errors feel opaque mostly because the error text points at the wrong layer. Once you sort them into chain, SAN, or expiry, each one has a short diagnostic path and an obvious fix. Get comfortable with openssl s_client and openssl x509 and you will resolve most incidents faster than it takes to find the right dashboard.

The expiry category is the one worth engineering away permanently. With maximum lifetimes dropping toward 47 days, any process that depends on a person remembering something will fail eventually. Automate the renewal, add the deploy hook, and alert on what is actually being served rather than what is sitting on disk. Do that once and TLS certificate errors go back to being a rare curiosity instead of a recurring outage.

Need help with TLS, certificates, or infrastructure automation?

I work with teams on exactly this kind of problem — broken certificate chains, ACME automation across multiple servers, expiry monitoring wired into Prometheus and Grafana, and the wider DevOps plumbing that keeps it all running quietly in the background.

If you would rather hand this off than debug it at two in the morning, you can hire me on Upwork.

The post TLS Certificate Errors: Chain Issues, SANs, and Expiry Automation appeared first on John Nessime.