Then you open the ModSecurity audit log and find eleven rule matches on a single request, none of them obviously the culprit.

Dealing with ModSecurity false positives on WordPress is not hard once you understand how the OWASP Core Rule Set decides to block. It is very hard if you go looking for “the rule that blocked me,” because under the default configuration that rule does not exist. This post covers reading the log, choosing the right exclusion mechanism, the WordPress failure families that generate most of the noise, and the tuning knobs that quietly gut your coverage.

The invisible part: no single rule blocked the request

The Core Rule Set runs in anomaly scoring mode by default. Individual detection rules do not deny anything. They match, they log, and they add to a running score with setvar. At the end of the request phase, a separate blocking evaluation rule compares the accumulated score against a threshold and returns the 403.

Scores come from rule severity. A CRITICAL match adds 5, ERROR adds 4, WARNING adds 3, NOTICE adds 2. The inbound threshold ships at 5 and the outbound threshold at 4. So one critical match is enough on its own, and so are two notices plus a warning.

That is the failure mode nobody warns you about. The log line that produced your 403 lives in REQUEST-949-BLOCKING-EVALUATION.conf and it says the inbound anomaly score was exceeded. It does not name what matched. Take that ID and disable it and you have not fixed a false positive, you have turned off blocking for the whole site while leaving every detection rule running and logging. The site will look fixed. It is not protected.

First rule of tuning, then: the blocking rule is a symptom. Read backwards from it.

Read the log before you touch a rule

Every alert carries a unique_id. That value ties the error log line to the full audit log record for the same transaction, and it is the only reliable way to see every rule that fired on one request instead of a scattering of lines from different visitors.

# Paths vary by build. Check SecAuditLog in modsecurity.conf for yours.
tail -n 200 /var/log/httpd/error_log | grep -i modsecurity

# Then pull the whole transaction using the unique_id from that output
grep -F 'Zm9vYmFyCg' /var/log/httpd/modsec_audit.log

From each match, four fields matter and the rest is noise:

• The rule ID, as [id "941320"]. This is what you exclude.
• The matched variable, at the end of the pattern-match message as something like at ARGS:content. This is what you exclude it from.
• The URI, which tells you how narrowly you can scope the exclusion.
• The paranoia level tag, as [tag "paranoia-level/2"]. If every alert is tagged level 2 or higher, the real fix may be a paranoia level change rather than an exclusion.

One warning most guides skip: audit logs of false positives contain the payload that triggered them. If a user’s password happens to look like a shell command, it lands in your log in plaintext. Treat these files as sensitive, rotate them, and redact before pasting anything into a public bug report.

Pick the right exclusion mechanism

ModSecurity gives you four ways to write an exclusion, differing on two axes: applied once at startup or evaluated per request, and removing the whole rule or just one variable from it.

• SecRuleRemoveById and SecRuleRemoveByTag delete a rule or a whole tagged category at configure time. Blunt, cheap, global.
• SecRuleUpdateTargetById and SecRuleUpdateTargetByTag strip one variable from a rule at configure time, leaving the rule live for everything else.
• ctl:ruleRemoveById and ctl:ruleRemoveByTag do the same removal, but only for transactions matching a condition you write.
• ctl:ruleRemoveTargetById and ctl:ruleRemoveTargetByTag strip one variable, conditionally. This is the surgical option and the one to reach for first.

Runtime exclusions cost a little, since each is an extra rule evaluated on every request. That overhead is small next to the regex work the rule set already does, and the precision is worth it.

Placement decides whether an exclusion works at all

This is where most people lose an afternoon. The two types go in opposite places, for mechanical reasons rather than arbitrary ones.

• Configure-time exclusions go after the CRS include. They remove rules, and a rule has to exist before you can remove it.
• Runtime exclusions go before the CRS include. They change how rules behave for this transaction, and changing a rule after it has already run does nothing.

The Core Rule Set ships two files for exactly this, both with an .example suffix you drop before they load. REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf holds runtime exclusions, and RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf holds configure-time ones. Use them. Local files that survive a rule set update are the entire point.

Five failure families behind most ModSecurity false positives

Almost everything you will hit on WordPress falls into one of these. Recognizing the family tells you the shape of the fix before you read a single log line.

1. Post content that legitimately contains markup

WordPress lets editors put HTML in posts. It maintains its own allowed-tag list, sanitizes server side, and that mechanism has been reviewed hard over the years. The Core Rule Set knows none of this. It sees an <h1> or an inline style attribute in a POST parameter and matches an XSS rule.

This produces the “it saved yesterday” ticket, because it is content-dependent. Plain paragraphs pass. A pasted table with attributes does not. The fix is a target exclusion scoped to the save endpoint and the specific parameter, never a global rule removal:

# Runtime exclusion. Goes in REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf
# Replace 941320 and the parameter with what your audit log actually shows.
SecRule REQUEST_URI "@beginsWith /wp-admin/post.php" 
    "id:1000,
    phase:1,
    pass,
    t:none,
    nolog,
    ctl:ruleRemoveTargetById=941320;ARGS:content"

Pick your own IDs from a range CRS will never use. Reusing an existing ID is a startup error rather than a silent one, which is the good kind of mistake.

2. REST and admin-ajax calls that fail with no visible error

This family is nastier, because there is no error page. The browser hits /wp-json/ or /wp-admin/admin-ajax.php, gets a 403, and the JavaScript either swallows it or shows a generic “updating failed” toast. Autosave stops. An editor panel stays empty. A form plugin silently drops submissions. Users report this as flakiness rather than a security block, so it sits unfixed for months. When someone says the editor is “just slow sometimes,” open the browser network tab and look for 403s before going anywhere near PHP.

Scoping differs here. REST routes are structured, so use @rx against REQUEST_FILENAME to hit a route family rather than one path. For admin-ajax.php, the useful discriminator is usually the action parameter, since that one endpoint multiplexes dozens of unrelated plugin handlers.

3. Media uploads that break at a size boundary

Multipart uploads have two independent failure paths that look identical from the browser. The first is a genuine rule match on the multipart structure, or on binary content containing a byte sequence some rule dislikes. That shows up with a rule ID and you tune it normally.

The second is an engine limit, not a rule. SecRequestBodyLimit caps the total body ModSecurity will buffer, SecRequestBodyNoFilesLimit caps the non-file portion, and SecRequestBodyLimitAction decides whether an oversized body is rejected outright or inspected partially. If uploads fail above a consistent size and no CRS rule ID appears anywhere in the log, stop tuning rules. That is a limit, and the fix is in engine config.

Raising them costs memory per concurrent request, and partial inspection leaves part of the body unscanned. Decide which trade you are making rather than bumping the number until the ticket closes.

4. Cookies with dynamic names

WordPress authentication cookies carry a hash in the name, so a literal exclusion is impossible. Configure-time directives accept a regular expression in the target if you wrap it in forward slashes:

# Configure-time exclusion. Goes in RESPONSE-999-EXCLUSION-RULES-AFTER-CRS.conf
SecRuleUpdateTargetById 942440 "!REQUEST_COOKIES:/^wordpress_logged_in_.*/"

The catch is significant: the ctl action supports no regular expressions at all. If you need a pattern match on a variable name, you are forced into the configure-time form, which applies site-wide rather than to one path. That single constraint drives more design decisions than people expect.

5. Chained rules that refuse to be tuned

Some rules are written as a chain of conditions. Target exclusions only apply to the first rule in the chain, so if the variable you want to strip is referenced further down, nothing you write has any effect and you will burn an hour convincing yourself the config file is not loading.

When a target exclusion visibly does nothing on a rule you can watch matching, check whether it is chained. If it is, your only option is removing the whole rule under a narrow condition. Accept the wider blast radius, or change the request instead.

Install the WordPress plugin before writing anything custom

The Core Rule Set maintains a WordPress rule exclusions package. In CRS 3 it was built in and switched on with a variable in crs-setup.conf. In CRS 4 it moved out into a separate plugin published in the CRS plugin registry, and it is not installed by default.

That catches people upgrading. Exclusions that used to work vanish, WordPress starts throwing 403s again, and the setup variable they remember is gone from the config file. The reasoning behind the split is sound: fewer rules loaded means less code that can carry a bug, and one of the old exclusion packages did carry a bypass.

Plugins load through wildcard includes in a specific order. Get the order wrong and the plugin silently does nothing:

Include /path/to/crs/crs-setup.conf

Include /path/to/crs/plugins/*-config.conf
Include /path/to/crs/plugins/*-before.conf

Include /path/to/crs/rules/*.conf

Include /path/to/crs/plugins/*-after.conf

Two limits before you lean on it. The plugin covers vanilla WordPress only, so false positives from third-party plugins, page builders and form plugins are still yours to tune. And the comment field is deliberately left out of the exclusions, because comment forms have a long history as a real attack surface. If your comment form throws false positives, that is a decision the maintainers made on purpose, and overriding it is a genuine risk trade rather than a gap in their package.

On a server hosting more than one application, scope the plugin to the WordPress vhost using WebAppID or a Host header condition. WordPress exclusions applied to your unrelated API are a hole you did not mean to open.

The knobs that look like a fix and are not

When exclusions get tedious, two shortcuts start looking attractive. Both cost more than they appear to.

Raising the anomaly threshold. Going from 5 to 20 makes most false positives vanish, and it also means any attack category that scores low can never trigger a block. Detections that only ever produce a single critical match become unblockable. The CRS documentation is direct here: keep the threshold at the default and change the paranoia level instead. Raising it is a temporary tuning aid, not a configuration.

Dropping the paranoia level. Legitimate, but often applied backwards. Level 1 is designed to produce few false positives. Levels 2 and up add progressively more aggressive rules and progressively more tuning work. If your alerts are all tagged level 2, dropping to level 1 is the correct fix and it is honest about what you gave up.

The better tool for moving up is the split between blocking and detection paranoia levels. Set tx.blocking_paranoia_level to what you actually enforce and tx.detection_paranoia_level higher. Rules from the higher level execute and log, but only rules up to the blocking level count toward the score. You get to see what level 2 would have caught, in production, on real traffic, without anyone getting blocked. Then you tune, then you promote. The performance cost lands as soon as you raise the detection level, though, since those rules run either way.

Two smaller settings help here. tx.sampling_percentage sends only a fraction of requests into the rule set, a gentler introduction for a busy site than blocking everything on day one. And tx.reporting_level can log requests that would have been blocked at a higher paranoia level, which is exactly the signal you want while climbing.

Troubleshooting

The exclusion has no effect. Check placement first. A runtime exclusion loaded after the rules, or a configure-time exclusion loaded before them, fails silently. Then check for a chained rule. Then confirm the file is being included at all by breaking it deliberately with a syntax error and seeing whether the server refuses to start.

It works on one site and not another. Different vhosts load different config. On control panels, per-domain overrides usually sit in a separate file from the global rule set.

It worked and then stopped. Something updated the rule set. If you edited CRS rule files directly, your edits are gone, which is the whole argument against editing them.

Nothing appears in the log at all. Confirm the rule engine is on, and that auditing is enabled with enough log parts to include the request body. An engine in detection only mode logs without blocking, which is confusing if you did not know you were in it.

You are on shared hosting and cannot edit any of this. Some hosts expose a per-domain exclusion box, some let you disable rule IDs, some give you nothing. That is a real ceiling. Providers that hand you root, such as InterServer or Contabo, let you own the rule set and its update cadence, which on a site with genuine tuning needs beats the panel convenience you trade away.

Common mistakes

• Disabling the blocking evaluation rule because it showed up in the log. It reports the score, it detects nothing, and removing it disables blocking site-wide.
• Editing CRS rule files directly. Every update reverts you, and you now maintain a fork by hand.
• Excluding by message rather than by ID. Messages carry dynamic data and special characters, so the match is fragile.
• Removing an entire attack tag because two rules misfired. SecRuleRemoveByTag attack-sqli is defensible when nothing behind the WAF touches SQL. On WordPress it is not.
• Excluding /wp-admin/ wholesale, or stacking multiple rule set vendors. Admin is where a compromised session does the most damage, and overlapping rule sets make attribution guesswork.
• Leaving known false positives in place. Real attacks get lost in the noise, and the logs accumulate user data you never intended to collect.

Best practices

• Start in detection only. Collect real traffic, tune what you find, then enable blocking. Enforcing first is how you learn about broken checkout at two in the morning.
• Install the WordPress exclusions plugin before writing custom rules. Do not hand-write what upstream already maintains.
• Write the narrowest exclusion that resolves the alert: one rule, one variable, one path. Comment it with the rule ID, the endpoint, and why you judged it safe. Six months later that comment is the only thing between you and a blind revert.
• Keep exclusion files in version control and deploy them with the rest of your config. Test rule set updates on staging, since new signatures land regularly.
• Ship audit logs somewhere queryable. Grepping a growing text file stops scaling fast, and a platform like Grafana Cloud turns “which rule fires most” into a query instead of an afternoon.
• If an edge WAF such as Cloudflare sits in front of the origin, decide deliberately which layer owns which rule. Two WAFs scanning the same body doubles your false positive surface for no extra coverage.

Frequently asked questions

How do I find which ModSecurity rule blocked my WordPress request?

Take the unique_id from the 403 entry in the error log and grep the audit log for it. That gives you every rule that fired on the transaction. The blocking evaluation rule is what returned the 403, but the detection rules listed alongside it are what you tune.

Should I disable ModSecurity for WordPress instead of tuning it?

It is a defensible choice if the alternative is blocking mode with unresolved false positives, since a WAF everyone has learned to route around is worse than none. But detection only mode gives you the logs with none of the breakage, and that is almost always the better middle ground while you work through the tuning.

Is the WordPress rule exclusions plugin enough on its own?

For a stock install on a standard theme, usually. It covers vanilla WordPress and both block-based and classic themes. It does not cover third-party plugins, page builders, or theme frameworks, and those generate most of the false positives on a real site.

Why do false positives come back after a rule set update?

Either you edited rule files directly and the update overwrote them, or the update shipped new signatures your existing exclusions do not cover. The first is avoidable by keeping every change in the exclusion files. The second is the cost of running a maintained rule set, and it is why updates belong on staging first.

Does ModSecurity slow WordPress down?

It adds per-request work, mostly regular expression evaluation against request data, and the cost scales with paranoia level and body size. The settings that matter are the paranoia level, the request body limits, and whether response body inspection is on. Measure on your own traffic rather than trusting someone else’s benchmark.

Is ModSecurity still maintained?

The engine moved to the OWASP Foundation after Trustwave stepped back from commercial support, and it still receives fixes, though feature work is slower than it once was. The Core Rule Set is a separate project on its own cadence and is actively developed. Coraza is a compatible engine running the same rule language, worth evaluating for new deployments.

Wrapping up

The thing to hold on to about ModSecurity false positives on WordPress is that the rule which returned your 403 is almost never the rule you need to change. Anomaly scoring separates detection from blocking on purpose, and reading the log backwards from the blocking decision to the detection rules that fed it is the whole skill.

Everything else follows. Install the maintained WordPress exclusions instead of reinventing them. Write the narrowest exclusion you can, in the file that survives updates. Move the paranoia level, not the threshold.

Need help tuning your WAF?

Most of the WAF work I take on is untangling exactly this: a rule set that is technically installed and practically ignored, because nobody had time to work through the false positives. I can help with:

• Triaging your audit logs and mapping every recurring false positive to the rule, variable and endpoint that produced it
• Writing scoped exclusion sets for WooCommerce, form plugins, page builders and custom REST routes that upstream packages do not cover
• Moving a site off direct rule file edits and onto exclusion files that survive rule set updates
• Planning a CRS 3 to CRS 4 migration, including the exclusion package to plugin transition and the paranoia level variable renames
• Running a staged rollout from detection only to blocking, with the reporting and sampling settings that make it safe
• Shipping ModSecurity audit logs into a queryable log platform so rule tuning becomes a dashboard instead of a grep

If you have a blocked request you cannot explain, send me the audit log entry and your exclusion file and I will tell you what is firing and why.

The post Your WordPress 403 Isn’t the Rule You Think: Tuning ModSecurity False Positives appeared first on John Nessime.

The fix looked obvious. Drop in a limit_req_zone, apply it, reload, done. Except the reload changed nothing at all, and the reason took an embarrassingly long time to find.

This post covers Nginx rate limiting and bot mitigation the way it actually behaves in front of a real site: how the limiting modules key requests, what burst and nodelay really do to your traffic shape, why a limit that looks correct in the config can be doing nothing (or throttling everyone at once), and how to layer bot handling so you are not playing whack-a-mole with user agent strings forever.

The failure that hides: your limit is keyed on the wrong address

This is the one that bites hardest, because the config parses, nginx -t passes, and the error log stays quiet. Nothing tells you it is wrong.

If anything sits in front of Nginx (Cloudflare, another CDN, a load balancer, or a local Varnish tier), then $remote_addr is the address of that proxy, not the visitor. Key a rate limit on it and you get one of two outcomes, both bad:

• Every visitor arriving through a given edge node shares a single bucket. One aggressive client burns the budget and legitimate users start collecting 503s.
• Or your rate is generous enough that the shared bucket never fills, in which case the limit is decorative and the abusive client sails straight through.

The real IP module fixes this, but only if you trust the right sources. It rewrites $remote_addr and $binary_remote_addr from a header, and only for connections coming from addresses you have explicitly declared trustworthy.

# http context. One entry per CDN range you actually receive traffic from.
set_real_ip_from 203.0.113.0/24;
set_real_ip_from 2001:db8::/32;

real_ip_header CF-Connecting-IP;   # or X-Forwarded-For for a generic proxy
real_ip_recursive on;

Two things matter here more than the syntax.

First, never widen set_real_ip_from to 0.0.0.0/0. That tells Nginx to believe a client-supplied header from anyone, which means an attacker can pick a new “client IP” on every request and your per-IP limit becomes unlimited. It also poisons your logs, your deny rules and anything downstream that trusts the forwarded address.

Second, after the real IP module has run, $binary_remote_addr holds the visitor address and $realip_remote_addr holds the original connecting address (the proxy). Key your limits on $binary_remote_addr. Keying on $realip_remote_addr puts you right back in the shared-bucket failure, silently.

The proxy ranges change over time, so treat the trust list as a file that gets regenerated on a schedule and reloaded, not something you paste in once. On a self-managed VPS from a provider like Contabo or InterServer, a small cron job that fetches the current ranges, rewrites an include file, runs nginx -t, and only reloads on success is about twenty lines of shell and saves you from a very confusing afternoon later.

What limit_req actually does, and what burst really means

Nginx rate limiting for requests uses a leaky bucket. The zone stores, per key, the current number of excessive requests. The rate defines how fast that excess drains. Two directives do the work: limit_req_zone declares the bucket in the http context, limit_req applies it in http, server or location.

http {
    limit_req_zone $binary_remote_addr zone=general:10m rate=10r/s;
    limit_req_zone $binary_remote_addr zone=login:10m   rate=20r/m;

    limit_req_status 429;
    limit_req_log_level warn;
}

The rate is expressed in requests per second (r/s) or requests per minute (r/m). If you want less than one request per second, use r/m. There is no native per-hour or per-day unit in the stock module, which surprises people writing password-reset limits.

burst without nodelay is a queue, not a rejection

This is the second-most common misunderstanding after the keying problem. By default, excessive requests are delayed so that they come out at the configured rate. They are only rejected once the number waiting exceeds burst.

location /search/ {
    limit_req zone=general burst=20;
}

At 10r/s, a client that fires 21 requests at once gets one served immediately, nineteen held and released over roughly two seconds, and the last one rejected. Those nineteen held requests are still open connections on your server. You have not reduced concurrency, you have converted a spike into a slow trickle and paid for it in worker slots and file descriptors.

That is genuinely useful for smoothing traffic to a fragile upstream. It is a bad idea for abuse control, where holding the connection is exactly what the attacker wants.

nodelay, delay, and picking between them

nodelay changes the behaviour to: serve everything in the burst window immediately, reject everything past it. The bucket still drains at the configured rate, so a client that sustains the burst rate will hit the wall and stay there.

location /wp-login.php {
    limit_req zone=login burst=5 nodelay;
    limit_req_status 429;
    # ...fastcgi_pass or proxy_pass here
}

The delay parameter is the middle ground: it sets the point within the burst at which requests start being delayed rather than served immediately. burst=20 delay=10 means the first ten excess requests go straight through, the next ten get queued, and anything beyond is rejected.

How I decide: if a human clicking around your site could plausibly generate the spike, use nodelay or a delay that covers it. A single page load pulling twelve uncached assets through Nginx is one “request” to the user and twelve to the limiter. If the endpoint is machine-facing and expensive (search, export, login, an API write path), tighten the rate, keep the burst small, and use nodelay so rejection is fast and cheap.

Also set limit_req_status 429. The default is 503, which tells clients and monitoring that your server is broken rather than that they are being throttled. 429 is the correct semantic, and well-behaved clients back off on it.

Sizing the zone, and the number most guides get wrong

You will see “10m holds about 160,000 IP addresses” repeated everywhere. That is the 32-bit figure. Nginx documents the stored state for limit_req as 64 bytes on 32-bit platforms and 128 bytes on 64-bit platforms, so one megabyte holds roughly 16 thousand states on 32-bit and roughly 8 thousand on 64-bit. On the 64-bit server you are almost certainly running, a 10m zone is closer to 80,000 states.

This matters because of what happens when the zone fills. Nginx evicts the least recently used state. If it still cannot allocate, the request is terminated with your configured error status. A zone that is constantly thrashing under a distributed crawl will start returning errors for reasons that have nothing to do with the rate you configured. If you are limiting a high-cardinality key, give the zone room.

limit_conn solves a different problem, and you probably need both

limit_req caps how fast requests arrive. It does nothing about a client that opens sixty connections and reads each response one byte at a time. That is a concurrency problem, and it belongs to limit_conn.

http {
    limit_conn_zone $binary_remote_addr zone=perip:10m;
    limit_conn_status 429;
}

server {
    limit_conn perip 20;

    location /downloads/ {
        limit_conn perip 4;
        limit_rate_after 2m;
        limit_rate 512k;
    }
}

Three behaviours worth knowing before you pick a number.

• Not every connection counts. Nginx only counts a connection once the full request header has been read and the request is being processed. A client stalling mid-header is not counted here, which is why client_header_timeout and client_body_timeout still matter.
• Under HTTP/2 and HTTP/3, each concurrent request counts as a separate connection. A limit of 4 that felt generous on HTTP/1.1 can break a modern browser loading a page with many assets. This is a very common self-inflicted outage.
• The limit_conn state is smaller than the limit_req state, so the same zone size holds roughly twice as many entries.

The pairing is the point: limit_req protects against volume, limit_conn protects against occupancy. Rate limiting alone leaves you open to slow-read exhaustion, and connection limiting alone does nothing about a client making one fast request after another on a keepalive connection.

Choosing keys: IP is a starting point, not the answer

The key can be any variable or combination of variables. That flexibility is where most of the value is, because per-IP limits have a real accuracy problem in both directions.

On one side, carrier-grade NAT, corporate egress, university networks and shared VPN exit nodes (NordVPN, Surfshark and every other provider works this way) put many genuine users behind one address. A tight per-IP limit will hit them. On the other side, a distributed scraper renting a few hundred cheap addresses never touches a per-IP limit at all, because no single address is doing much.

So layer keys instead of tuning one harder:

limit_req_zone $binary_remote_addr zone=perip:20m    rate=10r/s;
limit_req_zone $server_name        zone=persite:1m   rate=200r/s;

Applied together, the per-IP zone catches the loud individual and the per-site zone caps total damage regardless of how many addresses are involved. When several limit_req directives match, all of them are evaluated and the most restrictive result wins. If any one of them rejects, the request is rejected.

One inheritance rule catches people out: limit_req directives are inherited from the enclosing level only if the current level defines none. Add a single limit_req inside a location and you have dropped every limit set at server level for that location. If you want both, restate both.

Allowlisting with an empty key

Requests whose key evaluates to an empty string are not accounted at all. That is the clean way to exempt monitoring probes, your office, or a payment webhook source without maintaining a parallel set of locations.

geo $limit {
    default        1;
    10.0.0.0/8     0;
    192.0.2.10/32  0;
}

map $limit $limit_key {
    0 "";                    # empty key, never limited
    1 $binary_remote_addr;
}

limit_req_zone $limit_key zone=perip:20m rate=10r/s;

Nginx evaluates map variables only when they are used, so declaring a large map costs nothing on requests that never reference it.

Roll it out in dry run, not in production

Both modules have a dry run mode. Excessive requests are counted in the shared memory zone and logged exactly as they would be, but nothing is actually rejected or delayed. Combined with the status variables, this gives you a real answer to “would this limit have hurt anyone” before you find out from a customer.

log_format ratelimit '$remote_addr $status "$request" '
                     'req=$limit_req_status conn=$limit_conn_status '
                     'ua="$http_user_agent"';

server {
    limit_req_dry_run  on;
    limit_conn_dry_run on;

    access_log /var/log/nginx/ratelimit.log ratelimit;
}

$limit_req_status reports PASSED, DELAYED, REJECTED, DELAYED_DRY_RUN or REJECTED_DRY_RUN. $limit_conn_status reports PASSED, REJECTED or REJECTED_DRY_RUN. A numbered rollout that has never embarrassed me:

1. Add the zones and the limit_req directives with dry run on, plus the log format above.
2. Run for a full traffic cycle. A week if you have weekly patterns, longer if you have a monthly billing spike.
3. Count the would-be rejections by user agent and by address. If real browsers appear, your rate or burst is wrong, not your users.
4. Turn dry run off on the least critical location first. Watch 429 rates alongside your normal error budget.
5. Only then apply it to login, checkout, or anything that costs you money when it breaks.

Shipping the rate limit log into Loki and graphing it in Grafana makes step three take minutes instead of an evening with awk. Any log platform with decent label cardinality handling works; the point is having rejection counts broken down by zone and user agent, over time, next to your latency graphs.

Bot mitigation in layers, cheapest first

Rate limiting treats all clients the same. Bot mitigation is about deciding that some clients deserve different treatment before the limiter ever sees them. The useful mental model is a set of layers ordered by cost and by how easy each is to defeat.

Layer one: robots.txt, and being honest about what it does

robots.txt is a request, not a control. Crawlers that respect it will honour a Disallow on your search endpoint, your faceted filter URLs and your infinite calendar pages, and that alone removes a surprising amount of load. Crawlers that do not respect it will ignore the file completely, and some will read it specifically to find out where the interesting URLs are.

Use it to shape well-behaved traffic. Never use it as a security boundary. And be careful about blanket disallows on search engines you actually want: deindexing yourself is a much more expensive mistake than a bit of crawler load.

Layer two: user agent classification with map

A map on $http_user_agent is cheap, readable, and easy to revise. It is also trivially spoofable, so treat it as traffic shaping rather than enforcement.

map $http_user_agent $ua_class {
    default          human;
    ""               empty;
    "~*bot|crawler|spider|scrape"  bot;
}

map $ua_class $bot_limit_key {
    bot   $binary_remote_addr;
    empty $binary_remote_addr;
    human "";
}

limit_req_zone $bot_limit_key zone=bots:10m rate=30r/m;

That gives declared bots a much tighter budget than browsers while leaving human traffic on the general limit. An empty user agent is worth its own class: almost nothing legitimate sends one, and it is a reliable signal for cheap scraping tools.

Two cautions. Broad substring matching on bot will catch strings you did not intend, so review a sample of your own logs before deploying the pattern. And blocking rather than throttling is a bigger decision than it looks: if you return 403 to a crawler you actually want, you find out weeks later in your search traffic, not in your error log.

Layer three: verifying bots that claim a name

The interesting case is a request claiming to be a major search crawler. Impersonating one is the standard way to bypass a user agent based allowlist, so a claim needs verification.

The method the search vendors themselves document is a forward-confirmed reverse DNS check: reverse-resolve the client address, confirm the resulting hostname belongs to the vendor’s domain, then forward-resolve that hostname and confirm it returns the original address. An attacker cannot fake this without controlling the vendor’s DNS. Several vendors also publish their crawler IP ranges as machine-readable lists, which avoids the DNS round trip entirely.

Here is the honest limitation: stock Nginx has no directive that does this. There is no built-in reverse DNS verification in the core modules. Your realistic options are a third-party module, doing the verification out of band and feeding the result into a geo block or an included allowlist file, or pushing the decision up to a CDN or WAF layer that already does it. Anyone showing you a pure stock-Nginx config that “verifies Googlebot” is verifying a string, not an identity.

This is also the point where an edge provider earns its keep. Cloudflare, Fastly and similar edge platforms maintain verified bot lists and fingerprinting that no single origin server can reproduce, and they drop the traffic before it costs you a worker. If your bot problem is large and distributed, the edge is the right layer and Nginx is your backstop, not your front line.

Layer four: escalation to the firewall

A client that collects hundreds of 429s and keeps going is not going to stop. Serving it a rejection still costs you a TCP handshake, a TLS negotiation and a log line. At that point the cheapest response is to stop answering at all.

Fail2ban is the usual tool: a filter that matches your rate limit rejections in the Nginx error or access log, an action that inserts a firewall drop, and a ban time long enough to be inconvenient. Two things to get right. Match on a pattern that only fires for real limit rejections, or you will ban yourself during a deploy. And if you are behind a CDN, your firewall sees the CDN address, not the client, so the ban has to be applied at the edge instead. Banning a Cloudflare range at your origin firewall takes your whole site offline, and it is a mistake people make exactly once.

Return code 444 is worth knowing here. It is Nginx-specific and closes the connection without any response, which is cheaper than generating an error page for a client that will never read it.

Troubleshooting Nginx rate limiting

The limit does nothing. Check the key first. Log $binary_remote_addr alongside $realip_remote_addr and confirm they differ in the way you expect. Then check inheritance: a limit_req in the matched location discards the server level ones. Confirm which location actually matched by adding $request_uri and a marker header while testing.

Real users are getting 429 or 503. Turn dry run back on and read the log rather than guessing. Usually it is one of three things: a burst too small for a normal page load’s asset count, a limit_conn value set for HTTP/1.1 assumptions on an HTTP/2 server, or a shared-egress network where many people genuinely share one address.

Nothing appears in the error log. The default log level for rate limit refusals is error, and delays are logged one level below whatever you set. If you set limit_req_log_level warn, delays go to info, which your log level may be filtering out entirely. Grep for the limiting messages directly:

grep 'limiting requests' /var/log/nginx/error.log | tail -50
grep 'limiting connections' /var/log/nginx/error.log | tail -50

Intermittent errors under heavy load with no matching rate. Suspect zone exhaustion. Enlarge the zone and see whether the pattern changes. High-cardinality keys and distributed crawls are the usual cause.

Health checks and uptime probes are being throttled. Add them to the allowlist map. Monitoring hitting a rate limit produces alerts about an outage that is not happening, which is worse than no monitoring.

Common mistakes

• Trusting forwarded headers from every source, which turns a per-IP limit into no limit at all.
• Leaving limit_req_status at the default 503, so throttling looks like an outage in your own monitoring.
• Using burst without nodelay on an abuse path, and quietly filling worker slots with held connections.
• Setting a limit_conn value based on HTTP/1.1 intuition on a server serving HTTP/2 or HTTP/3.
• Adding one limit_req in a location and silently dropping every server-level limit for that path.
• Blocking user agent substrings without checking a sample of real logs first.
• Applying identical limits to reads and writes. A cacheable GET and a POST that writes to the database do not deserve the same budget.
• Banning CDN address ranges at the origin firewall because the logs show them as the client.

Best practices worth the effort

• Verify the key before tuning the rate. Every other decision depends on it being correct.
• Always start in dry run and measure a full traffic cycle before enforcing.
• Use separate zones per traffic class. Search, login, API, static and general traffic have different shapes and should not share a bucket.
• Pair limit_req with limit_conn. The first caps arrival rate, the second caps occupancy, and neither substitutes for the other.
• Layer a per-server zone above your per-IP zone so distributed traffic still hits a ceiling.
• Allowlist with an empty key rather than duplicating location blocks.
• Export $limit_req_status to your log platform and graph rejections next to latency and upstream errors.
• Keep CDN trust ranges in a generated include file that is refreshed on a schedule and validated with nginx -t before reload.
• Throttle before you block. Blocking failures are invisible until they show up in your search traffic.

Frequently asked questions

Does Nginx rate limiting stop a DDoS attack?

It helps with application-layer floods that reach your server, because rejecting a request is far cheaper than processing it. It does nothing about volumetric attacks that saturate your network link, since that traffic never gets to Nginx to be limited. For that you need capacity in front of the origin: a CDN or scrubbing service. Treat rate limiting as protection against abuse and accidental floods, not as DDoS mitigation.

What rate should I actually set?

There is no universal number, and any guide that gives you one has not seen your traffic. Measure instead: run dry run mode, look at your real per-IP request distribution, and set the rate above the busiest legitimate client with headroom. Then set a separate, much tighter zone for the specific endpoints that are expensive or attacked.

Should I use 429 or 503 for rejected requests?

429 in nearly every case. It communicates that the client should slow down, well-behaved clients back off on it, and it keeps your 5xx metrics meaningful. The default is 503 purely for historical reasons.

Can Nginx rate limit per hour or per day?

Not with the stock module. It accepts requests per second and requests per minute only, so the slowest native rate is one request per minute expressed as 1r/m. Longer windows need a third-party module, a shared store such as Redis via a scripting module, or enforcement in the application itself. For something like a password reset limit, the application is usually the better place anyway, because it can key on the account rather than the address.

Do rate limit zones work across multiple Nginx servers?

Not in the open source build. Each instance keeps its own shared memory zone, so a client spread across four load-balanced servers effectively gets four times the budget. The sync parameter on limit_req_zone synchronises state across a cluster, but it is part of the commercial subscription. Without it, either divide your intended rate by your instance count, ensure your balancer pins clients consistently, or move the limit to a shared store.

Will blocking AI crawlers hurt my SEO?

Blocking training crawlers is separate from blocking search indexing crawlers, and the distinction is worth checking per vendor before you write a rule, because some vendors run several crawlers with different purposes under related names. The risk is collateral damage: a broad user agent pattern that also matches a search crawler will remove you from results, and you will not notice quickly. Verify your pattern against real log samples, and prefer throttling over outright blocking while you are unsure.

Where should rate limiting live: the CDN, Nginx, or the application?

All three, doing different jobs. The CDN handles volume and known-bad reputation before it costs you anything. Nginx handles per-client shaping and protects your workers from anything that gets through. The application handles limits that need identity or business context, like per-account quotas, which Nginx cannot see. Skipping the Nginx layer because you have a CDN leaves you exposed the moment someone finds your origin address.

The one thing worth remembering

Nginx rate limiting is not hard to configure. It is easy to configure into a state where it silently does nothing, or silently punishes the wrong people, and both failures look identical from the outside: a config that parses, a clean nginx -t, and no complaints until there are.

So get the key right, prove it with a log line rather than assuming it, and run every new limit in dry run until the data tells you it is safe. Layer bot handling by cost, throttle before you block, and be honest about which layer can actually verify an identity. Everything else is tuning, and tuning is easy once the foundation is not lying to you.

Need a second pair of eyes on your Nginx edge?

Most of the work here is diagnostic rather than clever. Things I regularly help with:

• Auditing an existing limit_req and limit_conn setup to find out whether it is actually limiting anything, including the keying and inheritance traps above.
• Fixing real client IP handling behind Cloudflare or another CDN, with a generated trust list that refreshes safely instead of drifting.
• Designing per-endpoint zones for WordPress, WooCommerce or API origins so login, search and write paths get budgets that match their real cost.
• Running a measured dry run rollout and reporting back what would have been rejected, broken down by user agent and network, before anything is enforced.
• Building bot classification and escalation: user agent maps, allowlists, Fail2ban integration, and knowing which decisions belong at the edge instead.
• Wiring rate limit telemetry into Grafana and Loki so throttling is something you can see on a dashboard rather than discover from a ticket.

If you would like a look at yours, send me the relevant server block, a sample of the access log, and whatever is currently going wrong. That is usually enough to say something useful straight away.

The post Nginx Rate Limiting and Bot Mitigation Without Blocking Real Users appeared first on John Nessime.

That is the thing about DirectAdmin PHP-FPM pools that catches people out. You think you are tuning a site. You are not. You are tuning a user, and every domain that user owns is sharing the same worker budget, the same queue, and the same memory ceiling.

This post covers where DirectAdmin actually keeps the pool config, how to size pm.max_children from numbers your server gives you rather than from a blog post’s magic value, which per-site PHP limits are worth setting and which are theatre, why OPcache does not behave the way you expect on a multi-tenant box, and how to make all of it survive the next CustomBuild run.

The design fact that explains most of your 502s

DirectAdmin generates one FPM pool per user account, per PHP version. The pool is named after the Linux user. It listens on a Unix socket named after the user. It runs as that user. Apache or Nginx hands PHP requests to that socket for every domain and subdomain in the account.

So if a reseller client has twelve domains under one username, those twelve sites share a single value of pm.max_children. There is no per-domain pool. This is a long-standing feature request rather than an oversight, and it has a real consequence: the noisiest site in an account sets the availability of every other site in that account.

Once you have internalised that, the fix for a lot of “mystery 502” tickets stops being a config tweak and starts being an account layout decision. If a site matters commercially, give it its own DirectAdmin user. That is the cleanest isolation the panel offers without going off-piste, and it costs you nothing but a bit of admin tidiness.

The alternative, if you have User jails enabled, is DirectAdmin’s isolated FPM mode. That runs a separate php-fpm master process per jailed user instead of a shared master with many pools:

da config-set isolated_fpm 1
systemctl restart directadmin
da taskq --run 'action=rewrite&value=httpd'

It only applies to users who already have Jailed enabled. The trade-off is memory: one master process per user, plus a separate OPcache region for each, adds up quickly. DirectAdmin mitigates that by letting idle per-user FPM instances exit after a period of inactivity, but on a box with sixty accounts you will feel it. I would reach for this on a server hosting a handful of paying clients, not on a general shared box.

Where the pool config actually lives

This trips up almost everyone the first time, because there are two files with nearly the same name and only one of them matters for user sites.

• /usr/local/php83/etc/php-fpm.conf is the master config for that PHP version. Editing it by hand is how you learn that CustomBuild overwrites it.
• /usr/local/directadmin/data/users/USERNAME/php/php-fpm83.conf is the generated per-user pool. This is the file that governs a customer’s sites.
• /usr/local/directadmin/data/templates/php-fpm.conf is the template those per-user files are rendered from.

Substitute your PHP version for 83 throughout. To see what every account is currently running:

ls /usr/local/directadmin/data/users/*/php/php-fpm*.conf

grep -H 'pm.max_children' /usr/local/directadmin/data/users/*/php/php-fpm*.conf

The second command is the one I run first on any server I have just inherited. It gives you the entire worker budget of the box on one screen, and it is usually the moment you discover that forty accounts are all sitting on the stock default.

The DirectAdmin template ships with pm = ondemand and a low default child count, driven by a MAX_CHILDREN token. Ondemand means no workers exist until a request arrives, and idle workers are reaped after a timeout. On a shared box with many mostly-idle accounts, that is the right default and I would leave it alone. On a server dedicated to one busy WordPress site, the spawn latency on the first request after an idle period is real and switching to dynamic is defensible. Most people who switch to dynamic on a shared server are trading memory they do not have for a latency win they will not notice.

Sizing pm.max_children without guessing

Every tuning guide gives you a formula. The formula is fine. The input everyone gets wrong is worker memory, because they use a number from someone else’s server instead of measuring their own. A lean WordPress install and the same install with a page builder, a security plugin and WooCommerce are not remotely the same workload.

Measure it. This averages the resident memory of the workers currently running in one pool:

ps -eo rss,args | grep '[p]hp-fpm: pool wpuser' 
  | awk '{sum+=$1; n++} END {if (n) printf "%d workers, avg %.0f MBn", n, sum/n/1024}'

The bracket trick in the grep pattern stops grep matching its own process. rss is resident set size in kilobytes, which is why the awk divides twice. Run it during a busy period, not at three in the morning, and run it a few times.

Two honest caveats about RSS. It overstates real consumption, because workers forked from the same master share a lot of pages, including the OPcache region. And it understates your worst case, because a worker that has not yet hit a memory-hungry request looks cheap. Treat the average as a planning figure, not a guarantee, and leave headroom.

Then the arithmetic:

1. Take total RAM and subtract what MySQL or MariaDB, the web server, mail, and the panel itself need. On a WordPress box the database is usually the second-largest consumer and it is not optional.
2. Reserve a further slice for page cache and burst. I aim to leave a comfortable margin free rather than allocating to the last megabyte.
3. Divide what is left by your measured per-worker figure. That is your ceiling for all pools combined.
4. Split that ceiling across accounts by actual traffic, not evenly.

Step three is where people go wrong on cheap VPS plans. If you are running a dozen WordPress sites on one of the low-RAM tiers from a provider like Contabo or InterServer, the honest answer is often that you cannot give every account a generous pool and you need to either cut the site count or move up a tier. Setting pm.max_children to a number your RAM cannot back does not buy capacity. It buys the OOM killer, and the OOM killer usually takes the database with it, which turns a slow site into a broken one.

To raise the server-wide default:

da config-set php_fpm_max_children_default 30
systemctl restart directadmin
da build rewrite_confs
systemctl restart php-fpm83

For a single account, go to Admin Level → Custom HTTPD Configurations → domain.com and open the php-fpm entry for that PHP version. In the |CUSTOM1| textarea, set the token:

|?MAX_CHILDREN=40|

That syntax is a template token override, not a config line. It tells the template renderer to use 40 instead of the default when it writes pm.max_children for that user. Writing pm.max_children = 40 in that box instead is the single most common mistake here, and it does something worse than nothing: the template still emits its own pm.max_children, you now have the directive twice in one pool, and FPM refuses to start. On a server where one broken pool file stops the whole master, that takes down every site on the box.

Rewrite and restart after any change, then confirm the value actually landed in the generated file rather than trusting the panel.

Per-site PHP limits, and the difference that matters

The |CUSTOM2| textarea on the same page is where PHP settings go. The syntax is FPM pool syntax, not php.ini syntax:

php_admin_value[memory_limit] = 512M
php_admin_value[max_execution_time] = 120
php_value[upload_max_filesize] = 64M
php_value[post_max_size] = 64M
php_value[max_input_vars] = 5000

The admin in php_admin_value is not decoration. Settings written with php_value can be raised at runtime by the application through ini_set(). Settings written with php_admin_value cannot. That distinction is the whole game on a multi-tenant server.

WordPress and a good number of plugins will happily try to raise memory_limit at runtime, and some optimisation plugins push it to values that would let one bad request consume a large slice of the machine. If you want a genuine ceiling, use php_admin_value. If you want a sensible starting point that a developer can raise for a legitimate import job, use php_value. I use admin for memory and execution time, plain for upload sizes, because upload sizes are a support-ticket generator and not a stability risk.

max_input_vars deserves its own note. It is the one that silently truncates a large WooCommerce product save or a menu with hundreds of items, with no error anywhere. The user reports “my changes did not save” and every log is clean. If you support content-heavy sites, raise it before someone asks.

Letting site owners set their own

Under FPM there is no .htaccess route for PHP settings, because FPM is a separate process and never sees the file. DirectAdmin enables the per-directory alternative by default: a .user.ini file in public_html, controlled by the user_ini.filename setting. It handles the PHP_INI_PERDIR and PHP_INI_USER class of settings only, so it will never override anything you set with php_admin_value. That is the point of it.

The WP-CLI gap

None of this applies to WP-CLI. Cron jobs and shell commands run through the CLI SAPI and read the CLI php.ini, not your pool. So a plugin update that works fine in the browser can die on memory in a cron-driven wp cron event run, and you will hunt through the pool config finding nothing wrong. Check the CLI limits separately when a scheduled task fails but the same action succeeds in wp-admin.

OPcache is shared, and that is the invisible one

OPcache allocates its shared memory region when the FPM master starts, before any pool exists. Every pool under that master inherits the same region. You cannot meaningfully give one account its own OPcache budget through a pool directive, because by the time the pool is read the memory has already been carved out.

Two consequences, and the second is the one nobody plans for.

First, capacity. The default OPcache size is comfortable for one application and thin for thirty WordPress installs, each with its own core, theme and plugin tree. When it fills, OPcache starts evicting and eventually restarts, and every site on the version gets slower at once with nothing in any error log to explain it. Watch the wasted-memory and restart counters rather than guessing.

Second, the sizing you actually need scales with the total number of PHP files across all accounts on that PHP version, not the biggest site. Consolidating everyone onto one PHP version looks tidy and quietly makes this worse. Spreading accounts across two or three PHP versions gives each master its own region, which is an underrated side effect of not forcing everyone to upgrade on the same day.

OPcache is configured through CustomBuild rather than the pool. Edit the customised copy under CustomBuild’s custom directory so your changes survive updates, then rebuild and restart the FPM master.

Making changes survive the next rebuild

Anything you type directly into a generated file is temporary. The next rewrite_confs, PHP update or panel update will flatten it, usually weeks later, and the regression will look like it came from nowhere.

There are three durable places to put changes, in increasing order of blast radius:

• Per user, all PHP versions. /usr/local/directadmin/data/users/USERNAME/php/php-fpm.conf.custom1 and .custom2. These feed the same CUSTOM tokens as the panel textareas, so the settings follow the account when the user switches PHP version in Domain Setup. This is the one I reach for first.
• Server-wide, all users, all versions. /usr/local/directadmin/data/templates/custom/php-fpm.conf.custom1 and .custom2. If either exists, its contents are injected into every pool file DirectAdmin writes. Good for a baseline you want everywhere.
• A full custom template. Copy /usr/local/directadmin/data/templates/php-fpm.conf to the custom subdirectory and edit it. This is the only route for pool directives that have no token, but you now own that file forever and stop receiving upstream template improvements. Use it when nothing else works, and leave a comment at the top saying why.

Global first, then per-user on top. Both land in the same token, so a per-user value does not replace a global one, it appends. For a directive that can only appear once in a pool, that is a duplicate and a failed restart.

Troubleshooting

“server reached max_children setting”

The pool name in that warning tells you which account, not which site. Find the log first rather than guessing at a path:

grep -i 'error_log' /usr/local/php83/etc/php-fpm.conf

Before raising the limit, check whether the requests are legitimate. Cross-reference the timestamps against each domain’s access log in that account. Nine times out of ten it is one of three things: an uncached admin-ajax.php loop from a plugin, a bot crawling faceted search or filtered archive URLs that bypass page cache, or wp-cron.php firing on every request on a site with a heavy scheduled task. Raising pm.max_children to absorb bot traffic is paying for garbage with RAM.

502 or 504 with no PHP error

The request is dying at the FPM boundary rather than inside PHP. Usual causes are a worker killed by request_terminate_timeout, a full listen queue, or the master failing to start after a config change. Check the FPM service status before you check anything in WordPress. If FPM did not start at all, the panel will still show your settings saved and every site on that PHP version will be down.

Changes in the panel that do nothing

Read the generated per-user file after every change. If the value is not in there, the rewrite did not run or the token was wrong. If it is in there and PHP still reports something else, the master was not restarted, or something later in the pool is overriding it. A temporary phpinfo() file served by the site in question settles the argument in seconds. Delete it immediately afterwards.

Session or temp files that are not where you left them

If PHP writes to /tmp and you cannot see the files over SSH, that is systemd’s PrivateTmp giving the FPM service its own private namespace. Nothing is broken. The files exist, in a mount only that process can see.

disable_functions that will not budge

disable_functions is special. From a pool you can add to the list but not remove from it, because the php.ini value has already been applied. To allow a function for one account you have to stop setting it in php.ini at all and drive the whole thing from templates instead. DirectAdmin documents that pattern, and it is worth reading before you start rather than after you have half-done it, because the intermediate state removes protection server-wide.

Common mistakes

• Editing /usr/local/phpXX/etc/php-fpm.conf directly and being surprised when it reverts.
• Writing pm.max_children = 40 into a CUSTOM box instead of the |?MAX_CHILDREN=40| token, producing a duplicate directive and a dead master.
• Sizing workers from a memory figure found online rather than measured on the box.
• Using php_value for memory_limit on shared hosting, then wondering how a plugin raised it to 2G.
• Assuming pool settings apply to WP-CLI and cron.
• Raising limits to absorb bot traffic instead of blocking or caching it upstream.
• Putting a client’s revenue site in the same account as their abandoned projects.

Best practices for tuning DirectAdmin PHP-FPM pools

• One DirectAdmin user per site that matters. Isolation you get for free beats isolation you have to engineer.
• Keep ondemand on shared servers. Only move to dynamic when a single busy site owns the box.
• Set a global baseline in the custom template, then override per user. Do not hand-tune forty files.
• Use php_admin_value for anything that protects the server and php_value for anything that is only a convenience.
• Cut PHP requests before you add workers. Full-page caching, a CDN in front, and Cloudflare or similar handling bots removes far more load than any pool setting.
• Disable WordPress’s request-triggered cron and drive it from a real system cron, so scheduled work stops competing with visitors for workers.
• Alert on saturation, not just uptime. Worker utilisation per pool and PHP-FPM error log entries are the signals worth having in Netdata, Prometheus, or whatever you already run. A site that is queueing looks perfectly healthy to an HTTP check.
• Re-measure after major plugin changes. A page builder or a new security plugin moves per-worker memory enough to invalidate your arithmetic.

Frequently asked questions

Can I create a separate PHP-FPM pool for each domain in DirectAdmin?

Not through the panel. Pools are generated per user, per PHP version, and every domain in the account shares one. The supported ways to get separation are to move the domain to its own DirectAdmin user, or to enable isolated FPM mode for jailed users, which gives each user their own master process. Anything else means custom systemd units you maintain yourself, outside the panel’s knowledge.

What is a good pm.max_children value for WordPress?

There is no portable number, because the input is your measured per-worker memory and your available RAM after the database. Measure the average resident size of your workers under real traffic, subtract everything else the server needs from total RAM, and divide. A lean brochure site and a WooCommerce store with thirty plugins can differ by a factor of three or more.

Where does DirectAdmin store PHP-FPM pool configuration?

Generated per-user pools live at /usr/local/directadmin/data/users/USERNAME/php/php-fpmXX.conf, rendered from the template at /usr/local/directadmin/data/templates/php-fpm.conf. Do not edit either directly. Put durable changes in the custom files under the user’s php directory or in the templates/custom directory.

Why do my PHP settings disappear after a CustomBuild update?

Because they were written into a generated file. CustomBuild and DirectAdmin regenerate those files on update and on any config rewrite. Only the custom template files and the CUSTOM token areas persist.

Does .htaccess work for PHP settings under PHP-FPM?

No. FPM is a separate process from the web server and never reads .htaccess. Use a .user.ini file in the web root for the settings PHP allows at that level, or set the rest in the pool.

Should I switch from ondemand to dynamic or static?

On a shared server, no. Ondemand exists precisely so that dozens of idle pools cost you nothing. On a server dedicated to one high-traffic site, dynamic removes cold-start latency and static gives the most predictable memory profile. The question to ask is whether your idle pools outnumber your busy ones.

Can I give one WordPress site its own OPcache?

Not from the pool config. OPcache’s shared memory is allocated by the FPM master at startup and shared by every pool under it. The practical routes are isolated FPM mode, which gives jailed users their own master, or moving the site to a different PHP version so it sits under a different master.

The one thing to take away

DirectAdmin PHP-FPM pools are a per-user boundary, not a per-site one. Every tuning decision that follows, how many workers to allow, whether a memory limit should be a suggestion or a wall, whether a client’s twelve domains belong in one account, comes out of that single fact.

Measure your workers before you size them, use the CUSTOM token files so your work survives the next rebuild, and watch pool saturation rather than waiting for the 502 ticket. The site that goes down is rarely the site that caused it.

Need a hand with a DirectAdmin server?

Most of the work I do on DirectAdmin boxes is this kind of thing: sites that are fine in isolation and fall over together. If that sounds familiar, I can help with:

• Auditing every pool on a server and producing a worker budget that matches the RAM you actually have
• Tracking down which domain in a shared account is saturating the pool, and why
• Building a global custom template so per-user tuning stops being forty hand-edited files
• Splitting a multi-site DirectAdmin account into isolated users without breaking mail, DNS or SSL
• OPcache and per-version sizing on servers hosting many WordPress installs
• Wiring up pool-level monitoring and alerts so saturation shows up before the ticket does

Send me a pool file, the output of that RSS command, or the last hour of your PHP-FPM error log, and I will tell you what I see.

The post DirectAdmin PHP-FPM Pools: Why One WordPress Site Takes Down the Others appeared first on John Nessime.

Here is the awkward part: nothing was broken. No alert fired, no error rate moved, no dashboard turned red. Third-party API spend is one of the few production signals with no failure mode attached to it. A background job that polls a vendor every thirty seconds and gets back the exact same payload it got last time looks, from the outside, identical to a healthy integration. It just costs money every single time.

This post is about how to reduce SaaS API costs by changing the shape of your calls rather than by renegotiating your contract. Six levers, in the order I’d pull them: read the billing unit, kill pointless polling, collapse N+1 patterns, cache honestly, stop your own retry amplification, and attribute spend so you can tell whether any of it worked.

Start with the billing unit, not the call count

The single most common mistake I see is optimising HTTP requests when the vendor doesn’t bill HTTP requests. Before you touch any code, read the pricing page carefully enough to answer one question: what is the countable thing?

It varies more than people expect:

• Per request. The obvious one. Geocoding, enrichment, most REST endpoints.
• Per session. Google Maps Platform bills Places Autocomplete this way when you pass a session token: the whole keystroke stream plus the terminating Place Details call is grouped and billed as one unit. Omit the token, or reuse one across sessions, and every keystroke request is billed individually.
• Per token issued. Auth0’s machine-to-machine quotas count access tokens obtained through the client credentials flow, not the API calls you make with them. A service that fetches a fresh token on every outbound request is burning the meter for no reason.
• Per input and output token. LLM providers. Output is typically priced higher than input, which makes “be concise” a cost control and not just a style note.
• Per record, event, or message. Twilio, SendGrid, most streaming and messaging platforms. One API call can carry many billable units.
• Not at all. Stripe doesn’t bill you per API call; the constraint there is a rate limit, not a line item. Optimising those calls buys you reliability headroom, not money.

Getting this wrong sends you down the wrong path for weeks. Batching requests against an endpoint that bills per record changes your request count and leaves the invoice exactly where it was. Read the unit first.

Lever one: stop polling for things that rarely change

This is almost always the biggest single line, and it’s invisible because it’s steady. A sync job on a five-minute cron is 288 calls a day per resource. Multiply by resources, by environments, by the replicas you quietly scaled to three, and you get a number with no relationship at all to how often the data changed.

Conditional requests, when the vendor honours them

If the API returns an ETag or Last-Modified header, store it and send it back on the next poll as If-None-Match or If-Modified-Since. If nothing changed you get a 304 Not Modified with an empty body.

# First call: capture the ETag from the response headers
curl -sS -D - -o /dev/null 
  -H "Authorization: Bearer $TOKEN" 
  https://api.github.com/repos/OWNER/REPO/pulls

# Second call: send it back and check the status code only
curl -sS -o /dev/null -w '%{http_code}n' 
  -H "Authorization: Bearer $TOKEN" 
  -H 'If-None-Match: "PASTE_THE_ETAG_HERE"' 
  https://api.github.com/repos/OWNER/REPO/pulls

-D - dumps response headers to stdout so you can see the etag value. -w '%{http_code}' on the second call prints just the status, which is all you care about: 304 means the resource is unchanged and you saved a payload.

Whether that saves money depends on the vendor. GitHub documents that a conditional request returning 304 doesn’t count against your primary rate limit when it’s correctly authorised. Plenty of other vendors return 304 happily and bill it anyway. Test against your own invoice before you assume.

Three things that bite people with ETags:

• They’re per-page on paginated endpoints. A 304 on page one says nothing about pages two through five, so store one ETag per page.
• They can be scoped to the credential. Rotate a token and your stored ETags may stop matching, which quietly turns your conditional requests back into full ones.
• GraphQL endpoints generally don’t do conditional requests at all. You cache those yourself, keyed on a hash of the query and its variables.

Webhooks, when they exist

The proper fix is to stop asking. If the vendor supports webhooks, the call count collapses from “one per interval” to “one per actual change”, which is usually a difference of two orders of magnitude on a slow-moving resource.

Be honest about what it costs, though. Webhooks need a publicly reachable endpoint, signature verification, idempotent handling because vendors redeliver, a dead-letter path, and a reconciliation sweep for deliveries that never arrived. On a resource that changes every few seconds, polling is simpler and the savings are small. On one that changes twice a week, the webhook pays for itself immediately.

A middle option people forget: many APIs expose a cheap “has anything changed” endpoint, a sync cursor, or a delta token. Poll that on a tight interval and only fetch the expensive payload when it moves.

Lever two: collapse the N+1 pattern

You know this one from databases, and it shows up identically against third-party APIs. Fetch a list of 200 items, then fetch each item’s detail individually: that’s 201 billable calls where the vendor probably offers a way to do it in one or two. Look for:

• Expansion or embed parameters. Many REST APIs let you inline related objects in the list response so the detail calls disappear entirely.
• Field selection. Asking for fewer fields is free on per-request billing but matters a lot where the field tier drives the price. Google Places is explicit about this: requesting a single higher-tier field upgrades the whole response to that tier’s pricing.
• Bulk and batch endpoints. Salesforce, Algolia, most CRMs and search platforms have one. They’re usually documented in a corner nobody reads.
• Larger pages. If the endpoint accepts a page size of 200 and you’re defaulting to 25, you’re paying eight times as many calls to fetch identical data.

The page-size one is the cheapest win in this entire post and it’s a one-line change. Check your client library defaults today.

Lever three: cache with a boring, honest TTL

Caching third-party responses is the lever everyone reaches for first and gets subtly wrong. Three failure modes matter more than the cache itself.

Stampede. A popular key expires, fifty concurrent requests all miss, and fifty identical calls go out to the vendor. You just paid fifty times for one refresh, at the worst possible moment.

No negative caching. A lookup that returns “not found” is usually not cached at all, so the same failing lookup hammers the vendor forever. Cache your 404s. Short TTL, but cache them.

Cache key design. If your key doesn’t include the credential or tenant, you will eventually serve one customer’s data to another. This is the one mistake in this post that turns a cost problem into an incident.

You don’t necessarily need application code for this. An Nginx instance in front of the vendor handles all three, and it’s the approach I reach for first when several services hit the same API:

proxy_cache_path /var/cache/nginx/api levels=1:2 keys_zone=api_cache:20m
                 max_size=1g inactive=60m use_temp_path=off;

server {
    listen 127.0.0.1:8080;

    location / {
        proxy_pass https://api.vendor.example;
        proxy_set_header Host api.vendor.example;

        proxy_cache api_cache;
        proxy_cache_key "$request_method$request_uri$http_authorization";

        proxy_cache_valid 200 5m;
        proxy_cache_valid 404 30s;

        proxy_cache_lock on;
        proxy_cache_use_stale updating error timeout;
        proxy_cache_background_update on;

        add_header X-Cache-Status $upstream_cache_status;
    }
}

Reading that from the top down, because every directive is there for a reason:

• proxy_cache_key includes $http_authorization, so responses are isolated per credential. That’s the tenant-leak fix.
• proxy_cache_valid 404 30s is the negative caching. Failed lookups stop being free traffic for the vendor.
• proxy_cache_lock on is the stampede fix. Only the first request for a given key goes upstream; the rest wait for it.
• proxy_cache_use_stale updating plus proxy_cache_background_update on serves the stale copy while a single background request refreshes it. Latency stays flat and you don’t pay a thundering herd at every expiry.
• X-Cache-Status gives you HIT, MISS, STALE or UPDATING in the response so you can actually measure your hit rate instead of guessing at it.

The trade-off is worth stating plainly: you’re now serving data that can be five minutes stale, and you own another moving part. Fine for a currency rate or a product catalogue. Not fine for an inventory count on a checkout page. Set the TTL from how wrong the data is allowed to be, never from how much you want to save.

Where the caller is distributed, Redis with a short lock key does the same job in application code, and a small always-on VPS from something like InterServer or DigitalOcean is plenty to host either the proxy or the cache. At the edge, Cloudflare Workers with KV lets you cache vendor responses close to users, which helps when the calls originate from browsers rather than your backend.

Lever four: stop paying for your own retries

This produces the sudden spike rather than the slow creep, and people miss it because retries feel like resilience.

The vendor wobbles. Every client retries. With a fixed delay they retry in lockstep, so the vendor gets a synchronised wave on top of normal load, struggles longer, and generates more retries. AWS’s architecture write-up on exponential backoff and jitter is the canonical reference, and its central finding is worth repeating: backoff alone still leaves visible clusters of calls. Randomising each client’s delay is what decorrelates the herd, and in their simulation it cut total call count substantially.

import random, time

BASE = 0.5      # seconds
CAP  = 30.0     # never sleep longer than this

def sleep_before_retry(attempt):
    # capped exponential backoff, then full jitter
    window = min(CAP, BASE * (2 ** attempt))
    time.sleep(random.uniform(0, window))

The cap stops a long outage turning into hour-long sleeps. The random.uniform(0, window) is the jitter: two processes that failed at the same instant now wake at different times. Note that full jitter can occasionally produce a near-zero delay, which is fine for transient errors but too aggressive for throttling responses that reset on a per-minute window. For a 429, honour the Retry-After header if the vendor sends one, and use a jitter formula with a guaranteed floor if they don’t.

Three more rules that cost nothing to implement:

1. Never retry a 4xx except 429. A 400 or a 422 will fail identically forever. Every one of those retries is a billed call buying you nothing.
2. Cap retries as a fraction of traffic. A retry budget (say, retries may not exceed some small percentage of total requests) is the only control that sees the system-wide picture. Per-client backoff can’t.
3. Check your layers. If the SDK retries three times, your HTTP client retries three times, and your job runner retries three times, one logical operation is up to twenty-seven calls. This stacking is extremely common and almost never intentional.

For writes, retrying safely needs an idempotency key so the vendor deduplicates rather than creating two of whatever you asked for. Stripe’s Idempotency-Key header is the pattern most others copied. Without it, teams disable retries on writes entirely and then eat manual reconciliation work instead.

Lever five: move latency-tolerant work off the synchronous path

Ask a blunt question about each integration: does a human wait on this response?

Nightly enrichment, classification, report generation, bulk address validation, embedding backfills. Nobody is waiting. Several vendors, including the major LLM providers, offer asynchronous batch endpoints at a discount to the synchronous rate in exchange for a looser delivery window. Rates move, so check current terms rather than a number you read somewhere, but the mechanism is stable and the discount is usually substantial.

Even without a batch endpoint, moving work behind a queue lets you coalesce. Ten events touching the same customer record within a minute become one enrichment call instead of ten. That deduplication window is often worth more than the batch discount.

Lever six: attribute the spend or you’re guessing

Everything above is unfalsifiable until you can answer: which feature, which tenant, which job. The invoice gives you the total and never the cause. What I’d put in place, in order:

1. Route outbound calls through one place. A shared HTTP client wrapper or the caching proxy above. If every service instantiates its own client, you have no chokepoint and no measurement.
2. Emit a counter per call with labels for vendor, endpoint, status class, and cache result. Keep the label set small; per-tenant labels will blow up your metrics cardinality and trade one bill for another.
3. Build one unit-cost metric. Vendor spend per active tenant per month. It’s the only number that survives growth, because raw spend going up during a good quarter is not a problem.
4. Alert on call rate, not on the bill. A rate alert fires in minutes. An invoice fires in thirty days. Grafana, Prometheus, Datadog, CloudWatch, whatever you already run is fine; the point is that the alert exists at all.
5. Track cache hit rate as a first-class SLI. When someone changes a cache key format and the hit rate silently drops to zero, this is the graph that tells you before the invoice does.

Troubleshooting: the bill jumped and nothing shipped

Work down this list. In my experience the cause is usually in the first four.

• Replica count changed. Autoscaling or a Kubernetes replica bump multiplies every cron-driven poller. Check whether your scheduled jobs have leader election, or whether all five pods are dutifully polling the same endpoint.
• Cache hit rate collapsed. A changed key format, a Redis eviction policy under memory pressure, or a TTL someone dropped “temporarily” during debugging. Graph the hit rate over the period, not the current value.
• Retry loop against a persistent 4xx. Look for a flat, high rate of identical failing calls. This has the distinctive signature of high call volume with zero successful responses.
• A non-production environment pointed at production credentials. Staging, a CI pipeline, or an integration test suite running on every commit. Check whether calls are arriving from IPs you don’t recognise.
• Session or token handling regressed. A refactor that dropped a session token parameter, or a service that stopped caching its auth token and now mints a new one per request.
• Legitimate growth in an expensive segment. A few large customers using a costly feature move the bill without moving call count much. Not a bug, but worth ruling in before you go hunting for one.

If the vendor exposes per-key usage breakdowns in their dashboard, split your credentials by service before your next incident. Separate API keys per environment and per major consumer turn a whodunnit into a two-minute lookup.

Common mistakes

• Optimising the endpoint that’s easy to change rather than the one that dominates the bill. Rank by spend first.
• Caching without a per-credential key, and finding out the hard way in a multi-tenant system.
• Treating 304 responses as free when your specific vendor bills them anyway.
• Setting TTLs from a savings target rather than a staleness tolerance, and finding out during a customer escalation.
• Leaving retry logic enabled at three separate layers and calling the result “defence in depth”.
• Adding per-tenant labels to outbound-call metrics and moving the cost from the API vendor to the observability vendor.
• Shipping the optimisation and never checking the next invoice against the prediction.

Best practices worth keeping

• One outbound HTTP client wrapper per service, with timeouts, retries, and metrics configured in exactly one place.
• Separate API credentials per environment and per major consumer, so usage dashboards mean something.
• Cache auth tokens until shortly before expiry. Refresh early, not on every call.
• Add jitter to scheduled jobs as well as to retries. Everything set to run at the top of the hour arrives at the vendor as a spike.
• Write down the staleness tolerance for each cached resource next to its TTL, so the next person doesn’t have to reverse-engineer your reasoning.
• Re-check the pricing page after any vendor product update. Billing units change more often than endpoints do.

FAQ

How do I find out which service is driving my API spend?

Start with the vendor’s own usage dashboard broken down by API key. If all your services share one key, that’s the first thing to fix. Failing that, log every outbound call at the client wrapper with the endpoint and calling service, and aggregate for a week. A week of honest logs beats a month of theorising.

Do ETags actually reduce SaaS API costs, or just help with rate limits?

It depends entirely on the vendor. GitHub documents that an authorised conditional request returning 304 doesn’t count against the primary rate limit. Others count every request regardless of status. Run a controlled test: a known number of conditional calls over a billing period, then compare against the usage report. Don’t assume either way.

Is caching third-party API responses allowed?

Usually yes, but not always, and some terms of service restrict how long you may store certain data. Mapping and place data is a common example of restricted caching. Check the terms for the specific vendor before you build a permanent store, particularly if you’re caching anything personally identifiable.

How much can caching realistically cut an API bill?

Nobody can give you a number without knowing your traffic. Estimate it yourself: for a candidate endpoint, count distinct request keys versus total requests over a day. That ratio is roughly your ceiling for hit rate. Multiply by the endpoint’s share of spend. If the answer is a couple of percent, spend your time elsewhere.

Should I always replace polling with webhooks?

No. The comparison is change frequency against poll frequency. If the resource changes almost as often as you poll, you save little and take on signature verification, redelivery handling, and a reconciliation job. Webhooks win decisively on slow-changing resources and on anything where you’re polling more than a handful of objects.

Does rate limiting my own users reduce vendor costs?

Only when user actions map directly to vendor calls, such as a search box that fires an upstream request per keystroke. Debouncing input and enforcing per-plan quotas helps a lot there. For background processing driven by your own schedules, user rate limits change nothing.

Should I disable retries to save money?

No. Fix them instead. Capped exponential backoff with jitter, no retries on non-429 client errors, a retry budget, and retry logic living at exactly one layer. Disabling retries entirely trades a cost problem for a reliability problem, and reliability problems are more expensive.

The one thing worth remembering

Most efforts to reduce SaaS API costs fail because they start with code. They should start with two questions: what is the vendor actually counting, and which of my calls returned information I already had?

Answer those honestly and the work usually collapses to a handful of unglamorous changes. Raise a page size. Cache a token. Put a lock around a cache refresh. Delete a poller that was watching something that changes twice a week. None of it is clever. All of it shows up on the next invoice, which is the only review that counts.

Need help getting your API spend under control?

This is work I take on regularly. Typical engagements look like:

• Auditing outbound API traffic to find which endpoints, jobs and tenants are driving the bill, and ranking them by spend rather than by how obvious they are.
• Building a caching layer in front of chatty vendor APIs, with Nginx, Redis or Cloudflare Workers, including stampede protection and per-tenant key isolation.
• Replacing polling jobs with webhook receivers or delta-token syncs, with signature verification, idempotent handling and a dead-letter path.
• Auditing timeout and retry configuration across services to remove stacked retries and add proper backoff, jitter and retry budgets.
• Moving latency-tolerant workloads onto batch or queue-based paths, with deduplication windows that cut call volume before it reaches the vendor.
• Setting up call-rate and cache-hit-rate dashboards and alerts in Prometheus, Grafana, Datadog or CloudWatch so the invoice is never the first signal.

If you’ve got a cost graph that’s climbing, a job you suspect is over-polling, or a client wrapper you’re not sure about, send it over and I’ll tell you what I’d look at first.

The post Optimizing API Calls to Reduce SaaS Costs: Six Levers That Actually Move the Bill appeared first on John Nessime.

That framing is where most of these arguments go sideways. On one box serving a normal website, all three of these servers will handle your traffic without breaking a sweat. The web server is almost never the slow part. The difference shows up months later, at 2am, in who gets paged and why.

So this post compares Nginx vs Apache vs Caddy on the three things that genuinely change your on-call life: how each one behaves when it runs out of capacity, who owns the certificate lifecycle, and where configuration lives when someone other than you needs to change it. Then a decision procedure, the mistakes I keep running into, and an FAQ.

Why the benchmark comparison picks the wrong winner

Almost every throughput comparison you’ll find measures static file serving on an idle machine with no application behind it. That is a real measurement of a thing that is not your bottleneck. If you’re proxying to PHP-FPM, Gunicorn, or a Node process, the request spends milliseconds in the web server and everything else in your code and your database.

The honest exception: if you are running a CDN origin, terminating TLS at a high connection rate, or serving genuinely large volumes of static assets, the connection handling model does show up in your bill. That is a real workload and Nginx has earned its reputation there. It just isn’t most projects.

For everyone else, the useful question is not “which is fastest” but “which chores am I choosing to keep.” That’s what the rest of this is about.

Failure family one: the process model decides how you fall over

This is the invisible one. Nobody notices it during a migration and everybody notices it during a traffic spike.

Apache: a hard worker ceiling, and a queue behind it

Apache’s behaviour depends entirely on which Multi-Processing Module is loaded. With prefork, you get one process per connection, capped by MaxRequestWorkers. When you hit that cap, new connections don’t get rejected. They sit in the kernel’s listen backlog waiting for a worker to free up.

The symptom is the worst kind: the site “hangs.” Load average is unremarkable. The application log is quiet because the requests never reached the application. Your uptime check times out but a manual curl from the box works fine. People spend hours looking at the database.

Check which MPM you’re actually running before you theorise about anything:

# RHEL, AlmaLinux, Rocky
httpd -V | grep -i "Server MPM"

# Debian and Ubuntu
apache2ctl -V | grep -i "Server MPM"

The reason prefork survives is mod_php. It isn’t safe with Apache’s threaded MPMs, so PHP stacks historically pinned themselves to prefork and inherited its concurrency profile. The Apache project itself now points people at PHP-FPM over mod_proxy_fcgi instead, which frees you to run mpm_event and gets you HTTP/2 as a side effect, since mod_http2 disables itself under prefork.

If you inherit an Apache box, that migration is usually the single highest-value change available to you, and it does not require leaving Apache.

Nginx: cheap connections, different limits

Nginx runs a small number of worker processes, each with an event loop. An idle keepalive connection costs a file descriptor and a little memory, not a process. Ten thousand of them is not a crisis.

Its ceiling is worker_processes multiplied by worker_connections, plus whatever your file descriptor limit allows. When you hit it, Nginx says so plainly in the error log rather than silently queueing. That’s a genuine operational advantage: the failure announces itself.

The trade is that when Nginx is proxying, saturation moves downstream. You stop seeing worker exhaustion and start seeing 502s and 504s from an overloaded PHP-FPM pool or app server. The web server is fine; you now have to go find the real limit.

Caddy: goroutines, and a softer edge

Caddy is a Go binary and handles connections with goroutines that the runtime multiplexes onto OS threads. Practically, that sits between the other two: connections are cheap like Nginx, but you don’t tune a worker count at all.

The failure mode is correspondingly softer and, honestly, harder to reason about. Instead of a clean “workers exhausted” line, you get memory growth and garbage collection pressure. There’s no knob you turned wrong, which is pleasant right up until you need to explain what happened.

Failure family two: who owns the certificate lifecycle

This is the one that actually causes outages, and it is the sharpest genuine difference between the three.

With Nginx or Apache, certificate management is a separate moving part. Certbot or another ACME client writes new files to disk on a timer, and the running server keeps serving whatever it loaded at startup. If nothing tells it to reload, the certificate on disk is valid and the certificate on the wire is expired. Users see a browser interstitial while your monitoring, if it checks the filesystem, reports everything fine.

The fix is to attach the reload to the renewal itself, not to a separate cron entry that might drift:

# Run the renewal end to end without touching real certificates.
# This exercises the challenge and the hook, which is the point.
certbot renew --dry-run

# Attach the reload to renewal so it only fires when a cert actually changed
certbot renew --deploy-hook "systemctl reload nginx"

The distinction that matters: --deploy-hook runs only when a certificate was actually renewed, so you’re not reloading the server twice a day for nothing. A blanket post-hook reloads regardless.

Caddy removes this category of failure. Certificate issuance and renewal are built into the server itself, it swaps the certificate in memory when it renews, and there is no external hook to forget. Point several instances at the same storage backend and they coordinate rather than each hammering the CA. If you have ever been paged for an expired certificate, this alone is a serious argument.

It is not free of foot-guns, though. On-demand TLS, where Caddy obtains a certificate during the handshake for a hostname it has never seen, is genuinely useful for multi-tenant setups where customers point their own domains at you. Enabled without restriction, it is an open door: anyone who points a DNS record at your IP can make you request a certificate. Caddy’s own documentation is blunt that it must be both enabled and restricted, and the mechanism is an ask endpoint you host:

{
    on_demand_tls {
        ask http://localhost:9123/ask
    }
}

https:// {
    tls {
        on_demand
    }
    reverse_proxy 127.0.0.1:3000
}

Caddy asks that endpoint whether a hostname is allowed before it will issue. It should be a fast indexed lookup against your own tenant table, returning in milliseconds, because a TLS handshake is being held open while it answers. The older interval and burst rate-limiting options are no longer recommended and should be removed if you still have them in a config.

One more Caddy default worth knowing: the admin API listens on localhost port 2019 and accepts a full config replacement without authentication. That is fine bound to loopback and catastrophic if you helpfully expose it while debugging a container.

Failure family three: where the configuration lives

This is the axis people underweight, and it’s usually the one that actually decides the answer.

Apache supports .htaccess: per-directory configuration, read at request time, changeable by anything that can write to the directory. No reload, no root access. That is why the entire shared hosting world is built on Apache, and why control panels like cPanel and DirectAdmin, and hosts like InterServer, assume it. A WordPress plugin can rewrite its own rules and it just works.

The costs are real and they’re the mirror image of the benefit. With overrides enabled, Apache walks the directory tree looking for .htaccess files on every single request. And your security rules can be silently overridden by whatever gets deployed into a directory, by anyone who can deploy.

Nginx has no equivalent and this is deliberate. Everything is central, and a change requires a reload. Rewrites are handled with try_files rather than per-directory rule files. If you own the box, that’s a feature: nothing changes routing behind your back and the running configuration is exactly what’s in version control. If you host other people’s applications, it’s a support burden, because every customer rewrite rule becomes a ticket for you.

Caddy is central too, either a Caddyfile or JSON. The difference is that config changes can go through the admin API as an atomic swap rather than a file edit plus a signal, which suits automated pipelines nicely.

Nginx: where it wins and where it doesn’t

Where it wins: high connection counts on modest hardware, static and cached content, TLS termination in front of an application fleet, and the sheer weight of the ecosystem. Every stack you’ll meet has a known-good Nginx config, and everyone you hire has already read one.

Where it doesn’t: the DNS behaviour catches people constantly. A hostname written directly into proxy_pass, or into an upstream block, is resolved once at startup and cached for the lifetime of the process. When your upstream is a cloud load balancer or a container that moves, Nginx keeps sending traffic to an IP that stopped being yours. Nothing in the log says “stale DNS”; you just get connection timeouts to an address that looks plausible.

Forcing re-resolution means putting the name in a variable and giving Nginx a resolver, because that’s what moves the lookup from config parse time to request time:

location / {
    # resolver moves lookups to runtime; valid= caps how long a result is reused
    resolver 127.0.0.53 valid=30s ipv6=off;

    # the variable is the trick: it defers evaluation to per-request
    set $upstream_host app.internal.example.com;
    proxy_pass http://$upstream_host:8080$request_uri;
}

Note the $request_uri on the end. Once a variable appears in proxy_pass, Nginx stops doing its usual URI rewriting for you, so you have to pass the path explicitly. Miss that and every request lands on the upstream’s root. Also worth knowing: an upstream block defined elsewhere with the same name will take precedence over the runtime lookup, which produces a very confusing afternoon.

The other honest downside is governance. Nginx is owned by F5, and a longtime core developer forked it to freenginx after a dispute over how security issues were being handled, with an earlier fork, Angie, already in existence. Mainline Nginx remains overwhelmingly dominant and I would not choose a fork for a client project today. But “who decides what ships” is a fair question to ask about any dependency, and this one now has a public answer.

Apache: where it wins and where it doesn’t

Where it wins: anywhere applications need to configure the web server without touching the web server. Shared hosting, multi-tenant WordPress, legacy applications that ship their own rewrite rules. The module ecosystem is enormous and modules load at runtime rather than requiring a rebuild. And the documentation is, quietly, the best of the three: nearly every directive has a page explaining its interactions.

Where it doesn’t: memory per connection, if you’re still on prefork. Slow clients holding processes open is the specific scenario that hurts, because a process that’s waiting on a phone on bad mobile data is a process nobody else can use. Putting a CDN or a proxy in front absorbs a lot of that, which is a large part of why Apache origins behind Cloudflare behave so much better than bare ones.

I would not migrate a working Apache installation to Nginx purely for performance. I would move it from prefork and mod_php to mpm_event and PHP-FPM first, then measure, and quite often stop there.

Caddy: where it wins and where it doesn’t

Where it wins: anything where certificate management is the operational risk. Small teams without a dedicated ops person, internal tooling with a long tail of hostnames nobody tracks, multi-tenant products serving customer-owned domains, container stacks where a single static binary with no runtime dependencies is a real simplification. A working HTTPS reverse proxy is genuinely a few lines.

app.example.com {
    encode zstd gzip
    reverse_proxy app:3000
}

That obtains a certificate, redirects HTTP to HTTPS, negotiates compression and proxies, with no ACME client, no cron, no reload hook.

Where it doesn’t: the ecosystem gap is real. When something unusual breaks, there are fewer Stack Overflow answers and fewer engineers who have seen it before. Adding a plugin means rebuilding the binary with xcaddy rather than dropping in a module, which is fine in CI and annoying on a box you SSH into. And there’s a subtler cost: automation you don’t understand is still automation you’re responsible for. Being paged about ACME behaviour you’ve never had to learn is a worse afternoon than being paged about Certbot.

On upstream DNS, Caddy dials static upstreams by name so resolution happens at dial time rather than being frozen at config load. When the set of addresses changes, the dynamic a upstream module with a refresh interval is the built-in answer, which is a cleaner story than Nginx’s variable trick.

A decision procedure for Nginx vs Apache vs Caddy

Work down this list and stop at the first one that matches. It’ll be right more often than a benchmark.

1. Do applications you don’t control need to change server behaviour? If yes, Apache. Nothing else has .htaccess, and reimplementing it in tickets is not a plan.
2. Is TLS renewal your top operational risk? Small team, no dedicated ops, lots of hostnames, or customer-owned domains: Caddy. It deletes a whole class of incident.
3. Are you terminating a very high connection rate, or serving heavy static content? Nginx. This is the workload it was designed around.
4. Does the team already run one of these well? Keep it. Operational familiarity beats a 20% throughput difference you were never going to measure.
5. Is there a working Apache install and no specific complaint? Fix the MPM and PHP handler, then reassess. Most “we need Nginx” tickets are actually “we need to stop using prefork.”
6. None of the above? Nginx for anything that will be handed to another engineer. Caddy for anything you’ll run yourself.

Common mistakes

• Renewing certificates without reloading. The single most common self-inflicted TLS outage on Nginx and Apache. Use a deploy hook and verify it with a dry run.
• Migrating for benchmark numbers. You inherit a new failure mode and a team that has never debugged it, in exchange for latency nobody could perceive.
• Leaving Apache on prefork after moving to PHP-FPM. The migration is only half done, and you’re still paying prefork’s memory cost while also giving up HTTP/2.
• Hostnames baked into proxy_pass with dynamic upstreams. Works perfectly until the upstream IP changes, then fails in a way that looks like a network problem.
• Enabling Caddy’s on-demand TLS without an ask endpoint. You’ve handed strangers the ability to make you request certificates.
• Exposing Caddy’s admin API beyond loopback. Unauthenticated full config replacement is exactly as bad as it sounds.
• Monitoring the certificate file instead of the handshake. The file is almost never the thing that’s wrong. Check what the server actually presents, from outside the box.

How I’d decide, in practice

• Pick based on who has to change the config six months from now, not on requests per second today.
• Whichever you choose, dump the effective configuration rather than reading files: nginx -T, apachectl -S, caddy adapt --config /etc/caddy/Caddyfile. Include files hide surprises.
• Monitor TLS expiry as an external handshake check from a service like Better Stack or a Grafana Cloud synthetic, not as a filesystem check on the box.
• Put something in front. A CDN like Cloudflare or Fastly absorbs slow clients and connection floods, which flattens most of the architectural differences here.
• Run one server type per environment. Mixing Nginx in staging and Apache in production means your rewrite rules are untested by definition.
• Validate before reloading, every time. nginx -t, apachectl configtest and caddy validate all exist so you don’t find out from users.

Frequently asked questions

Is Caddy production ready?

Yes. It’s been running real workloads for years and its certificate automation removes more operational risk than its smaller ecosystem adds, for most teams. The caveat is support surface, not stability: when you hit something unusual, there are fewer people who have hit it before you.

Can I run Nginx and Apache together?

Yes, and it’s a common pattern: Nginx on 80 and 443 handling TLS and static files, proxying dynamic requests to Apache on a local port so applications keep their .htaccess behaviour. You get Nginx’s connection handling and Apache’s per-directory config. The cost is two servers to patch, two logs to correlate, and real client IP to forward correctly.

Which is best for WordPress?

Apache if plugins need to write rewrite rules, which on a typical multi-plugin site they do. Nginx if you control the site and are willing to translate those rules into try_files and static location blocks once. Plenty of high-traffic WordPress runs on Nginx; it just means plugin-generated rules are your job now.

Does Nginx really re-resolve DNS for upstreams?

Not by default. A literal hostname in proxy_pass or an upstream block is resolved at startup and reused indefinitely, regardless of TTL. You get runtime resolution by using a variable together with a resolver directive, and remembering that this changes how the request URI is passed.

Should I switch from mod_php to PHP-FPM?

In almost all cases yes. It’s the change that unlocks Apache’s threaded MPMs and HTTP/2, and it separates PHP’s lifecycle from the web server’s so you can restart one without the other. Check first for anything relying on PHP settings in Apache config or .htaccess, since those move to a .user.ini file with different syntax.

Does the freenginx fork mean I should stop using Nginx?

No. Mainline Nginx under F5 remains the overwhelmingly dominant option with the broadest packaging and support. The fork is worth knowing about as context on project governance rather than as a migration target, and I wouldn’t put a client on a fork without a specific reason.

Which uses the least memory?

Nginx, generally, with Caddy close behind and Apache furthest away when running prefork. But the gap narrows sharply once Apache is on mpm_event, and on a VPS from a provider like Hetzner or DigitalOcean the memory your application uses will dwarf all three. Measure your own stack before treating this as a deciding factor.

The one thing worth remembering

The Nginx vs Apache vs Caddy question is not a performance question. All three are fast enough for what you’re doing. What differs is which operational chores each one hands back to you: Apache gives applications control at the cost of memory and predictability, Nginx gives you central control at the cost of owning every rewrite rule and the certificate reload, and Caddy takes the certificate lifecycle off your plate at the cost of a smaller ecosystem when things get strange.

Pick the set of chores your team can actually carry. Then go fix the thing that’s genuinely slow, which is almost certainly not the web server.

Need a second opinion on your web server setup?

Most of the work I do here is unglamorous and pays for itself quickly. Things I can help with:

• Auditing an existing Nginx, Apache or Caddy config and telling you what will break under load before it does
• Migrating Apache from prefork and mod_php to mpm_event with PHP-FPM, without breaking existing .htaccess behaviour
• Translating a working Apache rewrite set into an Nginx configuration, with a test plan so you know it matches
• Fixing certificate renewal that runs but never reloads, including deploy hooks and external expiry monitoring
• Setting up Caddy for multi-tenant or customer-domain hosting with on-demand TLS properly restricted
• Diagnosing intermittent 502s, 504s and stale-upstream timeouts that look like network faults and aren’t

Send me your config, an error log excerpt, or the output of nginx -T and tell me what you’re seeing. I’d rather look at the real thing than guess from a description.

The post Nginx vs Apache vs Caddy: What Actually Decides It in Real Projects appeared first on John Nessime.

Your server is only sending the leaf certificate. The intermediate that links it to a trusted root is missing. Your browser papers over that: it either has the intermediate cached from another site, or it quietly fetches it using the AIA extension in the certificate. curl, Java, Go, Python and every other non-browser client do no such thing. They see a certificate signed by something they do not trust and stop.

This is the most common of all TLS certificate errors and it is close to invisible from a desktop. The padlock in your browser is not a test. It is a client with a large safety net doing you a favour.

This post covers the four things a client actually checks and how each one fails: the chain, the names, the clock, and the trust store. Then the automation half, which has its own quiet failure: a renewal that succeeds and never reaches the running server. Plus the two things about TLS that have genuinely changed recently and will break config you copied from an older guide.

Look at what the server is actually sending

Before theorising, get the facts. One command tells you almost everything.

# -servername sends SNI. Without it you get whatever the default
# vhost serves, which is a different certificate on most servers
# and the source of a lot of confusing results.
# -showcerts prints every certificate the server sent, in order.
openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null

Read the top of that output. There is a Certificate chain block listing what arrived, numbered from zero, with s: for subject and i: for issuer. Entry 0 is your certificate. Entry 1 should be the intermediate that issued it, and its subject should exactly match entry 0’s issuer.

If the chain block has only entry 0, that is your bug. Right at the bottom, Verify return code gives the verdict: 0 (ok) is what you want, 21 (unable to verify the first certificate) means the chain is incomplete, and 20 (unable to get local issuer certificate) means the issuer is not in the local trust store.

Run it from a machine that is not yours. A clean container is ideal, because your laptop has accumulated intermediates and corporate roots that a customer’s server has not.

Chain problems

Almost all of these come down to one file choice. ACME clients write out several files and two of them look interchangeable:

• cert.pem is the leaf alone. Using this is the mistake.
• chain.pem is the intermediates alone.
• fullchain.pem is leaf plus intermediates, in the right order. This is the one your server should point at.

# nginx
ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

# Apache. On current versions the chain goes in SSLCertificateFile.
# SSLCertificateChainFile is deprecated; guides that still use it
# are old enough that other advice in them is probably stale too.
SSLCertificateFile      /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile   /etc/letsencrypt/live/example.com/privkey.pem

Order matters as well as contents. The leaf must come first, then each intermediate that signs the one above it. A chain assembled in the wrong order will be rejected by strict clients even though every certificate in it is correct. You can check a chain offline before deploying it:

# Does this leaf chain to a trusted root using these intermediates?
# Exits non-zero and explains itself if not.
openssl verify -untrusted chain.pem cert.pem

Do not include the root certificate in the file you serve. The client either has it or does not, and sending it wastes bytes on every handshake without changing any outcome.

Name problems

The second family. A certificate can be perfectly valid and still be the wrong certificate for the name being requested.

# The hostname the client asked for must appear in here.
openssl x509 -in cert.pem -noout -text | grep -A1 "Subject Alternative Name"

Three things people get wrong here:

• The Common Name does not count. Modern clients ignore the CN field entirely for hostname matching. If the name is only in the CN and not in the SAN list, it will fail, and the error message will not explain that clearly.
• Wildcards are exactly one label deep. *.example.com covers www.example.com and api.example.com. It does not cover example.com itself, and it does not cover a.b.example.com. The bare domain has to be listed separately, and it usually is not.
• SNI decides which certificate you get. One IP serving many sites picks the certificate based on the name the client sends during the handshake. A client that does not send SNI, or an openssl command without -servername, gets the default vhost’s certificate and produces a name mismatch that has nothing to do with your actual configuration.

Time and trust problems

# Local file
openssl x509 -in cert.pem -noout -dates

# Live endpoint, subject and dates in one go
openssl s_client -connect example.com:443 -servername example.com </dev/null 2>/dev/null 
  | openssl x509 -noout -dates -subject

Expired is obvious. notBefore in the future is the interesting one, because it almost never means the certificate is wrong. It means the client’s clock is wrong. Containers with no NTP, a VM restored from a snapshot, an embedded device with a dead battery: all of them produce “certificate not yet valid” against a perfectly good certificate. Check the clock on the failing machine before you touch anything on the server.

Trust store problems look similar but come from a different place. The root that signed your chain has to be in the trust store of the machine making the request, and old systems have old trust stores. A device that has not received an OS update in years may simply not know about a certificate authority that appeared since. There is no server-side fix for that beyond choosing a CA with broader legacy coverage, which is one of the few genuine arguments for a paid certificate.

The renewal that succeeded and changed nothing

Here is the second invisible failure, and it catches experienced people. Your renewal timer runs, the ACME client obtains a new certificate, writes it to disk, logs success. The site goes down anyway.

Nginx read the certificate into memory when it started. It does not watch the file. Neither does Apache, HAProxy, Postfix, or almost anything else. Until the service is reloaded it keeps serving the certificate it loaded at boot, which is the one that just expired. Everything in your logs says the renewal worked, because it did. The delivery step was never wired up.

Certbot handles this with deploy hooks, which run only when a certificate was actually renewed. Put a script in the hooks directory and it applies to every certificate on the box:

#!/usr/bin/env bash
# /etc/letsencrypt/renewal-hooks/deploy/reload-services.sh
# Remember: chmod +x, or it silently never runs.
set -euo pipefail

# Validate the config BEFORE reloading. A reload with a broken
# config can take the site down at the exact moment you were
# trying to keep it up.
nginx -t
systemctl reload nginx

# Anything else that holds a copy of the certificate.
# systemctl reload postfix

Then prove the whole path works without waiting sixty days to find out:

# Exercises the full renewal against the staging environment.
# Installs nothing, counts against no rate limits, and will
# fail loudly if your challenge setup is broken.
certbot renew --dry-run

One caveat worth knowing: --dry-run does not run deploy hooks by default, precisely because nothing was deployed. So it validates the issuance path but not the reload. Test that separately by touching the service and confirming the served certificate changes.

Expiry automation, and why it is no longer optional

This part has a date attached, so it is worth stating plainly rather than dancing around it.

The CA/Browser Forum approved a phased reduction of maximum certificate lifetimes. The cap dropped from 398 days to 200 days on 15 March 2026, falls to 100 days on 15 March 2027, and reaches 47 days on 15 March 2029. Domain validation reuse periods shrink alongside it.

The practical consequence: any process that involves a person remembering to renew something once a year is already broken, and by the third phase you are renewing every six weeks. Whatever you were getting away with manually, stop. ACME with a timer is the baseline. Certbot is the default choice, acme.sh is lighter and easier to script around, and Caddy obtains and renews certificates with no configuration at all, which is a genuinely good reason to consider it for a simple reverse proxy.

Choosing a challenge type

• HTTP-01 proves control by serving a file over port 80. Simple, and it breaks whenever port 80 is closed, redirected wholesale to HTTPS in a way that mangles the challenge path, or sitting behind a proxy that does not forward it.
• DNS-01 proves control by writing a TXT record. More setup, needs an API token for your DNS provider, and it is the only option for wildcard certificates. It also works for hosts with no inbound internet access at all, which is why I reach for it on anything internal.

The trade-off with DNS-01 is a credential: your renewal process now holds a token that can edit your DNS zone. Scope it to the narrowest permission your provider offers, and if they support delegating just the _acme-challenge records to a separate zone, do that instead.

Monitor expiry externally

Do not rely on the renewal system reporting its own health. If it is broken, it is also probably not telling you. Let’s Encrypt has discontinued its expiration reminder emails, so if that was quietly acting as your safety net, it is not one any more.

Check from outside, against the live endpoint, the way a user would:

#!/usr/bin/env bash
set -euo pipefail

HOST="example.com"
DAYS=21

# -checkend takes a number of seconds and exits non-zero if the
# certificate expires within that window. No parsing dates by hand.
if ! openssl s_client -connect "$HOST:443" -servername "$HOST" </dev/null 2>/dev/null 
     | openssl x509 -noout -checkend $(( DAYS * 86400 )); then
  echo "TLS certificate for $HOST expires within $DAYS days" >&2
  exit 1
fi

If you already run Prometheus, the blackbox exporter gives you probe_ssl_earliest_cert_expiry for free, which is a timestamp you can alert on and graph in Grafana across every endpoint you own. That is the better answer at more than a handful of hosts, because it also catches the certificates nobody remembered existed.

Two pieces of config that are now wrong

OCSP stapling with Let’s Encrypt. Let’s Encrypt removed the OCSP responder URL from newly issued certificates in May 2025 and shut down its responders in August 2025. If your config still has this, it does nothing except print a warning on every reload:

# nginx: [warn] "ssl_stapling" ignored, no OCSP responder URL in the certificate
# Comment these out if your certificates come from Let's Encrypt.
# ssl_stapling on;
# ssl_stapling_verify on;

Nothing breaks by leaving them, but a warning that appears on every reload is a warning people learn to ignore, and then they ignore the next one too.

Hand-tuned cipher suite lists. If you pasted a ssl_ciphers line from a guide more than a couple of years old, re-check it against the current Mozilla SSL Configuration Generator. Lists that were considered hardened at the time now sometimes exclude things you want or include things you do not, and the defaults have improved considerably.

When there is a CDN in front

Behind Cloudflare or a similar edge, there are two separate TLS connections: browser to edge, and edge to your origin. They use different certificates and fail independently.

This creates a specific blind spot. Your origin certificate can be expired, self-signed or serving the wrong name, and visitors will see nothing wrong, because the edge presents its own valid certificate and, in the more permissive encryption modes, does not verify the origin. It works until you switch to strict origin validation, or until the edge is bypassed, and then everything fails at once for reasons that look unrelated to anything you changed.

Test the origin directly by name, bypassing DNS. If your origin is a VPS with a provider like InterServer and the public DNS points at the CDN, this is the only way to see what the origin is really serving:

# --resolve forces the hostname to a specific IP without touching
# /etc/hosts, so SNI and Host header stay correct.
curl -v --resolve example.com:443:203.0.113.10 https://example.com/

Troubleshooting by error message

unable to get local issuer certificate

Missing intermediate on the server, or a trust store on the client that lacks the root. Check the server first with -showcerts; if the chain is complete, the problem is on the client side.

certificate has expired, but I just renewed it

The service was never reloaded. Compare the dates on disk with the dates being served. If they differ, that is your answer, and the fix is a deploy hook rather than another manual renewal.

Hostname mismatch on a name that is definitely in the certificate

Either SNI is not reaching the right vhost, or the name is in the Common Name rather than the SAN list, or you are hitting a wildcard at the wrong depth. Check the SAN list and re-run s_client with an explicit -servername.

Works over IPv4, fails over IPv6 (or the reverse)

Two different machines are answering, and only one of them was updated. Force the family with curl -4 and curl -6 and compare. Load balancers with one member out of sync produce the same intermittent pattern.

Renewal fails with a challenge error

For HTTP-01, confirm that http://example.com/.well-known/acme-challenge/test is reachable from outside without being redirected or rewritten. A blanket redirect to HTTPS is the usual culprit. For DNS-01, check that the TXT record actually appears with dig, and give slow-propagating providers more time.

Everything looks right and it still fails

Run an external scan. SSL Labs or a local run of testssl.sh will check things you did not think to, including protocol support and chain issues from the perspective of clients you do not have installed.

Common mistakes

• Pointing the server at cert.pem instead of fullchain.pem.
• Treating a padlock in your own browser as proof the certificate is correctly deployed.
• Testing with openssl s_client without -servername and debugging the wrong certificate.
• Assuming *.example.com covers the bare domain.
• Putting the hostname only in the Common Name.
• Renewing without reloading the service.
• Forgetting chmod +x on a deploy hook, so it silently never runs.
• Relying on the renewal system to tell you when it is broken.
• Including the root certificate in the served chain.
• Leaving OCSP stapling enabled with certificates that no longer support it.
• Ignoring the origin certificate because the CDN hides its state.
• Debugging a “not yet valid” certificate on the server when the client’s clock is wrong.

Best practices

• Serve the full chain, leaf first, without the root.
• Verify from a clean machine or container, never from the laptop you browse on.
• Put every name you serve in the SAN list, including the bare domain alongside any wildcard.
• Automate renewal with ACME and a timer, and run certbot renew --dry-run after any change to the setup.
• Attach a deploy hook that validates the config before reloading, and make it executable.
• Monitor expiry externally against the live endpoint, and alert with weeks of headroom.
• Use DNS-01 for wildcards and internal hosts, with a narrowly scoped API token.
• Keep NTP running everywhere, including inside containers.
• Test the origin directly when there is a CDN in front, not just the public hostname.
• Inventory your certificates, because the one that takes you down is the one nobody remembered.

FAQ

Why does my certificate work in Chrome but fail in curl?

Your server is not sending the intermediate certificate. Browsers work around this by caching intermediates from other sites and by fetching them from the URL in the certificate’s AIA extension. Command-line tools and application HTTP libraries do neither. Switch the server to fullchain.pem and reload.

What is the difference between cert.pem, chain.pem and fullchain.pem?

Leaf only, intermediates only, and both concatenated in the correct order. Web servers want fullchain.pem. The private key stays in privkey.pem and never goes in any of them.

Does a wildcard certificate cover the root domain?

No. *.example.com matches exactly one label, so it covers www.example.com but not example.com and not a.b.example.com. Request the bare domain as an additional SAN, which most ACME clients will do if you pass both names.

How often should certificates renew now?

Whatever your CA issues, renew at roughly two-thirds of the lifetime so a failed attempt has room to retry before anything expires. Since maximum lifetimes are on a published downward schedule, build for automated renewal at any interval rather than tuning a number you will have to change again.

Should I still enable OCSP stapling?

Not with Let’s Encrypt certificates, which no longer carry a responder URL. If your CA still supports it, stapling remains worthwhile. Check whether your certificate actually contains an OCSP URL before enabling the directives, rather than assuming.

Is a paid certificate more secure than a free one?

Cryptographically, no. The encryption is identical. What money buys is broader compatibility with old trust stores, longer validation reuse, warranties, and human support when something goes wrong. For a normal public website, a free automated certificate is the right default.

How do I get a certificate for something with no public DNS or open ports?

DNS-01, on a public domain you control, pointed at whatever internal address you like. The CA only needs to see the TXT record; it never needs to reach the host. This is how internal services get publicly trusted certificates without being exposed.

The one thing to remember

Most TLS certificate errors are not about cryptography. They are about delivery: whether the server sends everything the client needs, whether the name it sends matches the name that was asked for, and whether the certificate sitting in memory is the same one sitting on disk.

So stop checking with your browser. Check with openssl s_client from a machine that owes you no favours, and make sure something outside your server is watching the expiry date. As lifetimes keep shrinking, the gap between a renewal process that works and one that merely appears to work gets narrower every year.

Certificates causing you trouble?

Certificate problems tend to arrive at bad moments and turn out to be one wrong file path. Things I help with:

• Diagnosing a certificate that works for some clients and not others, and telling you exactly which link in the chain is missing.
• Setting up ACME renewal with deploy hooks across nginx, Apache, HAProxy, mail servers and anything else holding a copy.
• Migrating to DNS-01 for wildcards and internal hosts, with properly scoped API credentials.
• Certificate inventory and expiry monitoring in Prometheus and Grafana, so nothing expires without warning.
• Preparing a certificate estate for shorter lifetimes, replacing anything that still depends on a person remembering.
• Origin and edge TLS configuration behind a CDN, including strict origin validation.

Send me the output of openssl s_client -connect yourhost:443 -servername yourhost -showcerts and the error your failing client reports, and I will tell you what is wrong.

The post It Works in My Browser: Debugging TLS Certificate Errors in Chains, SANs and Expiry appeared first on John Nessime.

That is the failure mode worth taking seriously. Not slowness, not downtime — the CDN cached a page that was personalised for one user and served it to everybody else. And you will not notice, because your browser holds a session and looks fine. The people seeing the broken version are the ones who never contact you.

Getting CDN setup right for a dynamic site is not about finding the right toggle in a dashboard. It is about deciding, per route, what is public and what is personal, and then making the origin say so clearly enough that no edge node has to guess.

Sort your routes before touching any config

Every URL on your site falls into one of three buckets. Write the list out — actually write it, this takes ten minutes and prevents the incident above.

• Static assets. CSS, JS, images, fonts. Identical for everyone, content-addressed if your build is sensible. Cache these hard, for a year.
• Public dynamic. Generated by the application but identical for every anonymous visitor. Blog posts, product pages, category listings. This is where the real performance win lives, and where most people leave everything uncached out of nervousness.
• Private dynamic. Anything personalised or authenticated. Carts, dashboards, checkout, admin, most API responses. Never cached at a shared cache, no exceptions.

The middle bucket is the whole game. If you cache only the first bucket you have built an asset CDN and left the expensive work untouched. If you accidentally cache the third, you have a data leak.

The header that separates the buckets

The single most useful thing to understand is the difference between private and no-store, because people use them interchangeably and they mean very different things.

# Static assets with a content hash in the filename
Cache-Control: public, max-age=31536000, immutable

# Public dynamic: edge caches it, browser revalidates
Cache-Control: public, s-maxage=300, max-age=0, must-revalidate

# Personalised: browser may cache, shared caches must not
Cache-Control: private, no-store

# Genuinely sensitive: nobody stores this anywhere
Cache-Control: no-store

private means “a browser cache may keep this, a shared cache may not”. no-store means “do not write this to disk anywhere”. For an authenticated dashboard you want both. For a bank statement you want no-store alone.

The other key pair is s-maxage versus max-age. s-maxage applies only to shared caches — the CDN — and overrides max-age there. That lets you cache a product page at the edge for five minutes while telling browsers to revalidate every time, so a price change propagates in five minutes rather than living in someone’s browser for an hour.

Set caching headers at the origin, per route, and let the CDN follow them. Configuring cache rules only in the CDN dashboard means the policy lives somewhere your application code cannot see and your staging environment does not have.

Vary, and how it quietly ruins your hit rate

Vary tells caches that the response depends on a request header, so a separate copy is stored per distinct value. Used precisely it is essential. Used carelessly it destroys your cache.

# Fine. A handful of encodings, a handful of variants.
Vary: Accept-Encoding

# Also fine if you genuinely serve different content per language
Vary: Accept-Encoding, Accept-Language

Now the one to avoid:

# Effectively disables caching. Every browser build is a distinct
# User-Agent string, so every visitor gets their own cache entry.
Vary: User-Agent

If you need device-specific responses, most CDNs expose a normalised device-type value with two or three possible values. Use that rather than the raw header.

Vary: Cookie deserves its own warning. It looks like a safe way to separate logged-in from anonymous traffic, and it is not — every visitor carries slightly different cookies from analytics and consent tools, so you get a near-unique cache key per person. The correct approach is to make the presence of a session cookie bypass the cache entirely, rather than varying on it.

Cookies are where sites actually break

Two rules, and between them they prevent most CDN incidents.

Rule one: a session cookie means bypass

If a request arrives carrying your session cookie, it should go to origin uncached. Most CDNs let you express this as a cache rule; in nginx, if you are running your own edge, it is a variable:

map $http_cookie $bypass_cache {
    default                 0;
    "~*wordpress_logged_in" 1;
    "~*woocommerce_items"   1;
    "~*comment_author"      1;
    "~*PHPSESSID"           1;
}

location / {
    proxy_cache            edge_cache;
    proxy_cache_bypass     $bypass_cache;   # do not serve from cache
    proxy_no_cache         $bypass_cache;   # do not write to cache
    proxy_cache_valid      200 301 302 5m;
    proxy_cache_use_stale  error timeout updating http_500 http_502 http_503 http_504;
    proxy_cache_lock       on;

    add_header X-Cache-Status $upstream_cache_status always;
    proxy_pass http://origin;
}

You need both directives. proxy_cache_bypass stops a logged-in user being served a cached anonymous page. proxy_no_cache stops their personalised response being written into the cache for everyone else. Setting only the first is a common and dangerous mistake — the read path is protected while the write path is wide open.

That X-Cache-Status header is not decoration. It is how you verify any of this actually works.

Rule two: strip cookies you do not need

Analytics and consent tools set cookies on every visitor. If your CDN treats any cookie as a bypass signal, your anonymous traffic stops being cacheable the moment you add a tag manager. Configure the CDN to ignore everything except the specific cookies that matter, and be explicit about which those are.

Query strings, and the cache-key explosion

By default many CDNs include the full query string in the cache key. Then a marketing campaign starts appending utm_source, utm_medium, fbclid and friends, and one page becomes thousands of cache entries that each miss exactly once.

Decide deliberately which parameters affect the response:

• Include parameters that change the content — ?page=2, ?sort=price, ?category=boots.
• Ignore tracking parameters entirely. They never change what the server renders.
• Normalise order where the CDN supports it, so ?a=1&b=2 and ?b=2&a=1 are one entry rather than two.

An allowlist is safer than a blocklist here. New tracking parameters appear constantly; the set of parameters your application actually reads changes rarely.

Keeping dynamic pages cached without going stale

The usual objection to caching public dynamic pages is that content changes. Two mechanisms solve this, and together they let you cache aggressively without serving stale content for long.

Stale-while-revalidate

Cache-Control: public, s-maxage=60, stale-while-revalidate=600, stale-if-error=86400

Read that as three separate promises. Fresh for sixty seconds. For the following ten minutes, serve the stale copy immediately while fetching a fresh one in the background — so nobody waits. And if the origin is throwing errors, keep serving the stale copy for a day rather than showing a 502.

That last directive turns your CDN into a partial outage shield. It is one of the highest-value headers on this page and it costs nothing.

Purge on change

Time-based expiry is a blunt instrument. If your application knows when a product price changed, tell the CDN immediately rather than waiting for a TTL.

Tag-based purging is the version worth building toward — tag responses by the entities they contain, then purge by tag when an entity changes:

# Origin response for a product page
Cache-Tag: product-1234, category-boots, layout-main

Update product 1234 and you purge one tag, invalidating the product page, every category listing it appears on, and the homepage if it is featured — without knowing any of those URLs in advance. Header name and support vary by provider, so check what yours calls it.

Verifying it before you point DNS

Test with curl, not a browser. Your browser has cookies, a warm local cache and an opinion; curl tells you the truth.

# What headers is the origin actually sending?
curl -sI https://example.com/products/boots | grep -i 'cache-control|vary|set-cookie'

# Twice in a row: second request should show a HIT
curl -sI https://example.com/products/boots | grep -i 'x-cache|age'
curl -sI https://example.com/products/boots | grep -i 'x-cache|age'

Now the test that matters most. Request an authenticated page with a real session cookie and confirm it never returns a cache hit:

curl -sI https://example.com/account 
  -H "Cookie: session=<a-real-session-value>" 
  | grep -i 'x-cache|cache-control'

You want BYPASS or MISS, and Cache-Control: private. If that ever shows HIT, stop and fix it before going further.

The leak test, which is the one people skip: fetch a personalised page with a session, then fetch the same URL with no cookies at all and confirm you get the anonymous version.

# With a session
curl -s https://example.com/account -H "Cookie: session=<value>" | grep -i 'welcome back'

# Immediately after, with nothing. Should NOT show the personalised content.
curl -s https://example.com/account | grep -i 'welcome back'

If the second command finds personalised content, the CDN cached a private response. That is the Monday-morning email, and you have just caught it before your customers did.

Test the edge before DNS points at it

curl -sI https://example.com/ --resolve example.com:443:<edge-ip>

That sends a real request to a specific edge node with the correct hostname and TLS SNI, without changing DNS. It is how you find out whether the whole thing works before any user does.

Things that break in ways nobody expects

Client IPs all become the CDN’s

Your logs fill with edge node addresses, rate limiting starts throttling everyone at once, and geolocation breaks. The real address arrives in a forwarded header, and your origin has to be configured to trust it — but only from the CDN’s address ranges. Trusting X-Forwarded-For from anywhere lets any visitor claim any IP they like, which turns a logging annoyance into an authentication bypass.

Redirect loops

The CDN talks to your origin over HTTP, the origin sees a non-HTTPS request and redirects to HTTPS, which comes back through the CDN, forever. Either terminate TLS at the origin too, or have the origin trust X-Forwarded-Proto when deciding whether to redirect.

POST responses getting cached

Shouldn’t happen by default, but a badly written cache rule can do it, and the result is one user’s form submission response served to the next person. Only cache GET and HEAD. Ever.

Websockets and streaming

Websocket upgrades and server-sent events need to pass through untouched. Buffering breaks streaming responses silently — the connection works, the data just arrives in one lump at the end, which looks like an application bug rather than a proxy setting.

Set-Cookie on a cacheable response

If a public page returns Set-Cookie and gets cached, every subsequent visitor receives that cookie — potentially somebody’s session identifier. Most CDNs refuse to cache responses carrying Set-Cookie, but not all, and not always. Check your origin is not setting cookies on pages you intend to cache.

Rolling it out without a bad week

1. Write down the route list. Three buckets, every path accounted for.
2. Set headers at the origin first, with the CDN not yet in front. Verify with curl.
3. Start with everything bypassed except static assets. A CDN that caches nothing still terminates TLS closer to users and is not a failure.
4. Add public dynamic routes one at a time, with short TTLs. Sixty seconds, then extend once you trust it.
5. Run the leak test after every rule change, not just at the end.
6. Watch origin request volume. It should drop noticeably. If it does not, something is bypassing that you think is caching.
7. Keep a purge command to hand and know how long a full purge takes before you need it.

Common mistakes

• Testing only in a logged-in browser, where cached-private-content bugs are invisible.
• Setting proxy_cache_bypass without proxy_no_cache, protecting reads but not writes.
• Vary: Cookie or Vary: User-Agent, which shatters the cache into near-unique entries.
• Leaving tracking parameters in the cache key.
• Configuring caching in the CDN dashboard only, so the policy is invisible to your code and absent from staging.
• Trusting forwarded IP headers from any source rather than only the CDN’s ranges.
• Using private where you needed no-store, or the reverse.
• Caching everything on day one and finding the problems in production.

Best practices

• Default to no caching, then allow specific routes. Opt-in beats opt-out when the failure mode is a data leak.
• Own the policy at the origin so it travels with your code and exists in staging.
• Use s-maxage for the edge and max-age for browsers. They are different problems.
• Always set stale-if-error. It is free outage insurance.
• Expose a cache status header and monitor hit ratio per route, not site-wide.
• Build purge-on-publish into the application rather than relying on TTLs alone.
• Make the leak test part of CI, so a future config change cannot reintroduce it quietly.

Frequently asked questions

Can a CDN cache dynamic pages at all?

Yes, and that is where most of the benefit is. Dynamic does not mean personalised. A blog post rendered from a database is identical for every anonymous visitor and can be cached at the edge for minutes. Only genuinely per-user content must bypass.

What is the difference between private and no-store?

private permits browser caching but forbids shared caches. no-store forbids writing the response to any cache at all. Use both for authenticated pages; use no-store alone for genuinely sensitive responses.

Why did my hit ratio collapse after adding a cookie banner?

Because the consent tool sets a cookie on every visitor and your CDN treats any cookie as a bypass signal, or you are varying on Cookie. Configure the CDN to ignore all cookies except the specific ones that indicate a session.

How do I verify private pages are not being cached?

Request the page with a real session cookie and confirm the cache status shows a bypass or miss, never a hit. Then request the same URL with no cookies and confirm the personalised content is absent. Do this after every cache rule change.

Should I cache API responses?

Public read-only endpoints, yes — with short TTLs and a cache key that includes whatever parameters affect the response. Anything requiring an Authorization header should be treated as private by default.

Do I still need caching at the origin?

Yes. The CDN protects you from repeat requests for the same URL; it does nothing for the first request, for uncacheable routes, or during a purge. Origin-level caching handles those.

Wrapping up

A CDN in front of a dynamic site is a classification problem wearing a performance costume. Sort your routes into public, private and static; express that at the origin with Cache-Control; make session cookies a hard bypass on both the read and write path; and keep tracking parameters out of the cache key.

Then test the way an anonymous visitor arrives, not the way you browse your own site. Two curl commands — one with a session, one without — catch the failure that matters, and they take about ten seconds. Run them every time you change a cache rule and the Monday email never arrives.

Want a CDN set up without the surprises?

I work with teams on edge caching for dynamic sites — route classification, cache headers that survive a redesign, purge-on-publish wired into the application, and the monitoring that tells you when hit ratio quietly drops.

If you would rather have this done carefully the first time, you can hire me on Upwork.

The post CDN Setup Without Breaking Dynamic Content appeared first on John Nessime.

For a single server, neither is the right answer. Prometheus and Grafana are two static Go binaries and a package. They run happily under systemd, use very little memory at this scale, and when something breaks you can read the logs and actually fix it. No container networking to debug, no orchestration layer between you and the problem.

This walks through the whole thing: install Prometheus and Grafana from scratch, wire in node_exporter, put the lot behind nginx with TLS, and lock down the ports that should never have been public in the first place. Commands work on both Debian-family and RHEL-family systems, with the differences called out.

What you are building

Four moving parts, all on one box:

• node_exporter exposes machine metrics — CPU, memory, disk, network — on port 9100.
• Prometheus scrapes that endpoint on a schedule and stores the samples in its own time series database on port 9090.
• Grafana queries Prometheus and draws the dashboards on port 3000.
• nginx terminates TLS and is the only thing listening on a public interface.

The critical design decision is in that last line. Prometheus and node_exporter ship with no authentication at all. Bound to a public interface, they hand your entire infrastructure inventory to anyone who runs a port scan. Everything binds to 127.0.0.1 and nginx is the single front door.

Resource-wise, a 2 GB VPS handles this comfortably for one server’s worth of metrics. Prometheus is the memory-hungry one, and its appetite scales with the number of active time series rather than the number of hosts.

Step 1: Users and directories

Neither service should run as root, and neither needs a login shell.

sudo useradd --system --no-create-home --shell /sbin/nologin prometheus
sudo useradd --system --no-create-home --shell /sbin/nologin node_exporter

sudo mkdir -p /etc/prometheus /var/lib/prometheus
sudo chown prometheus:prometheus /etc/prometheus /var/lib/prometheus

On Debian and Ubuntu the nologin shell lives at /usr/sbin/nologin rather than /sbin/nologin. Check with which nologin if useradd complains.

Step 2: Install Prometheus

Prometheus ships as a single static binary. Rather than hardcoding a version that will be stale next month, pull the current release tag from the API:

PROM_VERSION=$(curl -s https://api.github.com/repos/prometheus/prometheus/releases/latest 
  | grep -Po '"tag_name": "vK[^"]*')
echo "Installing Prometheus ${PROM_VERSION}"

cd /tmp
curl -LO "https://github.com/prometheus/prometheus/releases/download/v${PROM_VERSION}/prometheus-${PROM_VERSION}.linux-amd64.tar.gz"
tar xzf "prometheus-${PROM_VERSION}.linux-amd64.tar.gz"
cd "prometheus-${PROM_VERSION}.linux-amd64"

sudo cp prometheus promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus /usr/local/bin/promtool

If you are on ARM, swap linux-amd64 for linux-arm64.

The configuration file

Write /etc/prometheus/prometheus.yml:

global:
  scrape_interval: 15s
  evaluation_interval: 15s

scrape_configs:
  - job_name: prometheus
    static_configs:
      - targets: ["127.0.0.1:9090"]

  - job_name: node
    static_configs:
      - targets: ["127.0.0.1:9100"]
        labels:
          instance: "web-01"

Set that instance label deliberately. It follows every metric into every dashboard and alert, and renaming it later means your historical graphs split in two.

Validate before you start anything:

promtool check config /etc/prometheus/prometheus.yml

The systemd unit

Write /etc/systemd/system/prometheus.service:

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
Restart=on-failure
RestartSec=5
ExecStart=/usr/local/bin/prometheus 
  --config.file=/etc/prometheus/prometheus.yml 
  --storage.tsdb.path=/var/lib/prometheus 
  --storage.tsdb.retention.time=30d 
  --storage.tsdb.retention.size=8GB 
  --web.listen-address=127.0.0.1:9090
ExecReload=/bin/kill -HUP $MAINPID

NoNewPrivileges=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/var/lib/prometheus
PrivateTmp=true

[Install]
WantedBy=multi-user.target

Two things worth noticing. --web.listen-address=127.0.0.1:9090 is what keeps Prometheus off the public internet. And setting both retention flags matters — time-based retention alone will not save you if metric volume spikes, because Prometheus honours whichever limit it hits first.

sudo systemctl daemon-reload
sudo systemctl enable --now prometheus
sudo systemctl status prometheus

Step 3: node_exporter

Same pattern — a static binary and a unit file.

NE_VERSION=$(curl -s https://api.github.com/repos/prometheus/node_exporter/releases/latest 
  | grep -Po '"tag_name": "vK[^"]*')

cd /tmp
curl -LO "https://github.com/prometheus/node_exporter/releases/download/v${NE_VERSION}/node_exporter-${NE_VERSION}.linux-amd64.tar.gz"
tar xzf "node_exporter-${NE_VERSION}.linux-amd64.tar.gz"
sudo cp "node_exporter-${NE_VERSION}.linux-amd64/node_exporter" /usr/local/bin/
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter

Then /etc/systemd/system/node_exporter.service:

[Unit]
Description=Prometheus Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
Restart=on-failure
ExecStart=/usr/local/bin/node_exporter 
  --web.listen-address=127.0.0.1:9100

NoNewPrivileges=true
ProtectHome=true
PrivateTmp=true

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now node_exporter

# Confirm it is actually producing metrics
curl -s http://127.0.0.1:9100/metrics | head -20

Now open http://127.0.0.1:9090/targets — via an SSH tunnel, since it is not public — and both jobs should read UP.

# From your laptop
ssh -L 9090:127.0.0.1:9090 you@your-server

Step 4: Install Grafana

Grafana is packaged properly, so use the official repository rather than a tarball. It gets you signed updates through your normal patching process.

Debian and Ubuntu:

sudo apt-get install -y apt-transport-https software-properties-common wget
sudo mkdir -p /etc/apt/keyrings/
wget -q -O - https://apt.grafana.com/gpg.key 
  | gpg --dearmor | sudo tee /etc/apt/keyrings/grafana.gpg > /dev/null

echo "deb [signed-by=/etc/apt/keyrings/grafana.gpg] https://apt.grafana.com stable main" 
  | sudo tee /etc/apt/sources.list.d/grafana.list

sudo apt-get update
sudo apt-get install -y grafana

RHEL, AlmaLinux, Rocky:

sudo tee /etc/yum.repos.d/grafana.repo > /dev/null <<'EOF'
[grafana]
name=grafana
baseurl=https://rpm.grafana.com
repo_gpgcheck=1
enabled=1
gpgcheck=1
gpgkey=https://rpm.grafana.com/gpg.key
sslverify=1
sslcacert=/etc/pki/tls/certs/ca-bundle.crt
EOF

sudo dnf install -y grafana

Bind it to localhost too

Edit /etc/grafana/grafana.ini:

[server]
http_addr = 127.0.0.1
http_port = 3000
domain = metrics.example.com
root_url = https://metrics.example.com/

[security]
admin_user = admin
admin_password = put-something-real-here
cookie_secure = true

[users]
allow_sign_up = false

Getting root_url right matters more than it looks. Set it wrong and Grafana generates broken links and redirects behind the proxy, which produces a confusing half-working UI rather than a clean error.

sudo systemctl enable --now grafana-server

Step 5: Put nginx and TLS in front

This is the only piece that faces the internet.

server {
    listen 443 ssl;
    server_name metrics.example.com;

    ssl_certificate     /etc/letsencrypt/live/metrics.example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/metrics.example.com/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:3000;
        proxy_set_header Host              $host;
        proxy_set_header X-Real-IP         $remote_addr;
        proxy_set_header X-Forwarded-For   $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }

    # Grafana Live needs websockets, and breaks silently without this
    location /api/live/ {
        proxy_http_version 1.1;
        proxy_set_header Upgrade    $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_set_header Host       $host;
        proxy_pass http://127.0.0.1:3000;
    }
}

Note fullchain.pem rather than cert.pem. Getting that wrong produces a site that works in your browser and fails for anything else.

Then close the metrics ports at the firewall, belt and braces:

# firewalld
sudo firewall-cmd --permanent --add-service=https
sudo firewall-cmd --reload

# ufw
sudo ufw allow 443/tcp

# Verify nothing unexpected is listening publicly
sudo ss -tlnp | grep -E '9090|9100|3000'

Every line of that last command should show 127.0.0.1. If you see 0.0.0.0 or *, something is exposed and you should fix it before going any further.

Step 6: Connect Grafana to Prometheus

You can click through the UI, but provisioning it as a file means a rebuilt server comes back identical. Create /etc/grafana/provisioning/datasources/prometheus.yml:

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    url: http://127.0.0.1:9090
    isDefault: true

Restart Grafana, log in, and import dashboard ID 1860 from the Grafana dashboard library — Node Exporter Full. It is comprehensive, well maintained, and will show you more about your server than you knew was measurable.

Sizing the disk before it bites

Prometheus storage is predictable. Roughly:

disk = retention_seconds × samples_per_second × bytes_per_sample

Compressed samples land in the region of one to two bytes each. Measure your actual rate rather than guessing — run this in the Prometheus query box:

rate(prometheus_tsdb_head_samples_appended_total[5m])

One server with node_exporter at a 15 second interval is a small number, and 30 days fits in a few gigabytes. The number climbs fast once you add exporters, so keep --storage.tsdb.retention.size set as a hard ceiling. Prometheus filling the root partition takes the rest of the box down with it, which is a memorable way to discover your monitoring has no monitoring.

Troubleshooting

A target shows DOWN

Work outward from the exporter:

curl -s http://127.0.0.1:9100/metrics | head
sudo systemctl status node_exporter
sudo journalctl -u prometheus -n 50 --no-pager

On RHEL-family systems with SELinux enforcing, this is frequently SELinux rather than your config. Check before you start changing things:

sudo ausearch -m avc -ts recent

If that shows denials, write a policy for them. Do not reach for setenforce 0 — it fixes today’s symptom and quietly removes a layer of protection from the whole machine.

Grafana loads but every panel says “No data”

Almost always the datasource URL. It must be http://127.0.0.1:9090 from Grafana’s point of view, not your public hostname. Test the datasource from its settings page — Grafana reports the actual connection error, which is more useful than the empty panel.

Prometheus will not start

sudo journalctl -u prometheus -n 50 --no-pager
promtool check config /etc/prometheus/prometheus.yml
ls -la /var/lib/prometheus

The usual culprits are a YAML indentation error, or /var/lib/prometheus not being owned by the prometheus user after a manual file copy.

Broken styling or redirect loops behind the proxy

root_url in grafana.ini does not match how you are actually reaching the site. If you are serving from a subpath, you also need serve_from_sub_path = true.

Common mistakes

• Leaving 9090, 9100 or 3000 on a public interface. Prometheus has no authentication whatsoever.
• Keeping the default Grafana admin password because it prompts you to change it and you clicked past.
• Setting no retention limits and discovering the problem when the root partition fills.
• Dropping scrape_interval to 1s because more data seems better. It multiplies storage and buys you nothing at this scale.
• Never backing up /var/lib/grafana/grafana.db, which holds every dashboard you have built.
• Running everything as root because the tutorial you followed did.
• Building dashboards but configuring no alerts, so you still find out from a user.

Best practices

• Bind every component to localhost and expose exactly one TLS endpoint.
• Set both time and size retention. Whichever triggers first is the one you want.
• Provision datasources and dashboards as files, so the config lives in version control.
• Back up the Grafana SQLite database on the same schedule as everything else that matters.
• Add an external uptime check. Self-hosted monitoring cannot tell you that the server it lives on has gone.
• Write a handful of alerts you will genuinely act on. Alerts nobody responds to train people to ignore alerts.
• Use systemd’s sandboxing directives. They cost nothing and limit what a compromised exporter can reach.

Frequently asked questions

Should I use Docker instead?

For a single server, systemd is simpler. Two static binaries and a package, with no container networking layer between you and the logs. Containers earn their keep once you are managing several hosts or already run everything else that way.

How much RAM does this need?

A 2 GB VPS is comfortable for one server. Prometheus memory tracks active time series rather than host count, so it grows when you add exporters, not when you add machines with the same exporters.

Can I monitor more servers later?

Yes. Install node_exporter on each and add them as targets. The moment metrics cross a network, though, that traffic needs protecting — a VPN, an SSH tunnel, or TLS with client certificates. Do not simply open 9100 to the world.

Where do alerts come from?

Two options. Grafana’s built-in alerting works from dashboard queries and is the quicker path. Alertmanager, run alongside Prometheus, is more powerful for routing, grouping and silencing. Start with Grafana’s and move if you outgrow it.

How long should I keep metrics?

Thirty days covers most troubleshooting and capacity questions. Longer retention on a single box is usually a sign you want a remote-write backend rather than a bigger disk.

Do I need node_exporter if I only care about my application?

Install it anyway. When the application misbehaves, the first question is whether the machine underneath it is healthy, and node_exporter is what answers that.

Wrapping up

Once you install Prometheus and Grafana this way, the whole stack is four systemd units and three config files. Everything binds to localhost, nginx handles TLS, and there is no layer of abstraction between you and a problem when something breaks at an inconvenient hour.

Get the dashboard working first, then spend an evening writing three or four alerts you will actually respond to. A dashboard tells you what happened when you go looking. An alert is what stops you needing to look.

Want this set up properly?

I build monitoring stacks for teams who would rather not spend a weekend on it — Prometheus and Grafana on bare servers or in Kubernetes, exporters for the things you actually run, alert rules tuned so they mean something, and dashboards that answer real questions rather than filling a screen.

If that sounds useful, you can hire me on Upwork.

The post Prometheus + Grafana From Scratch on One Server appeared first on John Nessime.

Nearly every TLS certificate error I have had to chase down comes back to one of three things: the chain your server sends is incomplete, the certificate does not actually cover the hostname being requested, or it expired and no alert fired. This article walks through all three — how to identify which one you are looking at, the exact commands to prove it, and how to set up renewal automation so the third category stops being a category at all.

Why TLS certificate errors are so confusing to debug

Two things conspire to make this harder than it should be.

The first is that browsers are forgiving in ways other clients are not. When a server sends an incomplete certificate chain, most modern browsers will quietly fetch the missing intermediate using the Authority Information Access extension embedded in the certificate. They fix your misconfiguration on the fly and show you a padlock. Meanwhile curl, Python’s requests, Java clients, Go services, and older Android devices do no such thing. They fail. This is why “it works in Chrome” is worth almost nothing as a diagnostic signal.

The second is that error messages describe the symptom rather than the cause. “Unable to get local issuer certificate” sounds like a client problem. It usually is not — it usually means your server is not sending the full chain. Learning to translate the message into the actual failure is most of the work.

One command that tells you almost everything

Before touching config files, look at what the server is actually sending on the wire:

openssl s_client -connect example.com:443 -servername example.com -showcerts </dev/null

The -servername flag matters more than people expect. It sets SNI, and without it you may be handed the server’s default certificate rather than the one for the domain you care about. Plenty of confusing “the certificate is for the wrong domain” reports come from testing without SNI.

Three parts of the output are worth reading carefully:

• The “Certificate chain” block near the top, listing each certificate the server sent with its subject and issuer.
• The “Verify return code” at the bottom. 0 (ok) is what you want. 20 and 21 both point at chain problems.
• The subject line of the first certificate, which tells you whether you even got the certificate you expected.

If you want the certificate details without the chain noise, pipe it into x509:

echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null 
  | openssl x509 -noout -subject -issuer -dates -ext subjectAltName

That single line answers “who issued it”, “when does it expire”, and “what hostnames does it cover” in one shot. It is the first thing I run.

Family one: certificate chain issues

A publicly trusted certificate is almost never signed directly by a root. There is at least one intermediate CA in between. The client already trusts the root — it ships in the operating system or browser trust store — but it has no idea about the intermediate unless you hand it over. Your server’s job is to send the leaf certificate plus every intermediate needed to walk up to a trusted root.

The classic mistake: serving the leaf without the intermediates

If you use Let’s Encrypt, certbot writes several files into /etc/letsencrypt/live/example.com/. Two of them look interchangeable and are not:

• cert.pem — the leaf certificate on its own.
• fullchain.pem — the leaf plus the intermediates.
• chain.pem — the intermediates on their own.
• privkey.pem — the private key.

Nginx wants fullchain.pem. Point it at cert.pem and browsers will still work, so the mistake survives testing and surfaces weeks later when an API client fails.

server {
    listen 443 ssl;
    server_name example.com www.example.com;

    ssl_certificate     /etc/letsencrypt/live/example.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;
}

On Apache 2.4.8 and later, SSLCertificateFile can hold the leaf and the intermediates concatenated together, and SSLCertificateChainFile is deprecated. If you inherited a config that still uses the old directive, it will work, but it is worth cleaning up:

SSLCertificateFile    /etc/letsencrypt/live/example.com/fullchain.pem
SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem

Wrong order, or a root that should not be there

Order is part of the specification, not a suggestion. The leaf comes first, then each issuer in sequence walking upward. Some clients tolerate a shuffled bundle; several do not, and the ones that do not tend to be the ones you find out about from an angry customer.

Including the root certificate at the end is a different case: it is harmless from a trust perspective, since the client ignores it and uses its own copy, but it adds bytes to every single handshake for no benefit. Leave it out.

Verifying a chain offline

You can check a bundle before deploying it, which is far better than discovering the problem in production:

# Does the leaf verify against the intermediates you plan to serve?
openssl verify -untrusted chain.pem cert.pem

# Print the subject and issuer of every certificate in a bundle, in order
openssl crl2pkcs7 -nocrl -certfile fullchain.pem 
  | openssl pkcs7 -print_certs -noout

That second command is the one I reach for most. Read the output top to bottom: the issuer of each certificate should be the subject of the next one down. The moment that pattern breaks, you have found your gap.

The key and the certificate do not match

Less common, but it produces a spectacularly unhelpful error. After a renewal where files were copied around by hand, confirm the pair still belongs together — the two hashes must be identical:

openssl x509 -noout -modulus -in cert.pem | openssl md5
openssl rsa  -noout -modulus -in privkey.pem | openssl md5

Family two: SAN mismatches

The Common Name field is dead. It has been deprecated for hostname matching for years, and Chrome stopped honouring it entirely back in 2017. What matters now is the Subject Alternative Name extension. If the hostname a client requested is not in the SAN list, the connection fails, regardless of what the CN says.

Check what a certificate actually covers:

# From a file
openssl x509 -noout -ext subjectAltName -in cert.pem

# From a live server
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null 
  | openssl x509 -noout -ext subjectAltName

The four SAN mistakes that keep recurring

• Apex without www, or www without apex. These are two distinct names. A certificate for example.com does not cover www.example.com. Both need to be in the SAN list, and both need to be passed to certbot with separate -d flags.
• Assuming a wildcard covers everything. It does not. *.example.com matches exactly one label. It covers api.example.com. It does not cover example.com itself, and it does not cover staging.api.example.com. Multi-level subdomains need their own wildcard or their own entry.
• A new subdomain that never made it into the renewal. You add metrics.example.com to nginx, point DNS at it, and forget that the certificate was issued for a fixed list of names. Adding a name means reissuing, not just reloading.
• Connecting by IP address. Almost no public certificate includes an IP SAN, so hitting the server by IP will always fail hostname verification. That is expected behaviour, not a broken certificate.

One more that catches people out: if you use Cloudflare in proxied mode, the certificate a visitor sees is Cloudflare’s edge certificate, not yours. Debugging the origin means bypassing the proxy and testing the origin IP directly with the Host header set correctly, or temporarily turning the orange cloud grey.

Family three: expiry, and why automation is no longer optional

Expiry used to be a calendar problem. You bought a certificate for a year, set a reminder, and dealt with it once. That era is closing.

In April 2025 the CA/Browser Forum passed ballot SC-081v3, which phases the maximum validity period for publicly trusted TLS certificates down from 398 days to 47 days. The schedule runs in three steps: 200 days from 15 March 2026, 100 days from 15 March 2027, and 47 days from 15 March 2029. The first phase is already in force. Domain validation reuse windows shrink alongside it.

The 47-day target was chosen deliberately: short enough that manual renewal stops being viable at any meaningful scale. Automation is the point of the change, not a side effect of it.

If you run a handful of domains, this is a nuisance. If you run fifty, manual renewal is already finished as a strategy.

Getting ACME renewal right

Certbot installs a systemd timer or cron entry that attempts renewal twice a day and does nothing unless the certificate is close to expiry. Confirm it is actually there:

systemctl list-timers | grep certbot
certbot renew --dry-run

The dry run exercises the whole path against the staging environment without touching rate limits. Run it after any change to your web server config, not just after installing certbot. A renewal that worked last quarter can break because someone added a redirect that intercepts the ACME challenge path.

The trap: renewal succeeds, the site still serves the old certificate

This is the failure I see most often, and it is genuinely maddening because every log says success. Certbot writes the new certificate to disk. Nginx has the old one loaded in memory and keeps serving it until it is told otherwise. The certificate on disk is valid, the certificate on the wire is expired, and everything looks fine from the server’s point of view.

The fix is a deploy hook, which runs only when a certificate was actually renewed:

certbot renew --deploy-hook "systemctl reload nginx"

Better still, make it permanent by adding it to the renewal configuration at /etc/letsencrypt/renewal/example.com.conf, under the [renewalparams] section, so it survives however the renewal happens to be triggered:

renew_hook = systemctl reload nginx

Use reload rather than restart. A reload picks up the new certificate without dropping connections.

Remember that anything else terminating TLS needs the same treatment. A mail server, a database proxy, a Docker container with the certificate baked into its image — each one needs its own reload step, and containers in particular will happily run for months with an expired certificate mounted from the host.

Wildcards need DNS-01

Wildcard certificates cannot be issued through HTTP-01 validation. They require DNS-01, which means certbot needs to create a TXT record on your domain, which means API credentials for your DNS provider. Use a provider plugin rather than the manual mode — manual DNS-01 cannot be automated, and an unautomated wildcard on a 47-day cycle is an outage with a scheduled start time.

If issuance fails outright with a policy error, check your CAA records before assuming the ACME client is broken. A CAA record that names a different CA will block issuance entirely:

dig CAA example.com +short

Monitor from outside, not from the server

Checking the certificate file on disk tells you what should be served. Only an external check tells you what is being served. Given the reload trap above, that difference is the entire point.

A minimal check that works anywhere:

#!/usr/bin/env bash
# Usage: ./cert-days.sh example.com
# Exits non-zero if fewer than 21 days remain.
set -euo pipefail

DOMAIN="$1"
THRESHOLD=21

END=$(echo | openssl s_client -connect "${DOMAIN}:443" -servername "${DOMAIN}" 2>/dev/null 
      | openssl x509 -noout -enddate | cut -d= -f2)

DAYS=$(( ( $(date -d "${END}" +%s) - $(date +%s) ) / 86400 ))
echo "${DOMAIN}: ${DAYS} days remaining (expires ${END})"

[ "${DAYS}" -lt "${THRESHOLD}" ] && exit 1
exit 0

If you already run Prometheus, the blackbox exporter gives you this for free through the probe_ssl_earliest_cert_expiry metric. An alert rule looks like this:

groups:
  - name: tls
    rules:
      - alert: TLSCertificateExpiringSoon
        expr: (probe_ssl_earliest_cert_expiry - time()) / 86400 < 21
        for: 10m
        labels:
          severity: warning
        annotations:
          summary: "TLS certificate expiring in under 21 days"

Set the threshold well above your renewal window. With 200-day certificates renewing at 30 days remaining, a 21-day alert gives you nine days of warning that something in the pipeline broke. Once 47-day certificates arrive, those numbers compress hard, and an alert that fires too late is the same as no alert.

A workflow for diagnosing TLS certificate errors

When something breaks, work through it in this order. It takes about a minute and rules out whole families of causes as you go.

1. Reproduce outside the browser. Use curl -v or openssl s_client. If the browser works and curl does not, you are almost certainly looking at a chain problem.
2. Read the verify return code. Code 20 or 21 means a chain issue. A hostname mismatch message means a SAN issue. A date error means expiry — or a wrong system clock on the client.
3. Print the dates. Rules expiry in or out in one command, and tells you whether the certificate on the wire matches the one on disk.
4. Print the SANs. Confirm the exact hostname being requested appears in the list. Watch for apex versus www.
5. Walk the chain. Each certificate’s issuer should be the next one’s subject. Find where the sequence breaks.
6. Compare disk against wire. If the file is newer than what is being served, you have a reload problem, not a certificate problem.
7. Check the client’s trust store. If everything above is clean and one specific client still fails, the problem has moved to their end — an old Android device, a Java service with its own cacerts, or a container image built on a base with a stale CA bundle.

Common mistakes worth avoiding

• Testing only in a browser, and mistaking its silent chain repair for a working configuration.
• Pointing the web server at cert.pem instead of fullchain.pem.
• Renewing without reloading the service that holds the certificate in memory.
• Running certbot renew on a schedule but never running --dry-run after config changes.
• Copying key and certificate files between servers by hand and losing track of which pair belongs together.
• Monitoring the file on disk rather than the certificate actually being served.
• Forgetting that mail servers, load balancers, and containers terminate TLS too, and need their own renewal path.
• Adding a subdomain to the web server config and assuming the existing certificate covers it.

Best practices that hold up over time

• Automate issuance and renewal from day one, even for a single domain. The habit matters more than the scale.
• Always use a deploy hook, and always reload rather than restart.
• Alert on days remaining, measured externally, with a threshold at least twice your renewal window.
• Keep an inventory of every hostname you hold a certificate for. You cannot monitor what you have forgotten about.
• Prefer DNS-01 with a provider plugin if you need wildcards, and never rely on manual DNS validation.
• Set CAA records deliberately so you know which CAs are permitted to issue for your domain.
• Test with a non-browser client before calling a certificate deployment finished.

Frequently asked questions

Why does my site work in Chrome but fail in curl?

Almost always an incomplete chain. Browsers fetch missing intermediate certificates automatically using the Authority Information Access extension; curl does not. Point your web server at the full chain file rather than the leaf certificate and both will work.

What does “unable to get local issuer certificate” actually mean?

The client could not build a path from your certificate to a root it trusts. Nine times out of ten the server is not sending its intermediates. Occasionally the client’s own CA bundle is out of date, which is common in older container base images.

Does a wildcard certificate cover the root domain?

No. *.example.com matches a single label, so it covers www.example.com but not example.com and not a.b.example.com. Include the apex explicitly as an additional SAN entry when you request the certificate.

My certificate renewed but the site still shows it as expired. Why?

The service holding the certificate has not reloaded. The new file is on disk; the old one is still in memory. Add a deploy hook that reloads the service on renewal, then verify from outside with openssl s_client rather than trusting the file timestamp.

How short will certificate lifetimes actually get?

Under the CA/Browser Forum schedule, 200 days applies from March 2026, 100 days from March 2027, and 47 days from March 2029. Private and internal CAs are not bound by these rules, but public certificates are.

Is the Common Name field still used at all?

Not for hostname validation. Modern clients read only the Subject Alternative Name extension. A certificate whose CN matches but whose SAN list does not include the hostname will be rejected.

Wrapping up

TLS certificate errors feel opaque mostly because the error text points at the wrong layer. Once you sort them into chain, SAN, or expiry, each one has a short diagnostic path and an obvious fix. Get comfortable with openssl s_client and openssl x509 and you will resolve most incidents faster than it takes to find the right dashboard.

The expiry category is the one worth engineering away permanently. With maximum lifetimes dropping toward 47 days, any process that depends on a person remembering something will fail eventually. Automate the renewal, add the deploy hook, and alert on what is actually being served rather than what is sitting on disk. Do that once and TLS certificate errors go back to being a rare curiosity instead of a recurring outage.

Need help with TLS, certificates, or infrastructure automation?

I work with teams on exactly this kind of problem — broken certificate chains, ACME automation across multiple servers, expiry monitoring wired into Prometheus and Grafana, and the wider DevOps plumbing that keeps it all running quietly in the background.

If you would rather hand this off than debug it at two in the morning, you can hire me on Upwork.

The post TLS Certificate Errors: Chain Issues, SANs, and Expiry Automation appeared first on John Nessime.

The good news is that most WordPress problems are not random. They leave clues, and if you know where to look, identifying the root cause becomes much easier.

After managing WordPress servers and troubleshooting websites over the years, one lesson has become obvious: guessing rarely fixes anything. A structured debugging process almost always does.

This guide walks through the practical approach professionals use to diagnose WordPress issues, improve performance, and strengthen website security without creating additional problems.

Why WordPress Problems Happen

WordPress itself is remarkably stable. Most issues originate from the environment around it.

• Plugin conflicts
• Theme compatibility issues
• PHP version mismatches
• Database corruption
• Memory limitations
• File permission problems
• Server configuration mistakes
• Caching conflicts
• Security breaches
• Poor hosting performance

Instead of treating every symptom individually, it’s more effective to understand which layer is actually responsible.

Start With Safe Troubleshooting

Before changing anything, make a complete backup.

A proper backup should include:

• Entire WordPress files
• Database
• Uploads folder
• Configuration files
• Custom themes
• Custom plugins

Working without a backup is one of the biggest mistakes beginners make.

Enable WordPress Debug Mode

WordPress includes a powerful debugging system built directly into the core.

Edit the wp-config.php file and enable debugging.

define('WP_DEBUG', true);
define('WP_DEBUG_LOG', true);
define('WP_DEBUG_DISPLAY', false);

This records PHP warnings and errors inside the debug.log file without exposing them to website visitors.

Never leave debugging enabled permanently on a production website.

Common WordPress Errors Explained

White Screen of Death

The website displays nothing except a blank white page.

Typical causes include:

• PHP fatal errors
• Plugin conflicts
• Memory exhaustion
• Corrupted files

The debug log usually reveals the exact file responsible.

500 Internal Server Error

This generic server error often results from:

• Broken .htaccess rules
• PHP configuration
• File permission issues
• Server resource exhaustion

Checking the Apache or Nginx error logs is usually the fastest path to identifying the real problem.

Database Connection Error

This message means WordPress cannot communicate with MySQL.

Possible reasons include:

• Incorrect database credentials
• Database server unavailable
• Corrupted database tables
• Exceeded hosting limits

Finding Plugin Conflicts

Plugins are responsible for the majority of WordPress issues.

Instead of uninstalling everything randomly:

1. Disable all plugins.
2. Confirm whether the issue disappears.
3. Enable plugins one at a time.
4. Identify the plugin causing the conflict.

This systematic approach saves hours of unnecessary troubleshooting.

Theme Debugging

The active theme can introduce problems through:

• Outdated code
• Poor optimization
• Deprecated PHP functions
• JavaScript conflicts

Temporarily switching to a default WordPress theme like Twenty Twenty-Six quickly determines whether the issue originates from the theme.

Browser Developer Tools

Modern browsers include excellent diagnostic tools.

The Developer Console helps identify:

• JavaScript errors
• Failed network requests
• Slow resources
• CSS conflicts
• Mixed HTTPS content

Many frontend issues become immediately obvious after opening the browser console.

Improving Website Performance

Performance problems are often easier to detect than to solve.

Start by measuring instead of guessing.

Check Page Speed

Review:

• Largest Contentful Paint (LCP)
• Interaction to Next Paint (INP)
• Cumulative Layout Shift (CLS)
• Time to First Byte (TTFB)

These Core Web Vitals provide a realistic picture of user experience.

Optimize Images

• Convert images to WebP
• Resize oversized files
• Compress before uploading
• Lazy load below-the-fold images

Reduce Plugin Bloat

Installing dozens of plugins increases:

• Database queries
• PHP execution time
• JavaScript loading
• Memory consumption

Only keep plugins that provide measurable value.

Use Object Caching

Redis and Memcached dramatically reduce repeated database queries on busy websites.

Database Maintenance

Over time databases accumulate:

• Expired transients
• Spam comments
• Post revisions
• Unused metadata
• Orphaned records

Cleaning these periodically keeps queries efficient.

Server-Level Debugging

Not every issue originates inside WordPress.

Sometimes the operating system is the real culprit.

Monitor:

• CPU usage
• RAM usage
• Disk utilization
• Disk I/O
• PHP-FPM workers
• MySQL performance
• Apache workers
• Nginx requests

Server monitoring often exposes bottlenecks invisible from inside WordPress.

Security Troubleshooting

Performance degradation can sometimes indicate a compromised website.

Watch for:

• Unexpected administrator accounts
• Unknown plugins
• Modified core files
• Suspicious PHP files
• Spam redirects
• Hidden malware

If malware is suspected:

1. Take the website offline if necessary.
2. Create a forensic backup.
3. Scan every file.
4. Replace WordPress core files.
5. Rotate all passwords.
6. Regenerate security keys.
7. Review access logs.

Useful Log Files

Professional troubleshooting relies heavily on logs.

• WordPress debug.log
• Apache error.log
• Nginx error.log
• PHP-FPM logs
• MySQL logs
• Server syslog
• Security audit logs

Learning to read logs is one of the most valuable troubleshooting skills a WordPress administrator can develop.

A Practical Debugging Workflow

1. Reproduce the problem.
2. Check browser console.
3. Enable debug logging.
4. Review server logs.
5. Disable plugins.
6. Switch themes.
7. Verify PHP compatibility.
8. Check database health.
9. Test server resources.
10. Validate the fix.
11. Monitor after deployment.

Following the same workflow every time eliminates unnecessary trial and error.

Preventing Future Problems

The easiest issue to solve is the one that never happens.

Good maintenance habits include:

• Automatic backups
• Routine updates
• Security monitoring
• Uptime monitoring
• Performance dashboards
• Database optimization
• Log reviews
• Staging before production deployments

Preventive maintenance significantly reduces downtime and emergency fixes.

Final Thoughts

Debugging WordPress is less about memorizing error messages and more about following a logical process. Every warning, failed request, or slow query tells part of the story. By collecting evidence instead of making assumptions, you can solve problems faster and with greater confidence.

Whether you’re maintaining a personal blog, managing business websites, or supporting client projects, investing time in understanding debugging techniques pays dividends in stability, performance, and security. A well-maintained WordPress site not only loads faster and ranks better in search engines but also provides visitors with a more reliable and trustworthy experience.

Reliable websites are not built by avoiding problems—they’re built by knowing exactly how to diagnose and solve them when they appear.

The post The Complete WordPress Debugging Handbook: Fix Errors, Improve Performance & Secure Your Website Like a Pro appeared first on John Nessime.