<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Cron | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/cron/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/cron/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Mon, 14 Sep 2026 13:13:11 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.3</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>Cron | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/cron/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Cron vs systemd Timers: The Failure Modes That Decide It</title>
		<link>https://john-nessime.com/blog/devops/cron-vs-systemd-timers/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Fri, 28 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[System Administration]]></category>
		<category><![CDATA[anacron]]></category>
		<category><![CDATA[Automation]]></category>
		<category><![CDATA[Cron]]></category>
		<category><![CDATA[Crontab]]></category>
		<category><![CDATA[flock]]></category>
		<category><![CDATA[Job Scheduling]]></category>
		<category><![CDATA[journalctl]]></category>
		<category><![CDATA[Monitoring]]></category>
		<category><![CDATA[Reliability Engineering]]></category>
		<category><![CDATA[Shell Scripting]]></category>
		<category><![CDATA[Sysadmin]]></category>
		<category><![CDATA[Systemd]]></category>
		<category><![CDATA[systemd Timers]]></category>
		<category><![CDATA[Troubleshooting]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=254</guid>

					<description><![CDATA[<p>Both cron and systemd timers can run a command at 2 a.m. The difference is what happens the night it doesn't run, and how long it takes anyone to notice. A comparison by failure family, with a decision rule, the same job written both ways, and how to debug each one when it goes quiet.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/cron-vs-systemd-timers/">Cron vs systemd Timers: The Failure Modes That Decide It</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The ticket usually arrives from the wrong direction. Nobody reports &#8220;the scheduler is broken.&#8221; Somebody asks for a restore, and the newest dump in the backup directory is three weeks old.</p>



<p class="wp-block-paragraph">The crontab entry is still there. The script still works when you run it by hand. Nothing in the monitoring went red, because nothing was watching whether the job ran at all. It was watching whether the server was up.</p>



<p class="wp-block-paragraph">That is the real question behind cron vs systemd timers, and it has almost nothing to do with syntax. Both tools can run a command at 2 a.m. The difference is what happens on the night the command doesn&#8217;t run, and how long it takes anyone to find out.</p>



<p class="wp-block-paragraph">This post compares the two by failure family rather than by feature list: missed runs, overlapping runs, environment surprises, output that goes nowhere, and jobs that eat the machine. Then a decision procedure, the same job written both ways, and how to debug each one when it goes quiet.</p>



<h2 class="wp-block-heading">The failure that actually decides it</h2>



<p class="wp-block-paragraph">Scheduled jobs fail silently far more often than they fail loudly. A job that crashes with a stack trace gets fixed the same week. A job that stopped firing six weeks ago gets found during an incident.</p>



<p class="wp-block-paragraph">Cron&#8217;s default answer to &#8220;what happened?&#8221; is to mail the job&#8217;s output to the owning user. On a server with no local mail transfer agent configured, which is most servers built in the last decade, that output is generated and then discarded. The daemon logs that it <em>started</em> the command. It does not log what the command printed, and in most implementations it does not log the exit status either.</p>



<p class="wp-block-paragraph">So you get a log line saying cron ran your backup, every night, forever, including on the nights the backup wrote nothing. That line is the trap. It looks like evidence.</p>



<p class="wp-block-paragraph">systemd timers hand this to journald instead. Standard output and standard error from the triggered service land in the journal, tagged with the unit name, alongside the exit status and the runtime. That is the single biggest practical difference between the two, and it is the reason most of my new scheduled work goes into timers.</p>



<p class="wp-block-paragraph">Neither one alerts you. Both will happily not run for a month while you sleep well. Observability is not monitoring, and I&#8217;ll come back to that.</p>



<h2 class="wp-block-heading">Where cron still wins</h2>



<p class="wp-block-paragraph">Cron is one line in one file. There is no unit to reload, no second file to keep in sync, no install section to get wrong. <code>crontab -e</code>, five fields, a command, done. For a job that trims a log directory on a box you own, that economy is not nothing.</p>



<p class="wp-block-paragraph">It also runs where systemd does not. BSD hosts, Alpine images, containers, appliance firmware, embedded boxes, older enterprise systems still on a legacy init. If your configuration management has to cover a fleet that is not uniformly systemd, cron is the common denominator and it is not close.</p>



<p class="wp-block-paragraph">Per-user scheduling is genuinely simpler too. A user runs <code>crontab -e</code> and owns their own jobs, with no root involvement and no unit files in <code>/etc</code>. On shared hosting and on control panels like DirectAdmin or cPanel, this is the only model available, and the panel writes the crontab for you.</p>



<h3 class="wp-block-heading">Where cron loses</h3>



<ul class="wp-block-list">

<li>No dependency ordering. Cron cannot wait for the network to be up or the database to be ready. It fires at the wall clock time and hopes.</li>


<li>No overlap protection. If the 2 a.m. run is still going at 3 a.m., the 3 a.m. run starts anyway, on top of it.</li>


<li>No resource containment. A runaway job competes with production for CPU, memory and disk with nothing standing between them.</li>


<li>Minute resolution, and no catch-up if the machine was off.</li>


<li>A syntax with sharp edges that fail quietly, which is the next section.</li>

</ul>



<h2 class="wp-block-heading">Where systemd timers win</h2>



<p class="wp-block-paragraph">A timer separates <em>when</em> from <em>what</em>. The <code>.timer</code> unit holds the schedule. The <code>.service</code> unit holds the job. That split feels like bureaucracy until the first time you need to run the job right now, out of band, without touching the schedule. Then it is just <code>systemctl start myjob.service</code>, and the run is logged exactly like a scheduled one.</p>



<p class="wp-block-paragraph">Everything systemd knows how to do to a service, it will do to a scheduled job. Ordering after other units. Restart-on-failure. Runtime caps. Memory and CPU limits through cgroups. A dedicated user. A private <code>/tmp</code>. A failure handler unit that fires when the job exits non-zero.</p>



<p class="wp-block-paragraph">You also get a schedule you can interrogate before it bites you. <code>systemctl list-timers</code> prints every timer on the box with its next and last elapse. <code>systemd-analyze calendar</code> takes an expression and tells you exactly when it would fire, which is the closest thing either scheduler has to a unit test.</p>



<h3 class="wp-block-heading">Where systemd timers lose</h3>



<ul class="wp-block-list">

<li>Two files, a <code>daemon-reload</code> and an <code>enable</code> for every job. That is real friction when you have forty small jobs.</li>


<li>The calendar syntax is more expressive than cron&#8217;s and harder to eyeball. You will reach for <code>systemd-analyze calendar</code> more often than you expect.</li>


<li>Portability is gone. These units run on systemd hosts and nowhere else.</li>


<li>User timers need lingering enabled, or they stop when the user logs out and never start at boot. This surprises people once, memorably.</li>


<li>The journal is not a log file. If your log shipping only tails <code>/var/log</code>, timer output is invisible to it until you point the shipper at journald.</li>

</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Cron vs systemd timers, by failure family</h2>



<h3 class="wp-block-heading">The machine was off at 2 a.m.</h3>



<p class="wp-block-paragraph">Cron has no memory. If the box was down, rebooting, or suspended when the schedule came round, that run is gone. Nothing catches it up and nothing records that it was skipped.</p>



<p class="wp-block-paragraph">The traditional fix is anacron, which tracks the last successful run per job in a timestamp file and runs the job shortly after boot if the interval has lapsed. It is why the scripts in <code>/etc/cron.daily</code> still run on a laptop that is closed every night. The catch is that anacron works in whole days and only covers the <code>cron.daily</code>, <code>cron.weekly</code> and <code>cron.monthly</code> directories by default. It does not help your custom job at 02:00.</p>



<p class="wp-block-paragraph">systemd folds that behaviour into a single directive:</p>



<pre class="wp-block-code"><code>[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true</code></pre>



<p class="wp-block-paragraph">With <code>Persistent=true</code>, systemd writes a timestamp under <code>/var/lib/systemd/timers/</code> each time the timer fires. On boot it compares that stamp against the schedule, and if a run was missed it triggers immediately. This matters more than it sounds on any VPS you reboot for kernel updates, and a great deal on hosts at providers like Contabo or InterServer where you take maintenance windows on their timetable rather than yours.</p>



<p class="wp-block-paragraph">One caveat worth knowing: &#8220;immediately&#8221; means immediately. If you have twelve persistent timers and the box has been off for a week, all twelve fire within seconds of boot, on a machine that is still starting services. Pair <code>Persistent=true</code> with <code>RandomizedDelaySec=</code> so the catch-up is spread out.</p>



<h3 class="wp-block-heading">Two copies of the same job</h3>



<p class="wp-block-paragraph">This is the one that corrupts data. A sync job normally takes four minutes and runs every five. One night the remote end is slow, the run takes eleven minutes, and cron starts two more copies while the first is still writing.</p>



<p class="wp-block-paragraph">Cron will not stop this. You have to do it yourself, and the standard tool is <code>flock</code> from util-linux:</p>



<pre class="wp-block-code"><code>*/5 * * * * /usr/bin/flock -n /var/lock/sync.lock /usr/local/bin/sync.sh</code></pre>



<p class="wp-block-paragraph"><code>-n</code> means fail immediately rather than queue behind the running copy. Without it you build a backlog of processes all waiting for the same lock, which turns an overlap problem into a fork bomb with a slow fuse.</p>



<p class="wp-block-paragraph">systemd gets this for free from its own model. A unit is either active or it isn&#8217;t. If <code>sync.service</code> is still running when the timer elapses, there is no second copy to start, because starting an already-active unit is a no-op. You do not have to remember anything.</p>



<p class="wp-block-paragraph">One default is worth knowing. A calendar timer schedules its next elapse from the previous trigger time, so if a run overran, that next elapse is already in the past and the service fires again the moment it finishes. Newer systemd releases add a <code>DeferReactivation=</code> boolean that schedules from when the service went inactive instead, so an overrunning job waits for the next real slot. Check your distribution&#8217;s systemd version before relying on it.</p>



<h3 class="wp-block-heading">The environment your job actually gets</h3>



<p class="wp-block-paragraph">&#8220;It works when I run it manually&#8221; is the single most common scheduled-job bug, and both schedulers cause it, for the same reason: neither one gives your job an interactive login shell.</p>



<p class="wp-block-paragraph">Cron runs the command with a short, hardcoded environment. Your shell profile is not sourced. Your <code>PATH</code> is much shorter than the one you tested with, which is why a script that calls <code>docker</code>, <code>wp</code>, <code>aws</code> or a version-manager shim works from your terminal and fails from cron. The working directory is the user&#8217;s home, not wherever you happened to be standing.</p>



<p class="wp-block-paragraph">Cron adds one hazard that is entirely its own. Inside a crontab, the percent sign is a metacharacter: it becomes a newline, and everything after the first one is fed to the command as standard input. So this looks fine and is broken:</p>



<pre class="wp-block-code"><code># Truncated at the first %. The redirect never happens.
0 2 * * * /usr/bin/mysqldump app &gt; /backups/app-$(date +%F).sql

# Escaped. This is what you meant.
0 2 * * * /usr/bin/mysqldump app &gt; /backups/app-$(date +%F).sql</code></pre>



<p class="wp-block-paragraph">There is no warning. The line parses, cron runs the truncated fragment, and the log says the job started. Anything with a <code>date +</code> format string in a crontab deserves a second look.</p>



<p class="wp-block-paragraph">systemd services get a minimal environment too, so the class of bug is identical. What differs is that the fix is declarative and lives with the unit rather than being hidden inside a wrapper script:</p>



<pre class="wp-block-code"><code>[Service]
Type=oneshot
User=deploy
WorkingDirectory=/srv/app
Environment=PATH=/usr/local/bin:/usr/bin:/bin
EnvironmentFile=/etc/myapp/backup.env
ExecStart=/usr/local/bin/backup.sh</code></pre>



<p class="wp-block-paragraph">The honest summary: absolute paths for every binary solve most of this on either scheduler. The rest is solved by never putting logic in the schedule. Put it in a script and schedule the script.</p>



<h3 class="wp-block-heading">Everything firing at once</h3>



<p class="wp-block-paragraph">Write <code>0 0 * * *</code> on fifty hosts and fifty hosts hit your backup target, your package mirror or your API at exactly midnight. The classic cron answer is to hash the hostname into a minute offset in configuration management, which works and which every team reinvents.</p>



<p class="wp-block-paragraph">systemd has two directives here and they pull in opposite directions, which is the part people get wrong:</p>



<ul class="wp-block-list">

<li><code>RandomizedDelaySec=</code> spreads timers <em>apart</em>. It adds a delay between zero and the value you set, so identical schedules on many hosts land at different moments.</li>


<li><code>AccuracySec=</code> pulls timers <em>together</em>. It lets systemd shift the firing time within a window to coalesce wakeups and save power. It defaults to one minute, which is why a timer set to 02:00:00 often fires a few seconds late.</li>

</ul>



<p class="wp-block-paragraph">If you want a timer to fire close to the stated second, lower <code>AccuracySec=</code>. If you want to stagger a fleet, raise <code>RandomizedDelaySec=</code>. Setting only <code>AccuracySec=</code> to a large value and expecting it to spread load is the mistake, because coalescing is the opposite of spreading.</p>



<h3 class="wp-block-heading">A job that eats the box</h3>



<p class="wp-block-paragraph">Cron has no answer here beyond <code>nice</code> and <code>ionice</code> in front of the command, and a script that hangs forever will hang forever. A systemd service is a cgroup, so the limits are declarative and enforced:</p>



<pre class="wp-block-code"><code>[Service]
Type=oneshot
MemoryMax=512M
CPUQuota=40%
IOWeight=20
TimeoutStartSec=30min
OnFailure=notify-failure@%n.service
ExecStart=/usr/local/bin/reindex.sh</code></pre>



<p class="wp-block-paragraph">The two directives that earn their keep on a shared box are <code>TimeoutStartSec=</code>, which kills a <code>oneshot</code> job that has hung, and <code>OnFailure=</code>, which starts another unit when this one exits non-zero. That second one is how you get a scheduled job to page you without wiring an alerting call into every script.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The same job written both ways</h2>



<p class="wp-block-paragraph">A nightly database dump that must not overlap, must not run before the database is up, and must catch up if the host was rebooted. In cron:</p>



<pre class="wp-block-code"><code>0 2 * * * /usr/bin/flock -n /var/lock/db-backup.lock /usr/local/bin/db-backup.sh &gt;&gt; /var/log/db-backup.log 2&gt;&amp;1</code></pre>



<p class="wp-block-paragraph">That handles overlap and captures output. It does not handle &#8220;the database wasn&#8217;t ready&#8221; or &#8220;the host was down&#8221;, and the log file is now your problem to rotate.</p>



<p class="wp-block-paragraph">The systemd version is two files. First the service, which is the job:</p>



<pre class="wp-block-code"><code># /etc/systemd/system/db-backup.service
[Unit]
Description=Nightly application database dump
After=network-online.target mariadb.service
Wants=network-online.target

[Service]
Type=oneshot
User=backup
ExecStart=/usr/local/bin/db-backup.sh
TimeoutStartSec=45min</code></pre>



<p class="wp-block-paragraph">Then the timer, which is only the schedule:</p>



<pre class="wp-block-code"><code># /etc/systemd/system/db-backup.timer
[Unit]
Description=Run the nightly database dump

[Timer]
OnCalendar=*-*-* 02:00:00
Persistent=true
RandomizedDelaySec=15min

[Install]
WantedBy=timers.target</code></pre>



<p class="wp-block-paragraph">Because the two units share a base name, you do not need a <code>Unit=</code> line. systemd pairs <code>db-backup.timer</code> with <code>db-backup.service</code> by convention. Add <code>Unit=</code> only when the names differ.</p>



<p class="wp-block-paragraph">Then the part people skip, which is verifying it before walking away:</p>



<pre class="wp-block-code"><code># Check the units parse before you enable anything
systemd-analyze verify /etc/systemd/system/db-backup.*

# Confirm the calendar expression means what you think
systemd-analyze calendar --iterations=5 "*-*-* 02:00:00"

systemctl daemon-reload
systemctl enable --now db-backup.timer

# Next and last elapse for this timer
systemctl list-timers db-backup.timer

# Run it now, out of band, without touching the schedule
systemctl start db-backup.service
journalctl -u db-backup.service -n 50 --no-pager</code></pre>



<p class="wp-block-paragraph">Note that you enable the <em>timer</em>, not the service. Enabling the service instead would try to run the job at every boot, which is a memorable way to learn the difference.</p>



<h2 class="wp-block-heading">How I&#8217;d decide</h2>



<p class="wp-block-paragraph">Work down this list and stop at the first line that applies. It resolves most cases in under a minute.</p>



<ol class="wp-block-list">

<li>The host might not be systemd, now or later. Use cron. Portability beats every other consideration on this list.</li>


<li>The job is inside a container. Use neither. Use your orchestrator&#8217;s scheduler, or a timer on the host that invokes the container.</li>


<li>A missed run matters, or the job must not overlap, or it needs to wait for another service. Use a timer. Every one of those is a directive rather than a wrapper script.</li>


<li>The job is heavy: a dump, a reindex, an import, anything that can pin a core or fill memory. Use a timer, for the cgroup limits and the runtime cap.</li>


<li>You need to see output after the fact without building a logging path first. Use a timer. The journal is already there.</li>


<li>You need sub-minute scheduling. Use a timer with a monotonic directive. Cron&#8217;s floor is one minute.</li>


<li>Otherwise, cron is fine, and one line beats two files.</li>

</ol>



<p class="wp-block-paragraph">The migration question answers itself from that. Do not convert forty working crontab lines in a weekend. Write new scheduled work as timers, convert the jobs that have actually burned you, and leave the rest. Mixing both on one host is normal and supported. Just never schedule the same job in both places, which is easier to do than it sounds when a configuration management run and a manual edit disagree.</p>



<h2 class="wp-block-heading">Diagnosing a job that never fired</h2>



<h3 class="wp-block-heading">Cron</h3>



<p class="wp-block-paragraph">Establish what actually happened before you theorise. The daemon logs every job it starts, so the log tells you which of three situations you are in: cron never tried, cron tried and the command failed, or cron never saw the entry at all.</p>



<pre class="wp-block-code"><code># Is the daemon running at all?
systemctl status crond      # RHEL, AlmaLinux, Rocky
systemctl status cron       # Debian, Ubuntu

# What did it start, and when?
journalctl -u crond --since "24 hours ago"
grep CRON /var/log/syslog | tail -20     # Debian family
tail -50 /var/log/cron                   # RHEL family

# Whose crontab are you actually looking at?
crontab -l -u deploy
ls -l /etc/cron.d/</code></pre>



<p class="wp-block-paragraph">Then work through the usual suspects, roughly in order of how often they turn out to be the cause:</p>



<ul class="wp-block-list">

<li><strong>Wrong crontab.</strong> The job is in root&#8217;s crontab and you are reading the deploy user&#8217;s, or it&#8217;s in <code>/etc/cron.d</code> where there is an extra user field before the command.</li>


<li><strong>PATH, or an unescaped percent sign.</strong> Dump what the job really sees with a temporary entry like <code>* * * * * env &gt; /tmp/cron-env.txt 2&gt;&amp;1</code>, then compare it against your shell.</li>


<li><strong>Permissions.</strong> The script is not executable, or the crontab owner cannot read it.</li>


<li><strong>The schedule is not what you think.</strong> When both day-of-month and day-of-week are restricted, cron runs the job when <em>either</em> matches, not both. <code>0 0 15 * 1</code> fires on the 15th and on every Monday.</li>


<li><strong>The command captured no output</strong>, so a real error was mailed into the void. Append <code>&gt;&gt; /var/log/myjob.log 2&gt;&amp;1</code> and wait for the next run.</li>

</ul>



<h3 class="wp-block-heading">systemd</h3>



<pre class="wp-block-code"><code># Is the timer loaded and scheduled? --all shows inactive ones too
systemctl list-timers --all

# Timer state, last trigger, next elapse
systemctl status db-backup.timer

# What the job printed, and how it exited
journalctl -u db-backup.service --since "7 days ago"

# Prove the schedule independently of the unit
systemd-analyze calendar --iterations=3 "Mon..Fri *-*-* 06:30:00"</code></pre>



<ul class="wp-block-list">

<li><strong>The timer is not enabled.</strong> Creating the files does nothing. It needs <code>daemon-reload</code> and <code>enable --now</code>.</li>


<li><strong>You enabled the service instead of the timer.</strong> Check with <code>systemctl is-enabled</code> on both.</li>


<li><strong>A user timer with no lingering.</strong> Under <code>systemctl --user</code>, the manager stops when the session ends unless you run <code>loginctl enable-linger</code> for that user.</li>


<li><strong>The calendar expression is valid but wrong.</strong> Valid syntax that never matches will not error. <code>systemd-analyze calendar</code> is how you find out.</li>


<li><strong>The service failed before your script ran.</strong> A missing <code>User=</code>, an <code>ExecStart</code> path that does not exist, a <code>WorkingDirectory</code> that is not there. The journal names the reason.</li>

</ul>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<p class="wp-block-paragraph"><strong>&#8220;Cron is deprecated.&#8221;</strong> It isn&#8217;t. It is packaged and supported on every mainstream distribution, and control panels, hosting providers and countless application installers write crontab entries as a matter of course. Nobody is removing it.</p>



<p class="wp-block-paragraph"><strong>&#8220;Timers are too complicated for small jobs.&#8221;</strong> Two files is more typing, not more complexity. The complexity that matters is the wrapper script you write to get locking, logging and a runtime cap out of cron, and how well the next person understands it. Compare like for like.</p>



<p class="wp-block-paragraph"><strong>&#8220;The journal means I have logging sorted.&#8221;</strong> The journal is a ring buffer with a size cap. If nobody set <code>SystemMaxUse=</code> deliberately and nobody ships journal entries anywhere else, your evidence has a retention window you never chose. Check it before you rely on it.</p>



<p class="wp-block-paragraph"><strong>&#8220;Switching to timers will stop jobs failing silently.&#8221;</strong> This is the one I&#8217;d push back on hardest. Timers make failures <em>visible</em>. They do not make them <em>noticed</em>. Neither scheduler will ever tell you that a job did not run, because absence is not an event. That needs a dead man&#8217;s switch: the job pings a heartbeat endpoint on success, and something external alerts when the ping stops. Healthchecks.io, Cronitor and Better Stack all sell exactly this, and Healthchecks is open source if you would rather self-host it next to your own Grafana. If you already run Prometheus, writing a completion timestamp through the node exporter&#8217;s textfile collector and alerting on its age gets you the same property without another vendor.</p>



<p class="wp-block-paragraph"><strong>&#8220;Just move everything to timers.&#8221;</strong> A mass migration of working jobs is a lot of change for no new capability, and every converted job is a chance to fat-finger a path. Convert on cause, not on principle.</p>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Are systemd timers better than cron?</h3>



<p class="wp-block-paragraph">For production jobs on a systemd host, generally yes: logging, dependency ordering, overlap protection, catch-up and resource limits are all built in rather than bolted on. For a one-line job on a box you own, cron&#8217;s simplicity is a real advantage and the timer buys you nothing you will use.</p>



<h3 class="wp-block-heading">Is cron deprecated or being removed?</h3>



<p class="wp-block-paragraph">No. Cron implementations are still packaged and maintained across the major distributions. Some distributions do not install a cron daemon in minimal images by default, which is a packaging decision rather than a deprecation, so check with your package manager before assuming a fresh host has one.</p>



<h3 class="wp-block-heading">Can cron and systemd timers coexist on the same server?</h3>



<p class="wp-block-paragraph">Yes, and most real servers run both. The cron daemon itself is usually managed as a systemd service. The only rule is that a given job should be scheduled in exactly one place. Double-scheduling is what produces the &#8220;why did this run twice&#8221; ticket.</p>



<h3 class="wp-block-heading">How do I see when a systemd timer will next run?</h3>



<p class="wp-block-paragraph"><code>systemctl list-timers</code> shows next and last elapse for all active timers; add <code>--all</code> to include inactive ones. To test an expression without creating a unit, use <code>systemd-analyze calendar "Mon..Fri *-*-* 06:30:00"</code> with <code>--iterations=N</code> for more fire times.</p>



<h3 class="wp-block-heading">How do I stop a cron job from overlapping itself?</h3>



<p class="wp-block-paragraph">Wrap it in <code>flock -n</code> with a dedicated lock file. The <code>-n</code> flag makes the new run exit straight away instead of queuing. Do not use a PID file you wrote yourself; it will leak a stale lock the first time the job is killed, and then the job stops running forever.</p>



<h3 class="wp-block-heading">Do systemd timers work inside Docker containers?</h3>



<p class="wp-block-paragraph">Not usefully. A container normally runs a single process without systemd as PID 1, so there is no service manager to own a timer. Schedule from outside the container instead: a host timer that runs the container, or the scheduler your platform provides. Podman is the exception worth knowing, since it can generate systemd units for containers and the host&#8217;s systemd drives them.</p>



<h3 class="wp-block-heading">How do I convert a crontab line to a systemd timer?</h3>



<p class="wp-block-paragraph">Move the command into a <code>Type=oneshot</code> service with an absolute <code>ExecStart</code>, then translate the five cron fields into an <code>OnCalendar=</code> expression of the form <code>DayOfWeek Year-Month-Day Hour:Minute:Second</code>. Verify the expression with <code>systemd-analyze calendar</code>, enable the timer, then remove the crontab line only after you have seen a successful run in the journal.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">Cron vs systemd timers is not a contest between an old tool and a new one. Cron is a scheduler. A systemd timer is a scheduler attached to a supervisor, and the supervisor is what you are actually buying: dependency ordering, overlap protection, catch-up, resource limits, and output that lands somewhere you can read it.</p>



<p class="wp-block-paragraph">Pick timers when a missed or duplicated run costs you something. Pick cron when it doesn&#8217;t. And whichever you pick, add a heartbeat, because the failure that hurts is not the job that crashes. It&#8217;s the job that stopped running while everything stayed green.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on your scheduled jobs?</h2>



<p class="wp-block-paragraph">Scheduling is one of those areas where the problem is rarely the syntax. It&#8217;s the job nobody has checked since the person who wrote it left. This is the kind of work I take on:</p>



<ul class="wp-block-list">

<li>Auditing every crontab and timer on a host and telling you which jobs have quietly stopped running</li>


<li>Converting fragile cron entries to systemd timers with locking, runtime caps and failure handlers, without a big-bang migration</li>


<li>Fixing the classic &#8220;works by hand, fails on schedule&#8221; environment and PATH problems, including the percent-sign trap</li>


<li>Adding heartbeat monitoring and alerting so a missed run pages someone instead of sitting there</li>


<li>Backup and dump jobs that verify their own output rather than exiting zero on an empty file</li>


<li>Untangling duplicate schedules left behind by control panels, deploy scripts and configuration management</li>

</ul>



<p class="wp-block-paragraph">If something on your server isn&#8217;t running when it should, send me the crontab line or the unit file and the journal output around the time it should have fired. That&#8217;s usually enough to tell you what&#8217;s wrong before we talk about anything else.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/cron-vs-systemd-timers/">Cron vs systemd Timers: The Failure Modes That Decide It</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Post-Migration Checklist: What Breaks Quietly in the First 48 Hours</title>
		<link>https://john-nessime.com/blog/devops/post-migration-checklist-48-hours/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 20 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[System Administration]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Certbot]]></category>
		<category><![CDATA[Cron]]></category>
		<category><![CDATA[Cutover Planning]]></category>
		<category><![CDATA[Decommissioning]]></category>
		<category><![CDATA[DKIM]]></category>
		<category><![CDATA[DNS Migration]]></category>
		<category><![CDATA[Email Deliverability]]></category>
		<category><![CDATA[Let's Encrypt]]></category>
		<category><![CDATA[MX Records]]></category>
		<category><![CDATA[Restore Testing]]></category>
		<category><![CDATA[Rollback Plan]]></category>
		<category><![CDATA[Synthetic Monitoring]]></category>
		<category><![CDATA[Troubleshooting]]></category>
		<category><![CDATA[TTL]]></category>
		<category><![CDATA[Uptime Monitoring]]></category>
		<category><![CDATA[Website Migration]]></category>
		<category><![CDATA[WP-Cron]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=278</guid>

					<description><![CDATA[<p>Migrations rarely fail loudly. The site stays up while mail, cron, certificate renewal and the write path break silently behind it. A post-migration checklist for the first 48 hours, ordered by how long each failure can hide rather than by subsystem.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/post-migration-checklist-48-hours/">Post-Migration Checklist: What Breaks Quietly in the First 48 Hours</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The ticket you don&#8217;t want after a migration doesn&#8217;t say the site is down. It says nobody has received a contact form email since Friday.</p>



<p class="wp-block-paragraph">The site never went down. The uptime monitor stayed green all weekend. Analytics looked normal. And a few days of enquiries went somewhere nobody thought to look.</p>



<p class="wp-block-paragraph">That&#8217;s the shape of most migration damage. Not the 502. A 502 gets fixed inside ten minutes because everyone sees it at once, including the client. The expensive failures are the ones with no symptom on the front end: mail that authenticates on the old IP and not the new one, a cron job that never came across, a certificate that is valid today and will fail to renew long after everyone has stopped paying attention, orders written to a database that is about to be deleted.</p>



<p class="wp-block-paragraph">This post-migration checklist covers the first 48 hours after cutover. It&#8217;s ordered by when each failure actually becomes visible, not by category, because that ordering is what decides which problems you catch yourself and which ones reach you as a client ticket a fortnight later.</p>



<h2 class="wp-block-heading">Why a post-migration checklist should be ordered by detection latency</h2>



<p class="wp-block-paragraph">Most migration checklists you&#8217;ll find are grouped by subsystem: DNS, SSL, email, SEO, performance. That&#8217;s a fine way to write a document and a bad way to work a cutover, because it treats a 500 error on the homepage and a broken certificate renewal as equally urgent. They aren&#8217;t. One will be reported to you within minutes by someone who isn&#8217;t you. The other has a detection latency measured in weeks, and it lands at 3am.</p>



<p class="wp-block-paragraph">So sort your checks by how long the failure can hide. Short latency, catch it reactively. Long latency, go and look for it on purpose, while you still have the old server, the old data and a rollback path. That&#8217;s what makes 48 hours the right frame: long enough for a full daily cron cycle and for DNS to settle at any sane TTL, short enough that the old server is still sitting there with the evidence on it.</p>



<h2 class="wp-block-heading">Hour 0 to 1: prove you are actually looking at the new server</h2>



<p class="wp-block-paragraph">Almost everyone starts by loading the site in a browser and saying it looks fine. That check is worth close to nothing an hour after cutover. Your laptop may still hold the old answer, your ISP resolver definitely might, and if you added a hosts entry during testing you&#8217;re now testing the thing you already tested. Ask several resolvers directly, then bypass DNS entirely and talk to the new IP.</p>



<pre class="wp-block-code"><code># What is each public resolver answering, and how long will it keep answering it?
dig example.com A @1.1.1.1
dig example.com A @8.8.8.8
dig example.com A @9.9.9.9

# Talk to the new server directly, ignoring DNS completely
curl -sI --resolve example.com:443:203.0.113.10 https://example.com/

# Which certificate is that IP presenting for that hostname?
openssl s_client -connect 203.0.113.10:443 -servername example.com &lt;/dev/null 2&gt;/dev/null 
  | openssl x509 -noout -subject -issuer -dates</code></pre>



<p class="wp-block-paragraph">The full <code>dig</code> answer section, not <code>+short</code>, matters here because it shows the remaining TTL. That number tells you how much longer stale answers can survive, which is the single most useful figure you have in the first hour.</p>



<p class="wp-block-paragraph">The <code>--resolve</code> flag pins a hostname to an address for that one request, so you get the new server&#8217;s response with the correct SNI and Host header. It&#8217;s the closest thing to a definitive answer you&#8217;ll get while records are still in flight. If you want to see what the rest of the world sees, an exit in another country helps, and a VPN account with NordVPN or Surfshark is a cheaper way to get one than spinning up a box in a second region.</p>



<p class="wp-block-paragraph">Things to confirm in this window:</p>



<ul class="wp-block-list">
<li>The response comes from the new server, verified by something unique to it rather than by the page looking right. A temporary header or a file at a path that only exists on the new box removes all doubt. <em>Pass test: you can name the specific artefact that proves it.</em></li>

<li>The correct virtual host is selected, not a catch-all default. Sites with several domains on one server fail here quietly, serving the wrong document root over a valid certificate. <em>Pass test: every hostname you migrated returns its own content, checked individually.</em></li>

<li>The certificate chain is complete on the new box, not just the leaf. Browsers often paper over a missing intermediate using cached data; API clients and payment callbacks do not. <em>Pass test: an external SSL checker reports a complete chain from a machine that has never visited the site.</em></li>

<li>Redirect behaviour matches the old server exactly. HTTP to HTTPS, www to apex or the reverse, trailing slashes. <em>Pass test: a handful of known deep URLs return the same final status and location as before.</em></li>

<li>Any hosts file entries you added for testing are gone. Leave one in place and you&#8217;ll misread every check you run for the next two days.</li>
</ul>



<h2 class="wp-block-heading">Hour 1 to 6: the split-brain window, where you lose data instead of uptime</h2>



<p class="wp-block-paragraph">This is the part that costs real money, and it&#8217;s the part almost nobody checks, because during it both servers are healthy. Some resolvers still hand out the old address. Those visitors reach the old application, which happily accepts their form submission, their order, their uploaded file, their account signup. Then you delete that server.</p>



<p class="wp-block-paragraph">&#8220;Keep the old server running for 48 hours&#8221; is standard advice and it&#8217;s necessary, but on its own it makes the problem worse rather than better. A running old server that still accepts writes is a data-loss machine. What you want is an old server that stays reachable and stops accepting writes.</p>



<p class="wp-block-paragraph">Three options, in order of how much I&#8217;d trust them:</p>



<ol class="wp-block-list">
<li>Proxy the old server to the new one. Every request that lands on the old address gets forwarded to the new backend, so there is only ever one database. This is the only option that genuinely eliminates the split, and it&#8217;s worth the extra config on anything transactional.</li>

<li>Redirect the old server to a hostname that only resolves to the new IP. Cheap, works for GET traffic, does nothing useful for a POST that&#8217;s already in flight.</li>

<li>Put the old application into a read-only or maintenance state. Safe, but visitors on stale DNS see a broken site for the length of your TTL, so this only works if that number is genuinely small.</li>
</ol>



<p class="wp-block-paragraph">Whichever you pick, go and look at what the old box actually did after the cutover.</p>



<pre class="wp-block-code"><code># On the OLD server: has anything been written since cutover?
grep '"POST ' /var/log/nginx/access.log | tail -n 50

# Apache layouts vary; on DirectAdmin the per-domain log lives here
grep '"POST ' /var/log/httpd/domains/example.com.log | tail -n 50

# Is the old box still receiving mail for the domain?
tail -n 200 /var/log/maillog</code></pre>



<p class="wp-block-paragraph">Every POST in that log after the cutover timestamp is a potential lost record. Take the list to the application: compare row counts or highest IDs on the write-heavy tables between old and new, and reconcile before you decommission anything.</p>



<p class="wp-block-paragraph">Mail deserves its own pass. If MX records changed, messages go to whichever server the sending system had cached, and land in two different mailboxes. Keep both able to accept mail through the overlap, and check the old mailbox before you close the account, not after.</p>



<p class="wp-block-paragraph">Two more things here that won&#8217;t show up in any log you&#8217;re watching:</p>



<ul class="wp-block-list">
<li><strong>Anything pinned to an IP address.</strong> Payment gateway allowlists, webhook endpoints, database firewall rules, SMTP relay authorisations, third-party API allowlists. None of these follow a redirect. A webhook posted to the old IP returns a perfectly good 200 and the new application never hears about the payment.</li>

<li><strong>Outbound connections from the new IP.</strong> Your application is now calling other people&#8217;s APIs from an address they&#8217;ve never seen. If any of those partners allowlist by IP, the failure appears as a timeout in a background job rather than an error on a page.</li>
</ul>



<h2 class="wp-block-heading">Hour 6 to 24: everything that runs on a schedule</h2>



<p class="wp-block-paragraph">Files and databases get migrated. Scheduled work frequently does not, because it lives outside the document root and outside the database dump, and because nothing on the site breaks when it&#8217;s missing.</p>



<pre class="wp-block-code"><code># Every user's crontab, not just root's
for u in $(cut -d: -f1 /etc/passwd); do echo "== $u"; crontab -l -u "$u" 2&gt;/dev/null; done

# systemd timers, where a lot of "cron" actually lives now
systemctl list-timers --all

# WordPress: what is scheduled, and is anything overdue?
wp cron event list --fields=hook,next_run_relative,recurrence
wp config get DISABLE_WP_CRON</code></pre>



<p class="wp-block-paragraph">Run that on both servers and diff the output. That comparison is the whole check, and it takes two minutes.</p>



<p class="wp-block-paragraph">The specific traps worth knowing:</p>



<ul class="wp-block-list">
<li><strong>The old server&#8217;s cron is still running.</strong> Nobody stops it, because &#8220;keep it up for 48 hours&#8221; sounded like keep everything up. Now your nightly export runs twice, your reminder emails send twice, and your billing job runs against a database nobody is reading. Disable cron on the old box at cutover, before you disable anything else.</li>

<li><strong>WP-Cron was disabled in config and the replacement never came across.</strong> If <code>DISABLE_WP_CRON</code> is true and no system cron hits <code>wp-cron.php</code>, every scheduled task stops silently. Scheduled posts sit at &#8220;Missed schedule&#8221;, backups stop, WooCommerce action queues back up. The site itself works perfectly.</li>

<li><strong>The cron URL is wrong in a way that still returns success.</strong> A job calling an <code>http://</code> URL that now redirects to HTTPS gets a 3xx, exits zero, and looks healthy in the log while the task never executes. Same story for a cron still pointing at the old hostname or an old absolute path.</li>

<li><strong>Backups.</strong> The job may have moved and still be writing to storage credentials, a mount, or a destination path that only existed on the old host. Don&#8217;t check that the backup job ran. Check that a file arrived at the destination, and that it&#8217;s the size you expect.</li>

<li><strong>Log rotation.</strong> A fresh server often has no rotation configured for application logs. Nothing goes wrong for three days, then the disk fills at 2am and every write fails at once.</li>
</ul>



<h2 class="wp-block-heading">Hour 24 to 48: the delayed detonations</h2>



<p class="wp-block-paragraph">These are the checks with the longest detection latency, which makes them the ones most worth doing deliberately.</p>



<h3 class="wp-block-heading">Certificate renewal, not the certificate</h3>



<p class="wp-block-paragraph">A copied certificate is valid for weeks. What breaks in a migration is the machinery that replaces it: the webroot path in the renewal config no longer exists, the ACME account keys weren&#8217;t copied, port 80 is closed on the new firewall, or a WAF is now intercepting the challenge path. None of that surfaces until the renewal window opens, long after everyone has moved on.</p>



<pre class="wp-block-code"><code># What does certbot think it manages, and how does it validate?
certbot certificates

# The only check that matters: will the NEXT renewal succeed?
certbot renew --dry-run

# Is anything actually going to run it?
systemctl list-timers | grep -i certbot</code></pre>



<p class="wp-block-paragraph">The dry run validates against Let&#8217;s Encrypt&#8217;s staging environment, so it exercises the real challenge path without consuming your issuance quota. Resist the urge to fix a failure with a forced renewal: Let&#8217;s Encrypt applies rate limits per registered domain and a separate limit on identical certificates, and burning through them during a migration is how a fixable config problem turns into a week of waiting.</p>



<h3 class="wp-block-heading">Mail identity on an address with no history</h3>



<p class="wp-block-paragraph">If the new server sends mail, it&#8217;s sending from an IP that receiving systems have never seen. Authentication correctness and reputation are two separate problems and they fail differently. Authentication failures bounce or land in spam immediately. Reputation problems look like intermittent delivery to one provider and normal delivery to another.</p>



<pre class="wp-block-code"><code># Reverse DNS for the new sending IP
dig +short -x 203.0.113.10

# Does SPF still authorise everything that sends for you?
dig +short TXT example.com

# DKIM selector (replace 'selector') and DMARC policy
dig +short TXT selector._domainkey.example.com
dig +short TXT _dmarc.example.com</code></pre>



<p class="wp-block-paragraph">Send a test message to an address at a major provider and read the full headers. The Authentication-Results header tells you whether SPF, DKIM and DMARC each passed, and crucially whether the passing domain aligns with the visible From address. SPF can pass for the envelope sender and still fail DMARC when the From domain differs, which is a failure mode you cannot see from the sending side.</p>



<p class="wp-block-paragraph">Leave the old server&#8217;s SPF authorisation and DKIM public key published through the overlap. Mail that was queued or delayed on the old path still needs to verify. Sources differ on how long to keep them, and the honest answer is: until you have confirmed nothing is still sending or retrying from the old host.</p>



<p class="wp-block-paragraph">If transactional mail matters to the business, the migration is a reasonable moment to stop sending it from a web server at all. Handing it to a dedicated provider such as Postmark, Mailgun or Amazon SES moves reputation off an address you&#8217;ve just changed and gives you delivery logs you can actually query.</p>



<h3 class="wp-block-heading">Search crawl, redirects and staging leftovers</h3>



<p class="wp-block-paragraph">Crawlers hit harder than usual after a move, because they&#8217;re re-checking old URLs and discovering new ones at the same time. That surge is also your best early warning: it surfaces broken redirects faster than your own testing will.</p>



<ul class="wp-block-list">
<li>Fetch <code>robots.txt</code> from the live site and read it. A staging <code>Disallow: /</code> that survives cutover is the single most damaging thing on this list.</li>

<li>View source on several page types and check for a leftover noindex tag and for canonical tags pointing at a staging hostname.</li>

<li>Watch 404s in the access log rather than in a crawler report. It&#8217;s your missed-redirects list, sorted by how much anyone cares.</li>

<li>Leave the old URLs crawlable through the redirect. Blocking them means the redirects are never seen.</li>
</ul>



<h3 class="wp-block-heading">Resource behaviour under real traffic</h3>



<p class="wp-block-paragraph">Staging never told you what the process pool does at peak. Twenty-four hours of production traffic does.</p>



<pre class="wp-block-code"><code># Status code distribution since cutover
awk '{print $9}' /var/log/nginx/access.log | sort | uniq -c | sort -rn | head

# Disk, and the one people forget
df -h
df -i</code></pre>



<p class="wp-block-paragraph">Compare the ratio of 5xx to total requests against the old server&#8217;s logs for the same weekday. Absolute numbers mislead here; the ratio doesn&#8217;t. Watch memory across a full daily cycle rather than at a single moment, because the interesting shape is a slow climb into a limit, not a spike.</p>



<h2 class="wp-block-heading">Before you decommission the old server</h2>



<p class="wp-block-paragraph">The old server is the only copy of some things, and none of them are in your backup.</p>



<ul class="wp-block-list">
<li>Access and error logs covering the cutover window, which is your evidence for anything that surfaces next month.</li>

<li>Mail spools, and any messages that arrived after the MX change.</li>

<li>The full crontab set, exported to a file rather than a screenshot.</li>

<li>Web server, PHP and database configuration, so you can diff a behavioural difference you notice later.</li>

<li>A final database dump taken after write traffic stopped.</li>
</ul>



<p class="wp-block-paragraph">Keep the TTL low until you&#8217;ve decided you&#8217;re staying. Raising it back to normal is the last step of the migration, not the first thing you do once the site loads. Another month on an InterServer or Contabo VPS is cheap next to reconstructing a week of orders, and the overlap is the only period in which rolling back is still a five-minute operation.</p>



<h2 class="wp-block-heading">Troubleshooting: three symptoms and what they usually mean</h2>



<p class="wp-block-paragraph"><strong>&#8220;It works for me but not for them.&#8221;</strong> Almost always a resolution split. Get the affected user to report what their resolver returns, then check whether the answer is stale DNS, a corporate resolver ignoring your TTL, or a CDN edge still holding the old origin. If you use Cloudflare or a similar proxy in front, remember the origin change and the DNS change are two separate things and only one of them propagates.</p>



<p class="wp-block-paragraph"><strong>&#8220;The site is fine but our mail goes to spam now.&#8221;</strong> Work in this order: reverse DNS for the sending IP, then SPF including the new address, then DKIM signing on the new host, then DMARC alignment with the visible From domain. Only after all four pass should you start thinking about reputation and volume, and reputation on a new IP is a matter of ramping gradually rather than something you can configure.</p>



<p class="wp-block-paragraph"><strong>&#8220;Random 500s that never happened in staging.&#8221;</strong> Look at limits before you look at code. Process pool size, memory limits, database connection caps, open file limits, and file ownership on directories the application writes to. Permission problems after an rsync or archive restore are common and produce errors that look nothing like permission problems from the front end.</p>



<h2 class="wp-block-heading">Common mistakes in the first 48 hours</h2>



<ul class="wp-block-list">
<li>Treating a green uptime check as verification. It confirms one URL returns 200. It says nothing about mail, cron, writes or checkout.</li>

<li>Leaving the old server fully operational, cron and all, instead of leaving it reachable but inert.</li>

<li>Testing while a hosts file entry is still in place.</li>

<li>Raising the TTL back to normal on day one, which throws away your fast rollback.</li>

<li>Checking that the certificate is valid instead of checking that renewal works.</li>

<li>Silencing monitoring for the maintenance window and forgetting to unsilence it.</li>

<li>Declaring the migration done before a full daily cron cycle has completed on the new server.</li>
</ul>



<h2 class="wp-block-heading">If you only do five things</h2>



<ol class="wp-block-list">
<li>Stop writes and cron on the old server, but keep it reachable.</li>

<li>Run <code>certbot renew --dry-run</code> and confirm a timer or cron will actually invoke it.</li>

<li>Send a real test email to a major provider and read the Authentication-Results header.</li>

<li>Diff the scheduled jobs on both servers, then confirm one real backup file landed at its destination.</li>

<li>Grep the old server&#8217;s access log for POST requests after the cutover, and reconcile anything you find.</li>
</ol>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">How long should I keep the old server running after a migration?</h3>



<p class="wp-block-paragraph">Long enough that no resolver anywhere is still handing out the old address, plus enough margin to reconcile data and pull logs. For most sites that&#8217;s a few days. The cost of the extra billing period is trivial compared with losing your rollback path, but keep it inert: reachable, not writable, with cron disabled.</p>



<h3 class="wp-block-heading">Why is my site fine but email broken after a server migration?</h3>



<p class="wp-block-paragraph">Because web and mail are validated by different systems using different records. The web path only needs an A record and a certificate. Mail needs the MX record, reverse DNS on the sending IP, SPF listing that IP, DKIM signing configured on the new host, and DMARC alignment with your From domain. A migration can satisfy the first set completely and break every item in the second.</p>



<h3 class="wp-block-heading">What should I check first if something looks wrong right after cutover?</h3>



<p class="wp-block-paragraph">Establish which server you are talking to before you diagnose anything else. Use <code>curl --resolve</code> to pin the hostname to the new IP and compare that response with what you get through normal DNS. Half of post-cutover panic is people debugging the old server.</p>



<h3 class="wp-block-heading">Can I lose form submissions or orders during DNS propagation?</h3>



<p class="wp-block-paragraph">Yes, and it&#8217;s the most common serious loss in a migration. Visitors on cached DNS reach the old application, which writes to the old database. Nothing errors. Proxying the old server to the new backend removes the problem entirely; short of that, freeze writes during the overlap and check the old access log for POST requests before decommissioning.</p>



<h3 class="wp-block-heading">Why did my SSL certificate stop renewing weeks after the migration?</h3>



<p class="wp-block-paragraph">The certificate itself migrated fine; the renewal configuration didn&#8217;t. Common causes are a webroot path that no longer exists, an ACME account directory that wasn&#8217;t copied, port 80 blocked by the new firewall, a proxy or WAF intercepting the challenge path, or no timer scheduled to run the renewal at all. A dry run within 48 hours catches all of these.</p>



<h3 class="wp-block-heading">Do I need to tell search engines about a server migration?</h3>



<p class="wp-block-paragraph">If the URLs are unchanged and only the hosting moved, no announcement is required. Your job is to make sure crawlers can reach the new server, that nothing carries a staging noindex or a blocking robots rule, and that canonical tags point at live URLs. If the domain or URL structure changed, that&#8217;s a different migration with its own redirect and site-move process.</p>



<h3 class="wp-block-heading">What monitoring should I add specifically for the 48-hour window?</h3>



<p class="wp-block-paragraph">Add checks for the things a homepage ping cannot see: a synthetic transaction through a real form or checkout, a heartbeat from each scheduled job, certificate expiry, and a content check that asserts the right text is present rather than just a 200 status. Most uptime platforms handle the first and last; heartbeat monitoring for cron is the one people skip and the one that catches the silent failures.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">A migration doesn&#8217;t fail when the site goes down. It fails when the site stays up and something behind it stops, quietly, in a way nobody is looking for. Every item on this post-migration checklist exists because the failure it catches produces no visible symptom on the front end.</p>



<p class="wp-block-paragraph">So work the 48 hours in order of detection latency. Prove which server you&#8217;re on, close the write path on the old one, diff the scheduled jobs, dry-run the certificate renewal, and read the headers on a real test email. Then, and only then, raise the TTL and let the old box go.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on a cutover?</h2>



<p class="wp-block-paragraph">Migrations are one of the few pieces of infrastructure work where the review matters more than the execution. Things I help with:</p>



<ul class="wp-block-list">
<li>Planning the cutover so the write path is closed on the old server rather than left open, including proxy-forward setups for transactional sites.</li>

<li>Running the 48-hour verification pass on a migration someone else performed, and telling you plainly what&#8217;s still broken.</li>

<li>Repairing certificate renewal, ACME challenge paths and reload hooks that stopped working after the move.</li>

<li>Fixing mail on a new IP: reverse DNS, SPF, DKIM signing and DMARC alignment, or moving transactional mail off the web server entirely.</li>

<li>Rebuilding scheduled jobs, backup destinations and log rotation on the new host, with heartbeat monitoring so silence becomes an alert.</li>

<li>Reconciling data written to the old server during the split window, before anything gets deleted.</li>
</ul>



<p class="wp-block-paragraph">If you&#8217;re mid-migration or just came out of one, send me something concrete: the old server&#8217;s access log around the cutover, your crontab, or the headers from a test email. That&#8217;s usually enough to tell you what&#8217;s wrong.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/post-migration-checklist-48-hours/">Post-Migration Checklist: What Breaks Quietly in the First 48 Hours</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
