<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Git | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/git/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/git/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Sun, 20 Sep 2026 10:25:53 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>Git | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/git/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>GitLab CI vs GitHub Actions vs Jenkins: Choosing Without Regretting It Later</title>
		<link>https://john-nessime.com/blog/devops/gitlab-ci-vs-github-actions-vs-jenkins/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 13 Aug 2026 10:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Automation]]></category>
		<category><![CDATA[CI/CD]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[Deployment]]></category>
		<category><![CDATA[Git]]></category>
		<category><![CDATA[GitHub Actions]]></category>
		<category><![CDATA[GitLab CI]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Jenkins]]></category>
		<category><![CDATA[OIDC]]></category>
		<category><![CDATA[Pipeline Design]]></category>
		<category><![CDATA[Platform Engineering]]></category>
		<category><![CDATA[Secrets Management]]></category>
		<category><![CDATA[Self Hosting]]></category>
		<category><![CDATA[Supply Chain Security]]></category>
		<category><![CDATA[Vendor Lock-In]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=201</guid>

					<description><![CDATA[<p>A practical comparison of GitLab CI, GitHub Actions and Jenkins that skips the feature table. What actually decides the choice is where your code lives, whether the runner can reach the deploy target, and who owns the control plane at 2am. Includes the cost mechanics, the security failure modes, and a decision procedure you can run in an afternoon.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/gitlab-ci-vs-github-actions-vs-jenkins/">GitLab CI vs GitHub Actions vs Jenkins: Choosing Without Regretting It Later</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The question almost always arrives the same way. Someone pastes a long Jenkinsfile into a channel, adds a shrug emoji, and asks how hard it would be to move this to GitHub Actions. Nobody in the thread is asking the questions that will actually decide the outcome, because those questions are boring and the YAML is right there looking portable.</p>



<p class="wp-block-paragraph">Here is the thing that catches teams out: the pipeline definition is the cheapest part of a CI system to move. Rewriting stages and steps is a few days of tedious work for a normal service. What does not move is everything underneath it, and that is where a two-week migration turns into a two-quarter one.</p>



<p class="wp-block-paragraph">This post compares <strong>GitLab CI vs GitHub Actions vs Jenkins</strong> on the things that actually differ once you are running them in anger: who owns the runners, where secrets live, how the bill is calculated, and what breaks when an outside contributor opens a pull request. There is a profile of each with an honest &#8220;where it wins and where it doesn&#8217;t&#8221;, a section on cost mechanics, a section on the security failure modes, and a decision procedure at the end you can run in an afternoon.</p>



<h2 class="wp-block-heading">The part of the decision that actually bites</h2>



<p class="wp-block-paragraph">Draw a line under your pipeline file. Everything above the line is syntax. Everything below is platform.</p>



<p class="wp-block-paragraph">Below the line sits the runner fleet and how it is scaled, the secret store and how jobs authenticate to it, artifact and container registry storage with its retention rules, deploy credentials and the network path they travel, approval gates and who can click them, and the audit trail that someone in compliance will eventually ask about. None of that ports. All of it has to be rebuilt on the new platform, and each piece has a person or a team attached to it who has opinions.</p>



<p class="wp-block-paragraph">The failure mode this produces is specific and predictable. A team migrates the build and test half of the pipeline because that part is easy and demos well. Deploy stays on the old system &#8220;for now&#8221;, because deploy touches production and nobody wants to be the one who broke it during a migration. Six months later there are two CI systems, two secret stores, two sets of runners to patch, two audit trails, and a deploy path that nobody can explain end to end. The migration never technically failed. It just never finished, and the running cost doubled permanently.</p>



<p class="wp-block-paragraph">So the real question is not which YAML dialect you prefer. It is whether you are prepared to move the whole thing, and which platform makes the &#8220;below the line&#8221; half cheapest for your specific situation.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">What each platform is really optimising for</h2>



<p class="wp-block-paragraph">Feature tables make these three look more similar than they are. They are optimising for genuinely different things, and the differences show up as soon as you leave the happy path.</p>



<h3 class="wp-block-heading">GitHub Actions: shortest path from push to green</h3>



<p class="wp-block-paragraph">Actions is built around events in the GitHub repository. Something happens, a workflow fires, jobs land on hosted runners, and you did not have to provision anything. The marketplace means most common tasks are already someone else&#8217;s problem. Reusable workflows let a platform team define a pipeline once and have every service call it.</p>



<p class="wp-block-paragraph">A minimal hardened job header looks like this, and every line in it is there for a reason:</p>



<pre class="wp-block-code"><code>name: ci

on:
  pull_request:
    branches: [main]

permissions:
  contents: read      # start read-only, raise per job where needed
  id-token: write     # only for jobs that federate to a cloud provider

concurrency:
  group: ci-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@PIN_TO_A_40_CHAR_COMMIT_SHA
      - name: Use the PR title safely
        env:
          PR_TITLE: ${{ github.event.pull_request.title }}
        run: echo "$PR_TITLE"</code></pre>



<p class="wp-block-paragraph">The <code>permissions</code> block caps what the automatically provided job token can do, so a compromised step cannot push to your branches. The <code>concurrency</code> block cancels superseded runs when someone pushes three times in a minute, which is the single easiest compute saving available. Pinning the action to a commit SHA rather than a tag matters because tags are mutable and have been repointed to malicious commits in real supply chain incidents. And binding the pull request title to an environment variable instead of interpolating it directly into the shell is what stops an attacker-controlled string from being executed as a command.</p>



<p class="wp-block-paragraph"><strong>Where it wins:</strong> your code is already on GitHub, you want CI running today, and your deploy targets are cloud APIs you can reach over the internet with a federated identity token. For open source it is close to unbeatable, since standard hosted runners are free for public repositories.</p>



<p class="wp-block-paragraph"><strong>Where it doesn&#8217;t:</strong> anything needing deep control of the execution environment, long-running or stateful builds, deploys into networks the internet cannot reach, or a compliance posture that requires build logs and secrets to stay inside your perimeter. Self-hosted runners solve the network problem but hand you back the maintenance you were trying to avoid.</p>



<h3 class="wp-block-heading">GitLab CI: one platform owning the whole lifecycle</h3>



<p class="wp-block-paragraph">GitLab&#8217;s pitch is that source control, CI, package registry, container registry, security scanning and issue tracking are one product with one permission model and one audit trail. That is a real architectural advantage, not marketing. When a scanner finding, the merge request that introduced it, the pipeline that caught it and the registry image that contains it are all the same system, you spend a lot less time gluing tools together.</p>



<p class="wp-block-paragraph">The pipeline language rewards you for using its graph features rather than treating stages as a queue:</p>



<pre class="wp-block-code"><code>workflow:
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH

default:
  interruptible: true

stages: [build, test, deploy]

build:
  stage: build
  script:
    - make build
  artifacts:
    paths: [dist/]
    expire_in: 1 week

unit:
  stage: test
  needs: [build]
  script:
    - make test

deploy:
  stage: deploy
  needs: [unit]
  environment: production
  rules:
    - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
      when: manual
  script:
    - ./deploy.sh</code></pre>



<p class="wp-block-paragraph">The <code>workflow:rules</code> block is the one people skip and then wonder why every push runs two pipelines. It decides whether a pipeline is created at all. <code>needs</code> turns the stage list into a directed graph, so a job starts the moment its own dependency finishes rather than waiting for every job in the previous stage. <code>interruptible: true</code> makes jobs eligible for automatic cancellation when a newer pipeline supersedes them, which you then have to enable in the project&#8217;s CI settings; the keyword alone does nothing. And <code>expire_in</code> is worth setting deliberately, because artifact storage is billed separately from compute and quietly accumulates.</p>



<p class="wp-block-paragraph"><strong>Where it wins:</strong> you want one vendor for the whole lifecycle, you have compliance requirements that benefit from a single auth model and audit trail, or you need self-managed hosting without giving up the integrated experience. The self-managed edition with your own runners is genuinely one of the better value propositions in this space at moderate to high build volume.</p>



<p class="wp-block-paragraph"><strong>Where it doesn&#8217;t:</strong> your code is not in GitLab. The CI is tightly coupled to GitLab repositories, and running it against code hosted elsewhere is possible but perpetually awkward. The third-party ecosystem is also smaller, so more of what you need you will write yourself. And self-managing GitLab is a real operational commitment, not a Docker Compose file you forget about.</p>



<h3 class="wp-block-heading">Jenkins: whatever you need, on hardware you control</h3>



<p class="wp-block-paragraph">Jenkins gets dismissed as legacy by people who have never had a requirement it was the only tool to meet. It runs on your hardware, in your network, with no dependency on a vendor&#8217;s availability or pricing decisions. It talks to source control systems the other two do not. It can drive hardware test rigs, mainframe jobs, and build agents on operating systems nobody else offers. If your pipeline data legally cannot leave your network, this is a short list and Jenkins is on it.</p>



<pre class="wp-block-code"><code>pipeline {
  agent { label 'linux-ephemeral' }

  options {
    timeout(time: 30, unit: 'MINUTES')
    disableConcurrentBuilds()
    buildDiscarder(logRotator(numToKeepStr: '30'))
    timestamps()
  }

  stages {
    stage('Build') {
      steps {
        sh 'make build'
      }
    }
    stage('Deploy') {
      when { branch 'main' }
      steps {
        withCredentials([string(credentialsId: 'deploy-token', variable: 'DEPLOY_TOKEN')]) {
          sh './deploy.sh'
        }
      }
    }
  }

  post {
    always {
      cleanWs()
    }
  }
}</code></pre>



<p class="wp-block-paragraph">Two things in that file are worth pointing at. <code>timeout</code> and <code>buildDiscarder</code> are the difference between a controller that runs for years and one that fills its disk and hangs on stuck jobs; set them on every pipeline, not as an afterthought. And <code>timestamps()</code> and <code>cleanWs()</code> both come from plugins rather than core. That is Jenkins in a sentence: the capability exists, and it arrives as a dependency you now own, patch, and eventually discover is unmaintained.</p>



<p class="wp-block-paragraph"><strong>Where it wins:</strong> air-gapped or heavily restricted networks, source control that is not GitHub or GitLab, orchestration of things that are not code builds, and organisations with genuinely unusual workflows that no hosted product models. If you already have working Jenkins infrastructure and a person who understands it, the case for ripping it out is weaker than the migration slides suggest.</p>



<p class="wp-block-paragraph"><strong>Where it doesn&#8217;t:</strong> the controller is a stateful, single-point-of-failure server that someone has to own. Plugins are the extensibility model and also the attack surface, and the project publishes security advisories regularly enough that patching is a standing chore rather than an event. The Java baseline also moves; recent LTS lines have dropped support for older Java versions, so an upgrade is periodically a JVM migration too. If you cannot name the person responsible for that work, do not choose Jenkins.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">GitLab CI vs GitHub Actions vs Jenkins on cost</h2>



<p class="wp-block-paragraph">Published rates change often enough that quoting them ages a post badly. The billing mechanics change far more slowly, and the mechanics are what you actually need to model.</p>



<ul class="wp-block-list"><li><strong>GitHub Actions.</strong> Standard hosted runners are free for public repositories. Private repositories draw from a monthly pool of included minutes tied to your plan, and those minutes are Linux-equivalent: Windows and macOS jobs drain the pool at a multiplier, with macOS by far the steepest. Larger runners do not draw from the included pool at all and bill per minute from the first second. Billing is per job rounded up to the minute, minutes do not roll over, and usage is charged to the repository owner. Self-hosted runner usage carries no per-minute fee, though GitHub has floated changing that and then walked it back, so verify against current billing docs before you build a model on it.</li><li><strong>GitLab CI.</strong> Jobs on GitLab-hosted instance runners consume compute minutes, calculated as job duration multiplied by a cost factor for the machine size. The included pool is per top-level namespace, not per seat, which is the detail that surprises people: a five-person group and a fifty-person group on the same tier start from the same pool. Self-managed runners consume zero quota on every tier including Free, which is why &#8220;pay for seats, run your own runners&#8221; is such a common GitLab shape.</li><li><strong>Jenkins.</strong> No licence cost, which is where most cost comparisons stop and where the interesting part starts. You are paying for the controller, the agent fleet, the storage, and the engineering time to patch all three. That last item is the largest line and the one nobody puts in the spreadsheet.</li></ul>



<p class="wp-block-paragraph">Two observations that matter more than the platform choice itself. First, the levers with the biggest effect on a CI bill are the same everywhere: cancel superseded runs, cache dependencies properly, skip workflows on paths that did not change, and stop running the full matrix on every draft commit. A team that does none of these will pay more on the cheapest platform than a disciplined team pays on the most expensive one.</p>



<p class="wp-block-paragraph">Second, self-hosted runners are cheaper per minute and are not free. A pool of runners on a VPS provider like Hetzner, DigitalOcean or InterServer is inexpensive to rent and still needs image maintenance, autoscaling, disk cleanup and a security model. The break-even against hosted minutes is real, but it arrives later than people expect once you price the engineering hours honestly.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Security is where the differences stop being cosmetic</h2>



<p class="wp-block-paragraph">Every CI system is a machine that holds production credentials and executes code on demand. The platforms differ mostly in which mistakes they make easy.</p>



<ul class="wp-block-list"><li><strong>Untrusted pull requests.</strong> This is the classic GitHub Actions trap. The <code>pull_request</code> trigger runs fork code without access to your secrets. The <code>pull_request_target</code> trigger runs in the privileged context of the base repository, and if a workflow using it checks out the pull request&#8217;s code, an outside contributor can execute arbitrary code with your secrets in scope. GitHub&#8217;s own guidance is to avoid that trigger unless the workflow genuinely needs the privileged context, and never to check out untrusted code in it.</li><li><strong>Expression injection.</strong> Any attacker-controlled string, such as a branch name, issue title or commit message, that gets interpolated directly into a shell command is a code execution path. Bind it to an environment variable and reference the variable instead. This applies equally to GitLab, where the same values arrive as predefined CI variables.</li><li><strong>Self-hosted runners on public repositories.</strong> Don&#8217;t. A persistent runner that executes fork code is compromised for every subsequent job on it, including jobs that do have secrets. If you must self-host, use ephemeral runners that are destroyed after a single job, and keep public repositories on hosted runners.</li><li><strong>Mutable action and image references.</strong> Pin third-party actions to a full commit SHA. Tag repointing has been used in real supply chain compromises to reach thousands of repositories at once. The same logic applies to base images referenced by tag in GitLab jobs.</li><li><strong>Credential scope.</strong> Prefer short-lived federated credentials over long-lived secrets wherever the target supports it. GitHub and GitLab both issue OIDC tokens that cloud providers and HashiCorp Vault can exchange for short-lived access, and a token that expires in an hour is a much smaller incident than a static key that nobody rotates. Where you do use variables, mark them protected so they are only exposed on protected branches.</li><li><strong>Jenkins specifically.</strong> The controller holds the credentials store, so builds should never execute on it. Restrict agent-to-controller access, keep the plugin list as short as you can defend, and subscribe to the project&#8217;s security advisories, because plugin vulnerabilities are the most common way Jenkins installations get owned.</li></ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">A decision procedure you can run in an afternoon</h2>



<ol class="wp-block-list"><li><strong>Follow platform gravity first.</strong> Where does the code live, and where will it live in three years? CI that lives next to the code is cheaper to operate than CI that has to be integrated with it. If that single question has a clear answer, it decides most cases and the rest of this list is confirmation.</li><li><strong>Check whether a runner can reach the deploy target.</strong> Not the build. The deploy. If production sits behind a private network with no inbound path, you need self-hosted runners, a tunnel, or a system that already lives inside the perimeter. This is the requirement that most often overrides answer one.</li><li><strong>Name the on-call owner.</strong> Write down who gets paged when the CI system itself is down at 2am on a Sunday. If the honest answer is nobody, choose a hosted platform. Jenkins and self-managed GitLab both need that name to exist.</li><li><strong>Model the compute, not the sticker price.</strong> Take last month&#8217;s build minutes, split them by operating system and machine size, and apply each platform&#8217;s mechanics. macOS-heavy and Windows-heavy workloads change the answer significantly.</li><li><strong>Inventory what the plugins actually do.</strong> If you are leaving Jenkins, go through the plugin list and mark each one as replaced by a native feature, replaced by a marketplace action, needs writing, or can be dropped. The &#8220;needs writing&#8221; pile is your real migration estimate.</li><li><strong>Trial with your ugliest pipeline.</strong> Not the greenfield service. The one with the flaky integration test, the hardware dependency and the manual approval. Every platform handles a clean Node build beautifully. Only one of them will handle yours.</li></ol>



<h2 class="wp-block-heading">The hybrid pattern nobody puts on the slide</h2>



<p class="wp-block-paragraph">A very common real-world arrangement: GitHub Actions or GitLab CI runs build, test, scan and image publish, then a retained Jenkins instance handles deployment into restricted environments. It works, and for regulated organisations it is sometimes the only thing that does.</p>



<p class="wp-block-paragraph">Be honest about what it costs, though. You now have two systems to patch, two places secrets live, and a handoff between them that needs its own monitoring. It should be a deliberate architecture with a documented boundary, not the accidental result of a migration that stalled. The tell is whether anyone can draw the handoff on a whiteboard without hedging.</p>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<ul class="wp-block-list"><li><strong>&#8220;Jenkins is legacy.&#8221;</strong> It is old, which is not the same thing. It is actively maintained and still the only option for several categories of requirement. The real objection is the maintenance cost, so make that argument instead; it is stronger and it is true.</li><li><strong>&#8220;We&#8217;ll save money with self-hosted runners.&#8221;</strong> On per-minute compute, yes. Once you price the image maintenance, autoscaling and patching, the saving is much smaller than the spreadsheet suggested, and it only exists at volume.</li><li><strong>&#8220;GitLab CI is just GitHub Actions with different YAML.&#8221;</strong> The pipeline languages are comparable. The products are not. One is an event platform around a repository, the other is an integrated lifecycle platform. The difference shows up in permissions, scanning and audit, not in syntax.</li><li><strong>&#8220;We&#8217;ll stay vendor-neutral by keeping all logic in shell scripts.&#8221;</strong> Good instinct, incomplete. The scripts port. The triggers, permissions, secret injection, artifact handling and approval gates do not, and those are most of the migration work.</li><li><strong>&#8220;The bigger marketplace wins.&#8221;</strong> Marketplace size counts unreviewed third-party code you are about to execute with your credentials. It is a convenience and a supply chain liability at the same time. Treat it as both.</li></ul>



<h2 class="wp-block-heading">How I&#8217;d decide</h2>



<ul class="wp-block-list"><li>Code on GitHub, cloud deploy targets, no unusual compliance requirement: GitHub Actions, hosted runners, OIDC to the cloud provider, actions pinned to SHAs. This is the one I reach for first because it is the least infrastructure to own.</li><li>Wanting one vendor for source, CI, registry and scanning, especially with compliance pressure: GitLab, and take self-managed seriously if data residency matters.</li><li>Restricted networks, unusual hardware, or existing Jenkins that works and has an owner: keep Jenkins, but put the pipeline in a Jenkinsfile, the controller config in Configuration as Code, and the plugin list under version control. Jenkins pain is almost always click-configured Jenkins pain.</li><li>Whichever you pick, wire pipeline duration and failure rate into whatever you already use for metrics, whether that is Grafana, Datadog or something homegrown. A CI system you cannot see is one you cannot tune, and slow pipelines are a cost problem and a morale problem at the same time.</li><li>Do not migrate for aesthetics. Migrate because a specific requirement is unmet, and write that requirement down before you start.</li></ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Is Jenkins still worth using?</h3>



<p class="wp-block-paragraph">Yes, for specific situations. Air-gapped networks, source control that is not GitHub or GitLab, hardware-attached builds, and orchestration of things that are not software builds are all cases where Jenkins is the practical answer. For a standard cloud-deployed web service with code on GitHub, a hosted platform will cost you less operationally. The deciding factor is whether you have someone who owns the controller.</p>



<h3 class="wp-block-heading">Which is cheaper, GitLab CI or GitHub Actions?</h3>



<p class="wp-block-paragraph">It depends on your operating system mix and team size, because the two use different mechanics. GitHub applies multipliers so Windows and especially macOS jobs drain the included pool much faster than Linux. GitLab&#8217;s included compute pool is per namespace rather than per seat, so it does not grow as you hire. Both make self-hosted runners free of per-minute charges. Model your own last month of build minutes rather than trusting a general answer.</p>



<h3 class="wp-block-heading">Can I use GitLab CI with a GitHub repository?</h3>



<p class="wp-block-paragraph">There are mirroring and integration options, but the pipeline features are built around GitLab-hosted repositories and the experience degrades away from that. If your code is staying on GitHub, treat GitLab CI as a poor fit rather than a configuration challenge.</p>



<h3 class="wp-block-heading">How long does migrating from Jenkins to GitHub Actions or GitLab CI take?</h3>



<p class="wp-block-paragraph">Translating pipeline stages is usually days per service. The schedule is set by the parts that are not the pipeline: replacing plugin functionality, rebuilding the secret and identity model, provisioning runners with network access to deploy targets, and reproducing approval gates. Inventory those first, because that inventory is the estimate.</p>



<h3 class="wp-block-heading">Are self-hosted runners safe?</h3>



<p class="wp-block-paragraph">They are safe for private repositories with trusted contributors, provided the runner is ephemeral and destroyed after each job. They are not safe for public repositories, because a fork pull request can execute code on a persistent runner and reach everything that runs on it afterwards. Keep public repositories on hosted runners.</p>



<h3 class="wp-block-heading">Do I need to pin GitHub Actions to a commit SHA?</h3>



<p class="wp-block-paragraph">For any third-party action that runs with access to secrets, yes. Tags are mutable, and repointing a widely used tag to a malicious commit is a demonstrated attack path that has affected large numbers of repositories at once. Pinning costs you a dependency update per release and removes an entire class of compromise.</p>



<h3 class="wp-block-heading">Is it reasonable to run more than one CI system?</h3>



<p class="wp-block-paragraph">Reasonable if it is deliberate, expensive if it is accidental. A documented split where hosted CI builds and tests while an internal system deploys into restricted networks is a legitimate architecture. A split that exists because a migration stalled halfway is a permanent tax with no owner.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">Comparing <strong>GitLab CI vs GitHub Actions vs Jenkins</strong> on syntax and feature checklists produces a decision that feels rigorous and predicts almost nothing. The pipeline file is the part that ports. The runner fleet, the secret and identity model, the artifact storage and the network path to your deploy target are the parts that get rebuilt, and they are where the schedule and the risk actually live.</p>



<p class="wp-block-paragraph">Pick the platform closest to where your code lives, unless a hard requirement about network access, data residency or unusual execution environments overrides it. Then spend your effort on the layer beneath the YAML, because that is the layer you will be living with.</p>



<h2 class="wp-block-heading">Need help choosing or moving a CI pipeline?</h2>



<p class="wp-block-paragraph">This is a large part of what I do as a freelance DevOps engineer. Typical engagements around CI/CD platform decisions look like:</p>



<ul class="wp-block-list"><li>A platform evaluation with your actual build minutes, operating system mix and deploy targets modelled, so the recommendation is arithmetic rather than opinion.</li><li>Jenkins to GitHub Actions or GitLab CI migrations, including the plugin inventory, the secret and identity rebuild, and a staged cutover that does not leave you running two systems forever.</li><li>Pipeline security review: fork pull request triggers, expression injection paths, action pinning, runner isolation and token permission scoping.</li><li>Replacing long-lived cloud keys with OIDC federation on GitHub Actions or GitLab CI, including the trust policy and claim conditions on the cloud side.</li><li>Self-hosted runner fleets that autoscale, stay ephemeral, and do not quietly fill their disks.</li><li>CI cost reduction: caching, concurrency cancellation, path filters and matrix pruning, measured before and after.</li><li>Jenkins hardening and Configuration as Code, so the controller is reproducible instead of a server nobody wants to touch.</li></ul>



<p class="wp-block-paragraph">If you want a second opinion, send me the pipeline file, a plugin list or a month of build minutes and I will tell you what I would actually do with it.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/gitlab-ci-vs-github-actions-vs-jenkins/">GitLab CI vs GitHub Actions vs Jenkins: Choosing Without Regretting It Later</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Keep Kubernetes Secrets Out of Git Without Kidding Yourself</title>
		<link>https://john-nessime.com/blog/devops/keep-kubernetes-secrets-out-of-git/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sun, 09 Aug 2026 18:00:00 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Kubernetes]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Web Security]]></category>
		<category><![CDATA[Argo CD]]></category>
		<category><![CDATA[CI/CD]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[External Secrets Operator]]></category>
		<category><![CDATA[Flux CD]]></category>
		<category><![CDATA[Git]]></category>
		<category><![CDATA[GitOps]]></category>
		<category><![CDATA[HashiCorp Vault]]></category>
		<category><![CDATA[Infrastructure as Code]]></category>
		<category><![CDATA[Sealed Secrets]]></category>
		<category><![CDATA[Secrets Management]]></category>
		<category><![CDATA[SOPS]]></category>
		<category><![CDATA[Supply Chain Security]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=174</guid>

					<description><![CDATA[<p>Encrypting a Secret before you commit it only closes one of the five places that value comes to rest. Here is how to keep Kubernetes secrets out of Git properly: Sealed Secrets, SOPS and External Secrets compared honestly, the key-backup problem nobody plans for, and what to actually do when a credential is already in your history.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/keep-kubernetes-secrets-out-of-git/">Keep Kubernetes Secrets Out of Git Without Kidding Yourself</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The message usually arrives on a quiet afternoon. A secret scanner has flagged your infrastructure repo, and the finding is a Postgres connection string sitting in a <code>values.yaml</code> that someone committed a long time ago and deleted a few weeks later. The file is gone from the working tree. The commit is not.</p>



<p class="wp-block-paragraph">The reflex at that point is to reach for history rewriting. That is the wrong first move, and I will come back to why. But the bigger problem is the one that shows up six months after you clean up: teams migrate to Sealed Secrets or SOPS, tick the box, and quietly assume the whole class of problem is now solved. It is not. Encrypting a value before it enters Git closes exactly one door out of about five.</p>



<p class="wp-block-paragraph">This post covers how to <strong>keep Kubernetes secrets out of Git</strong> in a way that survives contact with a real cluster: the three families of solution and where each one genuinely wins, the disaster-recovery trap that bites hardest, how to stop plaintext reaching the repo in the first place, and what to actually do about credentials already in your history.</p>



<h2 class="wp-block-heading">The failure that actually bites: encrypted in Git, wide open in the cluster</h2>



<p class="wp-block-paragraph">Sealed Secrets and SOPS both terminate in the same place. The controller decrypts, the API server accepts, and what lands in etcd is an ordinary Kubernetes Secret. Ordinary means base64-encoded, which is an encoding, not a cipher. Anyone who can run <code>kubectl get secret -o yaml</code> in that namespace reads the value in one pipe through <code>base64 -d</code>.</p>



<p class="wp-block-paragraph">So the honest description of what these tools buy you is narrower than the marketing suggests. They stop the credential appearing in a public object store with permanent history and a CDN in front of it. They do nothing about the cluster&#8217;s own blast radius.</p>



<p class="wp-block-paragraph">Three things need to be true alongside whichever tool you pick:</p>



<ul class="wp-block-list">
<li><strong>Encryption at rest is on.</strong> A stock Kubernetes control plane writes Secrets to etcd unencrypted unless you configure an <code>EncryptionConfiguration</code>, ideally with a KMS provider. Managed distributions vary. On EKS, envelope encryption is something you enable, not something you inherit. Check yours rather than assuming.</li>

<li><strong>RBAC on secrets is actually restrictive.</strong> A depressing number of clusters grant blanket <code>get</code> and <code>list</code> on secrets to service accounts that only ever needed one. That single verb turns a namespace compromise into a credential dump.</li>

<li><strong>Secrets are mounted as files, not shoved into environment variables where you can help it.</strong> Environment variables leak through <code>/proc/&lt;pid&gt;/environ</code>, through anything that dumps the environment on crash, and through every debug endpoint an application framework has ever shipped. A projected volume with tight permissions is the quieter option.</li>
</ul>



<p class="wp-block-paragraph">Quick sanity check on the RBAC side, which takes about ten seconds and surprises people:</p>



<pre class="wp-block-code"><code># Can the default service account in a namespace read every secret in it?
kubectl auth can-i get secrets 
  --namespace production 
  --as system:serviceaccount:production:default</code></pre>



<p class="wp-block-paragraph">If that returns <code>yes</code>, your Git hygiene is not the weakest link in the chain.</p>



<h2 class="wp-block-heading">Already leaked? Rotate first, rewrite second, and understand what rewriting does not do</h2>



<p class="wp-block-paragraph">Order matters here more than technique. GitHub&#8217;s own guidance is blunt about it: once a credential has been pushed, revoke or rotate it first, because that step alone removes the attacker&#8217;s ability to use it. History surgery is the optional follow-up, not the fix.</p>



<p class="wp-block-paragraph">The reason is that a force-push does not reach everywhere the commit went. Objects survive in forks, which inherit the full object graph at fork time. They survive in colleagues&#8217; clones. On GitHub they remain reachable by SHA through cached views and through pull requests that referenced them, and purging those caches is a support request, not a git command. If the repo was public even briefly, treat the credential as burned and move on.</p>



<p class="wp-block-paragraph">When you do rewrite, use <code>git filter-repo</code>. It has replaced <code>filter-branch</code> as the recommended tool and it is dramatically faster on any repo with real history.</p>



<pre class="wp-block-code"><code># Work on a bare mirror, never your day-to-day clone.
git clone --mirror git@github.com:org/infra.git infra-clean.git
cd infra-clean.git

# Strip one path from every commit that ever touched it.
git filter-repo --path clusters/production/values.yaml --invert-paths

# filter-repo deletes the origin remote on purpose, so you cannot
# force-push a half-finished rewrite by muscle memory. Add it back.
git remote add origin git@github.com:org/infra.git
git push --force --all
git push --force --tags</code></pre>



<p class="wp-block-paragraph">Then the unglamorous part: close or merge open pull requests before you start, because every SHA downstream of the rewrite changes. Tell every collaborator to delete their clone and clone again. A <code>git pull</code> will happily drag the old objects back in.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Two families of solution, and how to tell which one you are in</h2>



<p class="wp-block-paragraph">Every approach to this problem is either <em>encrypt the value and commit the ciphertext</em>, or <em>commit a pointer and fetch the value at runtime</em>. Everything else is implementation detail. The choice is mostly about whether you already run a secret store, and whether you can tolerate the CLI step in the middle of your workflow.</p>



<h3 class="wp-block-heading">Sealed Secrets: the lowest-friction way to start</h3>



<p class="wp-block-paragraph">A controller in the cluster holds an RSA private key and publishes the matching certificate. The <code>kubeseal</code> CLI encrypts a Secret manifest against that certificate and produces a <code>SealedSecret</code> custom resource. Only that controller can decrypt it, so the resulting YAML is safe to commit.</p>



<pre class="wp-block-code"><code># Fetch the controller's public certificate. This is safe to commit;
# it can only encrypt, never decrypt.
kubeseal --fetch-cert 
  --controller-name=sealed-secrets 
  --controller-namespace=kube-system 
  &gt; pub-cert.pem

# Build a Secret manifest locally. --dry-run=client means it is rendered
# and never sent to the API server.
kubectl create secret generic db-auth 
  --from-literal=password='&lt;value&gt;' 
  --dry-run=client -o yaml &gt; db-auth.yaml

# Encrypt against the cert. Offline, so no cluster access needed here.
kubeseal --cert pub-cert.pem --format yaml &lt; db-auth.yaml &gt; db-auth-sealed.yaml

rm db-auth.yaml</code></pre>



<p class="wp-block-paragraph">Because you can hand out <code>pub-cert.pem</code>, developers can seal secrets without any cluster credentials at all. That is the underrated part, and it is the reason this is the one I reach for first on a small team with no existing vault.</p>



<p class="wp-block-paragraph">Two details that catch people. First, sealing is scoped by default: a <code>SealedSecret</code> is bound to its exact name and namespace, and moving the file to a different namespace makes it undecryptable. That is deliberate, it stops a developer sealing a secret into a namespace they should not have access to, and you can relax it with <code>--scope namespace-wide</code> or <code>--scope cluster-wide</code> if you genuinely need to. Second, the Helm chart names the controller <code>sealed-secrets</code> while the CLI looks for <code>sealed-secrets-controller</code> by default, so <code>--controller-name</code> is not optional in practice.</p>



<p class="wp-block-paragraph">Where it stops being the right answer: multi-cluster. Each cluster generates its own key pair, so a sealed file is not portable, and a fleet means either copying private keys around or maintaining per-cluster ciphertext for the same value. Rotation is also manual. There is no dynamic secret story here at all.</p>



<h3 class="wp-block-heading">SOPS with age: encrypt files, not just Kubernetes objects</h3>



<p class="wp-block-paragraph">SOPS encrypts the values inside a structured file and leaves the keys readable, so a diff still tells you which field changed even if it cannot tell you what it changed to. Paired with <code>age</code>, which uses short X25519 keys instead of the GPG keyring experience, it is pleasant to live with.</p>



<p class="wp-block-paragraph">A <code>.sops.yaml</code> at the repo root drives everything through path matching, which is how you get per-environment key separation without anyone having to remember flags:</p>



<pre class="wp-block-code"><code>creation_rules:
  - path_regex: clusters/production/.*.yaml$
    encrypted_regex: '^(data|stringData)$'
    age: age1...   # production recipient
  - path_regex: clusters/staging/.*.yaml$
    encrypted_regex: '^(data|stringData)$'
    age: age1...   # staging recipient</code></pre>



<p class="wp-block-paragraph">The <code>encrypted_regex</code> line is the one worth understanding rather than copying. Without it SOPS encrypts everything including <code>metadata</code>, and you lose the readable-diff property that made this approach attractive. Restricting it to <code>data</code> and <code>stringData</code> keeps names, namespaces and labels reviewable in a pull request.</p>



<p class="wp-block-paragraph">Flux decrypts SOPS natively: you point a <code>Kustomization</code> at a Secret holding the age private key via <code>decryption.provider: sops</code> and it handles the rest. Argo CD does not have equivalent built-in support, so you bolt it on with a config management plugin such as KSOPS or the Argo CD Vault Plugin. Worth confirming against current Argo CD docs before you commit to it, because plugin mechanics there have changed more than once.</p>



<p class="wp-block-paragraph">SOPS also handles files that are not Kubernetes manifests: Terraform variable files, Ansible vars, plain <code>.env</code>. If you have config living outside the cluster, that reach is a real advantage over Sealed Secrets.</p>



<h3 class="wp-block-heading">External Secrets Operator: stop putting the value in Git at all</h3>



<p class="wp-block-paragraph">The third approach sidesteps encryption entirely. You commit a reference, and a controller resolves it against a real secret store: HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, Google Secret Manager, or one of the newer hosted options like Doppler, Infisical or 1Password. The repo contains a path, never a value.</p>



<pre class="wp-block-code"><code>apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
  name: db-auth
  namespace: production
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: vault-backend
    kind: ClusterSecretStore
  target:
    name: db-auth
  data:
    - secretKey: password
      remoteRef:
        key: production/postgres
        property: password</code></pre>



<p class="wp-block-paragraph">This is the only one of the three that gives you rotation without a commit. Change the value in Vault, wait for <code>refreshInterval</code>, and the cluster Secret updates. It also gives you a real audit trail, because reads happen against a system built to log them.</p>



<p class="wp-block-paragraph">The cost is honest and worth stating. You have introduced a runtime dependency: if the store is unreachable, new workloads cannot get their credentials. You still have a bootstrap credential problem, because the operator needs to authenticate to the store somehow, and that root of trust has to get into the cluster by some other path. And Git is no longer the complete description of your system, which is a genuine philosophical cost if GitOps purity matters to you.</p>



<p class="wp-block-paragraph"><strong>One live upgrade trap.</strong> The <code>external-secrets.io/v1beta1</code> API was removed in ESO 0.17.0. Before that, in the 0.16 series, a conversion webhook served both versions and rewrote stored objects to <code>v1</code>, which produced permanent drift in Argo CD for anyone whose Git manifests still said <code>v1beta1</code>. Argo sees <code>v1</code> in the cluster, Git says <code>v1beta1</code>, and it reconciles forever. Update your manifests to <code>v1</code> before you cross that boundary, not after.</p>



<h3 class="wp-block-heading">Secrets Store CSI Driver: when a Secret object is one object too many</h3>



<p class="wp-block-paragraph">Worth knowing about even if you do not use it. The CSI driver mounts values from an external store directly into the pod as a volume, so no Kubernetes Secret object needs to exist at all. That closes the etcd exposure completely. It costs you the ability to consume the value as an environment variable, and it means the secret is only available to pods that mount it, which is either exactly what you want or a nuisance depending on the workload. Use it where the compliance posture demands that etcd never sees the value.</p>



<h2 class="wp-block-heading">The key you forgot to back up</h2>



<p class="wp-block-paragraph">This is the failure I would put money on, and it is the one nobody rehearses. Every encrypt-into-Git approach concentrates all your risk into one small piece of key material. Lose it and your repository turns into a directory of well-organised noise.</p>



<p class="wp-block-paragraph">Concretely: rebuild a cluster from scratch, install a fresh Sealed Secrets controller, and it generates a new key pair. Every <code>SealedSecret</code> in your repo is now undecryptable. Your GitOps repo, the one that was supposed to let you recreate everything, cannot recreate anything that needed a credential. You find this out during the incident, not before it.</p>



<ul class="wp-block-list">
<li><strong>Back up the sealing key out of band.</strong> The controller&#8217;s private key lives in a Secret in its namespace. Export it, encrypt it, and store it somewhere that is not the cluster and not the repo. A password manager or an offline copy is fine. Test the restore.</li>

<li><strong>Same for age.</strong> The private key is one line of text. Its size makes it feel unimportant. It is the entire thing.</li>

<li><strong>Add a second recipient for anything that matters.</strong> SOPS encrypts the data key once per recipient, so listing both an age key and a KMS key means either path can decrypt. The developer works with age locally, the cluster decrypts through KMS, and no single lost key is fatal.</li>

<li><strong>Know your re-key procedure before you need it.</strong> For SOPS that is <code>sops updatekeys</code> across every file matched by the rule. For Sealed Secrets it means re-sealing everything against the new certificate. Neither is hard. Both are miserable to work out under pressure.</li>
</ul>



<h2 class="wp-block-heading">Stopping plaintext from reaching the repo in the first place</h2>



<p class="wp-block-paragraph">All of the above assumes the encryption step happens. The commit that leaks is always the one where somebody skipped it, usually while debugging at the end of a long day.</p>



<ol class="wp-block-list">
<li><strong>Put a scanner in the pre-commit hook.</strong> <code>gitleaks</code> and <code>trufflehog</code> both work well here. Local hooks are bypassable, which is fine, because their job is catching accidents rather than stopping a determined person.</li>

<li><strong>Run the same scan in CI, on the whole history, not just the diff.</strong> The diff-only scan misses everything that predates the day you added the scanner.</li>

<li><strong>Turn on server-side push protection.</strong> GitHub&#8217;s push protection rejects known credential patterns at push time. GitLab and platforms like GitGuardian offer equivalents. This is the layer that actually holds, because it does not depend on anyone&#8217;s local setup.</li>

<li><strong>Make the plaintext path short-lived by construction.</strong> Never write a plaintext manifest to a tracked path. Use <code>/tmp</code>, or pipe straight into the encrypt step so the value never lands on disk at all.</li>

<li><strong>Add <code>*.dec.yaml</code>, <code>secret.yaml</code> and friends to <code>.gitignore</code> as a convention.</strong> Not a control, just one less way to be careless.</li>

<li><strong>Remember your shell history.</strong> A <code>--from-literal</code> with a real password ends up in <code>~/.bash_history</code> in cleartext. Use <code>--from-file</code>, or read from a variable, or prefix the command with a space if your shell is configured to skip those.</li>
</ol>



<h2 class="wp-block-heading">Troubleshooting the four things that go wrong</h2>



<h3 class="wp-block-heading">The SealedSecret applies cleanly but no Secret appears</h3>



<p class="wp-block-paragraph">Almost always a scope mismatch. The resource was sealed for one name or namespace and applied to another. The API server accepts the custom resource happily; the controller is where the failure surfaces.</p>



<pre class="wp-block-code"><code># The controller logs are the only place this error lives.
kubectl logs -n kube-system deploy/sealed-secrets --tail=50

# Confirm what the CRD actually thinks its name and namespace are.
kubectl get sealedsecret db-auth -n production -o yaml</code></pre>



<p class="wp-block-paragraph">The second common cause: the file was sealed against a certificate from a different cluster, or from before a key rotation. Re-fetch the cert and re-seal.</p>



<h3 class="wp-block-heading">The ExternalSecret sits there doing nothing</h3>



<p class="wp-block-paragraph">The status conditions carry the real message, and the events under <code>describe</code> usually name the provider error directly.</p>



<pre class="wp-block-code"><code>kubectl get externalsecret db-auth -n production
kubectl describe externalsecret db-auth -n production
kubectl describe clustersecretstore vault-backend</code></pre>



<p class="wp-block-paragraph">Check the store before the secret. If the <code>SecretStore</code> is not <code>Ready</code>, nothing referencing it will ever sync, and the error you want is on the store, not the <code>ExternalSecret</code>. When using a <code>ClusterSecretStore</code>, remember that any secret reference inside it needs an explicit <code>namespace</code>, because a cluster-scoped object has no namespace of its own to fall back on.</p>



<h3 class="wp-block-heading">Argo CD or Flux reports permanent drift</h3>



<p class="wp-block-paragraph">Two usual causes. Either an API version conversion is rewriting stored objects to a version your Git manifests do not use, as with the ESO <code>v1beta1</code> removal above, or a controller is adding fields to the generated Secret that your reconciler then tries to remove. Compare the live object against the rendered manifest field by field before you start changing anything.</p>



<h3 class="wp-block-heading">SOPS decryption fails only in the cluster</h3>



<p class="wp-block-paragraph">You encrypted for one recipient and the cluster holds a different key. This is the classic outcome of listing multiple recipients in <code>.sops.yaml</code> but running <code>sops --encrypt</code> with an explicit <code>--age</code> flag that overrides the rule. Decrypt locally to prove your key works, then check which recipients are actually listed in the file&#8217;s SOPS metadata block. The recipient list is stored in the file, in the clear, which makes this a fast thing to diagnose.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Rewriting history before rotating the credential, and treating the rewrite as the remediation.</li>

<li>Assuming a private repo is safe enough. Private repos get forked internally, cloned onto laptops, and made public by accident.</li>

<li>Committing the encrypted file and the plaintext source next to it, because the encrypt step wrote to a new filename and nobody deleted the original.</li>

<li>Never backing up the sealing key or the age identity.</li>

<li>Leaving encryption at rest off, so the credential moves from a Git leak to an etcd snapshot leak.</li>

<li>Encrypting the entire manifest with SOPS instead of just <code>data</code> and <code>stringData</code>, and losing every useful diff.</li>

<li>Using one age key or one sealing controller for all environments, so a staging compromise reaches production ciphertext.</li>

<li>Piping a Helm release&#8217;s rendered output into a debug artefact in CI. Helm renders secrets in the clear, and CI artefacts are frequently world-readable inside an organisation.</li>
</ul>



<h2 class="wp-block-heading">How I would actually decide</h2>



<p class="wp-block-paragraph">Skip the feature matrix and answer three questions in order.</p>



<ol class="wp-block-list">
<li><strong>Do you already run a secret store?</strong> If Vault or a cloud secret manager is already in your environment and someone owns it, use External Secrets Operator. You get rotation and audit for almost no additional conceptual load, and the argument is over.</li>

<li><strong>Do you have config outside Kubernetes?</strong> Terraform variables, Ansible vars, application <code>.env</code> files. If yes, SOPS with age, because one tool covers all of it and the per-path rules give you environment separation for free.</li>

<li><strong>Neither?</strong> Sealed Secrets. One cluster, a handful of secrets, a team that wants this solved this afternoon. It is the shortest distance to a repo with no plaintext in it, and migrating away later is straightforward because the plaintext values are recoverable from a running cluster.</li>
</ol>



<p class="wp-block-paragraph">Whichever you land on, the cluster-side work is the same and it is not optional: encryption at rest, tight RBAC on secrets, files rather than environment variables, and a tested backup of whatever key material the scheme depends on. On a managed control plane from a provider like DigitalOcean, Linode or a hyperscaler, the encryption-at-rest configuration is partly theirs and partly yours, so read the specific documentation rather than trusting a general answer. On a self-managed cluster running on your own VPS instances, for instance a k3s setup on InterServer or similar, all of it is yours and none of it is on by default.</p>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Is base64 in a Kubernetes Secret a form of encryption?</h3>



<p class="wp-block-paragraph">No. Base64 is an encoding that exists so binary values survive YAML parsing. It is reversed with a single command and provides no confidentiality whatsoever. Anyone reading the manifest reads the secret.</p>



<h3 class="wp-block-heading">Should I use Sealed Secrets or External Secrets Operator?</h3>



<p class="wp-block-paragraph">If you already run Vault, AWS Secrets Manager or an equivalent, use External Secrets Operator: you get rotation without commits and a proper audit trail. If you have no secret store and no appetite to run one, Sealed Secrets is a faster path with fewer moving parts. The trade-off is that Sealed Secrets has no rotation story and its keys are per-cluster.</p>



<h3 class="wp-block-heading">What happens if I lose the Sealed Secrets private key?</h3>



<p class="wp-block-paragraph">Every <code>SealedSecret</code> in your repository becomes permanently undecryptable, and you regenerate every credential from source. This is the single most common way this setup fails in practice. Back the key up somewhere outside the cluster and the repo, and test restoring it.</p>



<h3 class="wp-block-heading">Do I still need to rewrite Git history after rotating a leaked credential?</h3>



<p class="wp-block-paragraph">Often not. Once the credential is revoked it cannot be used, which usually resolves the actual risk. Rewriting is worth the disruption when the value is not rotatable, when it is personal data rather than a credential, or when a compliance process requires it. Weigh it against breaking every open pull request and forcing everyone to re-clone.</p>



<h3 class="wp-block-heading">Can Argo CD decrypt SOPS files on its own?</h3>



<p class="wp-block-paragraph">Not natively, unlike Flux, which has SOPS decryption built into its Kustomize controller. With Argo CD you add a config management plugin such as KSOPS or the Argo CD Vault Plugin. Check the current Argo CD documentation for the supported plugin mechanism before building around it.</p>



<h3 class="wp-block-heading">Is it safe to commit the Sealed Secrets certificate or the age public key?</h3>



<p class="wp-block-paragraph">Yes, and committing them is the point. Both are public halves of an asymmetric pair and can only encrypt. Putting them in the repo is what lets developers seal new secrets without cluster access. The private halves never go anywhere near Git.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">If you take one idea away, make it this: the work to <strong>keep Kubernetes secrets out of Git</strong> and the work to keep them safe in the cluster are two separate jobs, and finishing the first one feels a lot like finishing both. It is not. A repo full of <code>SealedSecret</code> resources sitting in front of an unencrypted etcd with permissive RBAC has moved the exposure, not removed it.</p>



<p class="wp-block-paragraph">Pick the approach that matches what you already run. Back up the key on day one, not after the first rebuild. Turn on encryption at rest. And when something does leak, rotate before you reach for <code>filter-repo</code>, because rotation is the part that actually stops the bleeding.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help sorting out secrets in your cluster?</h2>



<p class="wp-block-paragraph">This is work I do regularly, and most of it is less dramatic than it sounds once someone has done it before. Things I can help with:</p>



<ul class="wp-block-list">
<li>Auditing a GitOps repo for committed credentials, including full history, and producing a prioritised rotation list rather than a wall of scanner output</li>

<li>Setting up Sealed Secrets, SOPS with age, or External Secrets Operator against Vault or a cloud secret manager, including the key backup and restore procedure</li>

<li>Migrating an existing repo off plaintext or off a scheme that no longer fits, without a big-bang cutover</li>

<li>Fixing Argo CD or Flux reconciliation loops caused by secret controllers and API version conversions</li>

<li>Enabling and verifying etcd encryption at rest, and tightening RBAC on secrets across namespaces</li>

<li>Adding pre-commit and CI secret scanning that people will not immediately disable because of false positives</li>
</ul>



<p class="wp-block-paragraph">If you have a controller log, a stuck <code>ExternalSecret</code>, or a scanner report you are not sure how to triage, send it over and I will tell you what I make of it.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/keep-kubernetes-secrets-out-of-git/">Keep Kubernetes Secrets Out of Git Without Kidding Yourself</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Two Sources of Truth: Git-Based WordPress Deployment From Local to Production</title>
		<link>https://john-nessime.com/blog/devops/git-based-wordpress-deployment/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sat, 01 Aug 2026 09:24:28 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[WordPress]]></category>
		<category><![CDATA[Automation]]></category>
		<category><![CDATA[Bash]]></category>
		<category><![CDATA[Bedrock]]></category>
		<category><![CDATA[CI/CD]]></category>
		<category><![CDATA[Composer]]></category>
		<category><![CDATA[Deployment]]></category>
		<category><![CDATA[Git]]></category>
		<category><![CDATA[GitHub Actions]]></category>
		<category><![CDATA[rsync]]></category>
		<category><![CDATA[Staging]]></category>
		<category><![CDATA[Troubleshooting]]></category>
		<category><![CDATA[WordPress Database]]></category>
		<category><![CDATA[WordPress Hosting]]></category>
		<category><![CDATA[WordPress Migration]]></category>
		<category><![CDATA[WP-CLI]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=60</guid>

					<description><![CDATA[<p>A plugin quietly reverts after every deploy and nobody knows why. That is what happens when Git and wp-admin both think they own the file tree. Here is how to build a deployment workflow where the repo is the source of truth and the database only ever travels one way.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/git-based-wordpress-deployment/">Two Sources of Truth: Git-Based WordPress Deployment From Local to Production</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Someone reports that the contact form stopped working. You check the log, and nothing has touched that plugin in three weeks. You SSH into production, look at the plugin directory, and the version on disk is older than the version in your repo. Older than production had yesterday, in fact.</p>



<p class="wp-block-paragraph">Then it clicks. Somebody logged into wp-admin last month, saw an update notice, clicked it. The plugin got updated on the server and never in git. Last night&#8217;s deploy synced the repo over the top and put the old version back, along with the vulnerability the update was patching. No error, no warning. The deploy did exactly what it was told.</p>



<p class="wp-block-paragraph">That is the failure mode that kills most attempts at <strong>Git-based WordPress deployment</strong>, and it has nothing to do with your pipeline. WordPress is designed to modify itself. It installs plugins, updates core, writes files, all from the browser. The moment you put it under version control you have two systems that both believe they own the file tree, and they will quietly overwrite each other until somebody notices.</p>



<p class="wp-block-paragraph">This post covers the whole chain: what belongs in the repo, how to shut the second write path, how configuration differs per environment, which way the database is allowed to travel, and how to make staging safe enough that you can restore production data into it without emailing your customers by accident.</p>



<h2 class="wp-block-heading">The two things that make WordPress awkward under Git</h2>



<p class="wp-block-paragraph">Everything in this workflow follows from two facts.</p>



<p class="wp-block-paragraph"><strong>State is split across files and a database.</strong> Your theme is code. Your posts are data. But so are your plugin settings, your menus, your widget layouts, and your permalink structure, and those live in the database next to the content. Git only manages half your site, and it is not obvious which half a given change landed in.</p>



<p class="wp-block-paragraph"><strong>The two halves travel in opposite directions.</strong> Code moves forward, local to staging to production. Content moves backward, production to staging to local. Once a site is live, production is the only authoritative source of orders, comments and posts, and pushing a database upward destroys everything written since your last pull.</p>



<p class="wp-block-paragraph">Write that rule somewhere the whole team can see it. Almost every catastrophic WordPress deployment story is somebody pushing a database in the wrong direction.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Decide what actually goes in the repo</h2>



<p class="wp-block-paragraph">Three approaches, in increasing order of discipline and payoff. Pick one deliberately rather than drifting into the first.</p>



<ol class="wp-block-list">
<li><strong>Theme only.</strong> The repo contains one custom theme, sometimes a custom plugin. Everything else is installed through wp-admin. Easy to start, and it means you have no reproducible build: rebuilding the site from the repo is impossible.</li>
<li><strong>All of wp-content, minus uploads.</strong> Themes, plugins and mu-plugins in git; core and media excluded. This is the pragmatic middle and where most teams land. You get reproducible plugin versions and a real diff when a plugin changes.</li>
<li><strong>Composer-managed, Bedrock-style.</strong> Core, plugins and themes are all declared as dependencies. The repo holds a <code>composer.json</code> and your own code, nothing else. The lockfile is the source of truth and <code>composer install</code> rebuilds the site anywhere.</li>
</ol>



<p class="wp-block-paragraph">Option three is the one I reach for on anything with more than one developer, because &#8220;which plugin version is production running&#8221; becomes a question with an answer. The honest cost: premium plugins that are not on a Composer repository need either a private repository or a manual step, and that friction is real. If most of your plugin stack is commercial, option two is a reasonable place to stop.</p>



<p class="wp-block-paragraph">One thing to check if you are following an older tutorial. The Composer repository landscape for WordPress plugins shifted recently: WPackagist was acquired by WP Engine, and the Roots team launched WP Packages as an independent alternative with different package naming. Both work. Any guide written before that change will use the older <code>wpackagist-plugin/</code> prefix, so make sure the repository URL and the prefixes in your <code>composer.json</code> match each other.</p>



<h3 class="wp-block-heading">The gitignore that matters</h3>



<pre class="wp-block-code"><code># Secrets and machine-specific config. Never committed.
wp-config.php
.env

# Content, not code. This gets pulled down from production,
# it never gets deployed up.
wp-content/uploads/

# Generated at runtime or by the build.
wp-content/cache/
wp-content/upgrade/
wp-content/debug.log
node_modules/
vendor/</code></pre>



<p class="wp-block-paragraph">Ignoring <code>vendor/</code> assumes your deploy runs <code>composer install</code>. If your pipeline just rsyncs the repo with no build step, commit it instead, or you will ship a site with no dependencies. Decide which, then be consistent, because half-and-half is how you end up with a missing autoloader at two in the morning.</p>



<h2 class="wp-block-heading">Close the second write path</h2>



<p class="wp-block-paragraph">This is the step people skip, and it is the one that makes everything else work. If production can still install and update plugins from the browser, your repo is not the source of truth. It is a suggestion.</p>



<pre class="wp-block-code"><code>// wp-config.php on production and staging.

// Removes the ability to install, update or delete plugins and
// themes from the admin. Also disables the built-in file editor,
// so DISALLOW_FILE_EDIT is implied by this one.
define( 'DISALLOW_FILE_MODS', true );

// Stops core from updating itself behind your back, including
// the automatic minor updates that would otherwise change files
// your deploy is about to overwrite.
define( 'AUTOMATIC_UPDATER_DISABLED', true );</code></pre>



<p class="wp-block-paragraph">Be clear-eyed about the trade-off, because it is a real one. You have just taken away automatic security updates. That is only an improvement if you replace them with something: a scheduled dependency update job, a weekly review of the update list, a bot that opens a pull request when a plugin version changes. If you disable updates and then do nothing, you have made the site less safe, not more controlled.</p>



<p class="wp-block-paragraph">The update flow now runs through the repo. On your local machine:</p>



<pre class="wp-block-code"><code># Composer-managed: bump the lockfile, commit it, deploy.
composer update wp-plugin/some-plugin

# wp-content-in-git: let WP-CLI do the download locally,
# then commit the resulting file changes as a normal diff.
wp plugin update some-plugin</code></pre>



<p class="wp-block-paragraph">The second form is worth appreciating. A plugin update becomes a reviewable diff. When something breaks two days later, <code>git log</code> tells you exactly which files changed and <code>git revert</code> puts them back.</p>



<h2 class="wp-block-heading">Configuration per environment</h2>



<p class="wp-block-paragraph">Database credentials, salts and API keys differ per environment and none of them belong in git. Keep <code>wp-config.php</code> out of the repo and place it on each server, or use an environment file with a committed <code>.env.example</code> showing the required keys with no values.</p>



<p class="wp-block-paragraph">Core has a first-class way to tell environments apart, and it is underused:</p>



<pre class="wp-block-code"><code>// Per environment, in wp-config.php.
// Recognised values: 'local', 'development', 'staging', 'production'.
define( 'WP_ENVIRONMENT_TYPE', 'staging' );</code></pre>



<pre class="wp-block-code"><code>&lt;?php
// mu-plugin, committed to the repo, safe on every environment
// because it checks where it is running.

if ( wp_get_environment_type() !== 'production' ) {

    // Keep non-production sites out of search results.
    add_filter( 'pre_option_blog_public', '__return_zero' );

    // Short-circuit wp_mail() entirely. Returning a non-null
    // value stops WordPress before it hands anything to PHPMailer.
    add_filter( 'pre_wp_mail', '__return_false' );
}</code></pre>



<p class="wp-block-paragraph">That mail filter is the single most valuable line in this post. Restore a production database onto staging, forget this, and the first cron run mails real order confirmations, password resets and abandoned-cart nudges to real customers from a URL that does not work. It is silent until it is very loud.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Moving the database in the allowed direction</h2>



<p class="wp-block-paragraph">Set up WP-CLI aliases once and the rest becomes short commands instead of a runbook nobody follows.</p>



<pre class="wp-block-code"><code># wp-cli.yml in the project root, committed to the repo.
@staging:
  ssh: deploy@staging.example.com/var/www/staging.example.com
@production:
  ssh: deploy@example.com/var/www/example.com</code></pre>



<p class="wp-block-paragraph">Now pulling production down to your machine is one pipe. The trailing hyphen means &#8220;write the dump to standard output&#8221;, and the leading hyphen on the import means &#8220;read it from standard input&#8221;, so nothing touches disk:</p>



<pre class="wp-block-code"><code># Back up your local database first, then overwrite it.
wp db export local-backup.sql
wp @production db export - | wp db import -</code></pre>



<p class="wp-block-paragraph">The URLs inside that dump still point at production, and they are buried in serialized PHP arrays where a plain SQL find-and-replace corrupts the string length prefixes. This is exactly what <code>wp search-replace</code> exists for: it unserializes, replaces, and reserializes properly.</p>



<pre class="wp-block-code"><code># Always dry-run first and actually read the table list it prints.
wp search-replace 'https://example.com' 'https://example.test' 
  --all-tables-with-prefix --skip-columns=guid --dry-run

# Then for real.
wp search-replace 'https://example.com' 'https://example.test' 
  --all-tables-with-prefix --skip-columns=guid

wp cache flush</code></pre>



<p class="wp-block-paragraph">Two flags worth understanding rather than copying:</p>



<ul class="wp-block-list">
<li><strong><code>--skip-columns=guid</code></strong> because the <code>guid</code> column is a permanent identifier that feed readers and external systems use to decide whether they have already seen a post. Rewriting it makes subscribers re-receive your entire archive, and you find out from an angry email weeks later.</li>
<li><strong><code>--all-tables-with-prefix</code></strong> because plugins create their own tables using your prefix, and those tables are not registered with WordPress. Without this, the replacement misses them and you get half-migrated URLs in places nobody checks.</li>
</ul>



<p class="wp-block-paragraph">Uploads are the other half of the content problem. Do not put them in git; a media library will outgrow a repository quickly and every clone pays for it. Sync them separately with <code>rsync</code> when you need them, or skip them entirely locally and let a plugin proxy missing images from production. Offloading media to object storage sidesteps the question completely and is worth considering on any site with a large library.</p>



<h2 class="wp-block-heading">The deploy itself</h2>



<p class="wp-block-paragraph">Three broad options. <code>git pull</code> on the server is the simplest and the one I would move away from first: it puts a <code>.git</code> directory next to your document root, it has no build step, and a failed pull leaves the site half-updated. Rsync from CI is the pragmatic default. Atomic releases with a symlink swap, the way Deployer or Capistrano work, are the most correct because the site switches versions instantly and rollback is a symlink change.</p>



<p class="wp-block-paragraph">A workable rsync deploy from GitHub Actions, with staging on merge and production on tag:</p>



<pre class="wp-block-code"><code>name: Deploy
on:
  push:
    branches: [main]     # goes to staging
    tags: ['v*']         # goes to production

jobs:
  deploy:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    steps:
      - uses: actions/checkout@v4    # pin to a full SHA in production

      - name: Build
        run: |
          composer install --no-dev --optimize-autoloader
          npm ci
          npm run build

      - name: Sync files
        run: |
          echo "$DEPLOY_KEY" &gt; deploy_key
          chmod 600 deploy_key
          rsync -az --delete 
            -e "ssh -i deploy_key -o StrictHostKeyChecking=accept-new" 
            --exclude-from=.deployignore 
            ./ "$TARGET"
        env:
          DEPLOY_KEY: ${{ secrets.DEPLOY_KEY }}
          TARGET: ${{ secrets.DEPLOY_TARGET }}</code></pre>



<p class="wp-block-paragraph"><code>--delete</code> is what makes the target match the source exactly, which is the entire point of a deploy. It is also the flag that will erase your uploads directory if <code>.deployignore</code> is wrong. Test the exclude list against staging before you ever point this at production, and use <code>--dry-run</code> the first time.</p>



<p class="wp-block-paragraph">Files landing on the server is not the same as a working deploy. Run a post-deploy step over SSH:</p>



<pre class="wp-block-code"><code>#!/usr/bin/env bash
set -euo pipefail

cd /var/www/example.com

# Applies schema changes that a core or plugin version bump needs.
# Safe to run when there is nothing to do.
wp core update-db

# Rewrite rules live in the database, so a plugin that registers
# new routes needs this or you get 404s on the new URLs.
wp rewrite flush

# Options and transients are cached; stale ones survive a file deploy.
wp cache flush

# Prove the site answers before you call the deploy finished.
# --fail makes curl exit non-zero on a 5xx instead of shrugging.
curl --fail --silent --show-error https://example.com/ &gt; /dev/null</code></pre>



<p class="wp-block-paragraph">If you run PHP-FPM with OPcache, add a reset to that script. Otherwise PHP happily serves the previous version of your code from its compiled cache and you spend twenty minutes convinced the deploy did not run.</p>



<h2 class="wp-block-heading">Making staging safe to hold real data</h2>



<p class="wp-block-paragraph">Staging is only useful if it mirrors production, and the moment it does it becomes dangerous. Every one of these has bitten somebody:</p>



<ul class="wp-block-list">
<li><strong>Outbound email.</strong> Covered above. Kill it at the <code>wp_mail()</code> level rather than trusting a plugin setting somebody can toggle.</li>
<li><strong>Cron.</strong> A restored production database brings production&#8217;s scheduled jobs with it. Set <code>DISABLE_WP_CRON</code> on staging and do not add a system cron job unless you specifically need to test scheduling.</li>
<li><strong>Payment gateways.</strong> The database carries live API keys. Overwrite them with test-mode credentials as part of your restore script, not by hand afterwards.</li>
<li><strong>Search indexing.</strong> Filter <code>blog_public</code> as shown, and put HTTP basic auth in front of the whole environment. Duplicate content ranking above your real site is an avoidable embarrassment.</li>
<li><strong>Personal data.</strong> A production restore is real customer data on a server that probably has weaker access controls. Either anonymise user records during the restore or treat staging with the same seriousness as production.</li>
</ul>



<p class="wp-block-paragraph">Write the restore as a single script that does all of this in order, so nobody has to remember step four. If it is a checklist in a wiki, it will be skipped.</p>



<h2 class="wp-block-heading">Troubleshooting</h2>



<h3 class="wp-block-heading">The deploy ran but the site is unchanged</h3>



<p class="wp-block-paragraph">In order: OPcache serving compiled bytecode, a page cache or CDN holding the old response, or rsync excluding the directory you edited. Check them in that order. Confirm the file on disk actually changed before blaming anything upstream of it.</p>



<h3 class="wp-block-heading">White screen immediately after deploying</h3>



<p class="wp-block-paragraph">Usually a missing autoloader, meaning <code>vendor/</code> was neither committed nor built. Check the PHP error log rather than guessing. A memory limit lower on the server than locally is the other common cause, and it produces the same blank page.</p>



<h3 class="wp-block-heading">Site works, admin redirects in a loop</h3>



<p class="wp-block-paragraph"><code>siteurl</code> and <code>home</code> in the options table disagree with the URL you are actually using. Almost always a search-replace that missed. Check with <code>wp option get siteurl</code> and <code>wp option get home</code>, and set them explicitly with <code>wp option update</code> rather than running the replacement again.</p>



<h3 class="wp-block-heading">Serialized data broken after a migration</h3>



<p class="wp-block-paragraph">Somebody ran <code>sed</code> or a SQL <code>REPLACE</code> on the dump. The length prefix inside each serialized string no longer matches the string, so PHP silently fails to unserialize and widgets, theme options and ACF fields come back empty. Restore from the dump and redo it with <code>wp search-replace</code>. There is no partial recovery worth attempting.</p>



<h3 class="wp-block-heading">Permalinks return 404 after deploy</h3>



<p class="wp-block-paragraph">Rewrite rules are stored in the database and did not get regenerated. Run <code>wp rewrite flush</code>. If it recurs on every deploy, add it to the post-deploy script permanently.</p>



<h3 class="wp-block-heading">Plugin reverts to an older version after every deploy</h3>



<p class="wp-block-paragraph">Somebody is still updating in wp-admin. That is not a pipeline bug, it is the second write path being open. Set <code>DISALLOW_FILE_MODS</code> and give the team a documented way to request an update.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Committing <code>wp-config.php</code>, and with it your database password and salts.</li>
<li>Putting <code>wp-content/uploads</code> in the repo, then wondering why clones take ten minutes.</li>
<li>Pushing a staging database over production because &#8220;the content is the same&#8221;. It never is.</li>
<li>Using <code>sed</code> or SQL to swap URLs instead of <code>wp search-replace</code>.</li>
<li>Replacing the <code>guid</code> column, and re-sending your archive to every RSS subscriber.</li>
<li>Running <code>rsync --delete</code> against production without testing the exclude list.</li>
<li>Leaving updates enabled in wp-admin, so the repo and the server drift apart silently.</li>
<li>Disabling automatic updates without putting a review process in its place.</li>
<li>Restoring production data onto staging with mail still enabled.</li>
<li>Forgetting <code>DISABLE_WP_CRON</code> on staging, so scheduled jobs fire twice across two environments.</li>
<li>Treating &#8220;the files copied&#8221; as proof the deploy worked, with no smoke test.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Make the repo the only way code reaches a server, and enforce it with <code>DISALLOW_FILE_MODS</code>.</li>
<li>Keep code moving forward and content moving backward, and state that rule where the team can see it.</li>
<li>Deploy staging from a branch and production from a tag, so production releases are deliberate.</li>
<li>Dry-run every search-replace and every rsync the first time.</li>
<li>Back up the target database immediately before any deploy that touches it.</li>
<li>Script the staging restore end to end, including mail, cron, keys and anonymisation.</li>
<li>Finish every deploy with a real HTTP request through the public URL, not a file listing.</li>
<li>Use environment-aware code with <code>wp_get_environment_type()</code> instead of commented-out blocks.</li>
<li>Keep uploads out of git and sync or offload them separately.</li>
<li>Replace automatic updates with a scheduled dependency review, and put it in a calendar.</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Should WordPress core be in the repository?</h3>



<p class="wp-block-paragraph">Only if you are not managing it another way. Committing core works and makes the repo self-contained, at the cost of noisy diffs on every update. Declaring core as a Composer dependency is cleaner and gives you a pinned version without the noise. What you should not do is leave core unmanaged, so nobody can say which version production runs.</p>



<h3 class="wp-block-heading">How do I handle the database in a Git workflow?</h3>



<p class="wp-block-paragraph">You do not put it in Git. Git handles code; the database is handled by scripted pulls in one direction. Production to staging to local, with a search-replace on import. The only time a database legitimately goes upward is the initial launch, before there is real content to lose.</p>



<h3 class="wp-block-heading">What about plugin settings changed in staging?</h3>



<p class="wp-block-paragraph">This is the genuinely hard part and there is no clean answer. Settings live in the database, so they do not deploy. Options are: reapply them by hand in production with a documented checklist, script them as WP-CLI commands in a deploy step, or use a plugin that exports configuration to files. Whichever you pick, write it down, because &#8220;I&#8217;ll remember&#8221; is how a setting gets applied to staging and never production.</p>



<h3 class="wp-block-heading">Is Bedrock necessary?</h3>



<p class="wp-block-paragraph">No. It is a well-made set of defaults for Composer-managed WordPress with environment-based configuration, and it saves assembling those pieces yourself. Plenty of teams run a perfectly disciplined Git deployment with a standard directory layout. Adopt it if you want the conventions; do not adopt it as a prerequisite.</p>



<h3 class="wp-block-heading">Do I need three environments?</h3>



<p class="wp-block-paragraph">You need local and production. Staging earns its cost when you have multiple people, a client who signs off on changes, or a site where breakage costs money. On a personal blog it is overhead. On a store, skipping it means production is your test environment.</p>



<h3 class="wp-block-heading">Can I do this on shared hosting?</h3>



<p class="wp-block-paragraph">Partly. You need SSH and ideally WP-CLI, and plenty of shared plans have neither. Some managed WordPress hosts offer a git push deployment target, which handles the file half for you. If you want the full workflow with your own build steps and post-deploy hooks, a small VPS from a provider like InterServer gives you the SSH access and cron control the workflow assumes.</p>



<h3 class="wp-block-heading">How do I roll back a bad deploy?</h3>



<p class="wp-block-paragraph">Files are easy: redeploy the previous tag, or swap the symlink back if you use atomic releases. The database is the hard part, which is why you take a dump immediately before any deploy that runs a migration. Rolling files back without rolling the database back can leave you worse off than the broken version, so know which of the two actually changed before you act.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The one thing to remember</h2>



<p class="wp-block-paragraph">Git-based WordPress deployment is not really a pipeline problem. The pipeline is the easy part, and you can build a working one in an afternoon. The hard part is making the repo the only way code reaches a server, and then keeping the database flowing in exactly one direction.</p>



<p class="wp-block-paragraph">Get those two right and everything else is detail. Get them wrong and you will keep having the same confusing morning, staring at a plugin version that nobody remembers changing.</p>



<h2 class="wp-block-heading">Want this set up properly on your site?</h2>



<p class="wp-block-paragraph">Retrofitting version control onto a live WordPress site is fiddly work, mostly because you have to do it without downtime and without losing whatever has been changed directly on the server. Things I take on:</p>



<ul class="wp-block-list">
<li>Moving an existing production site into Git without downtime, including reconciling whatever has drifted on the server.</li>
<li>Converting a site to Composer-managed plugins and core, with a working lockfile and a sane update process.</li>
<li>Building local, staging and production environments that actually match, with per-environment configuration.</li>
<li>Deploy pipelines in GitHub Actions or GitLab CI: build, rsync or atomic releases, post-deploy WP-CLI steps, smoke test, rollback path.</li>
<li>Scripted database refreshes from production to staging with mail, cron, API keys and personal data handled automatically.</li>
<li>Server setup for the workflow: SSH deploy users, correct file ownership, OPcache resets, cron.</li>
</ul>



<p class="wp-block-paragraph">Tell me how your site is hosted and how plugins currently get updated, and I will tell you what the first step should be.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/git-based-wordpress-deployment/">Two Sources of Truth: Git-Based WordPress Deployment From Local to Production</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Your Deploy Key Is a Root Shell: GitHub Actions VPS Deployment Over SSH, Done Carefully</title>
		<link>https://john-nessime.com/blog/devops/github-actions-vps-deployment/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sat, 01 Aug 2026 09:24:21 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Web Security]]></category>
		<category><![CDATA[Automation]]></category>
		<category><![CDATA[Bash]]></category>
		<category><![CDATA[CI/CD]]></category>
		<category><![CDATA[Deployment]]></category>
		<category><![CDATA[Git]]></category>
		<category><![CDATA[GitHub Actions]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Linux]]></category>
		<category><![CDATA[OpenSSH]]></category>
		<category><![CDATA[Production]]></category>
		<category><![CDATA[rsync]]></category>
		<category><![CDATA[SSH]]></category>
		<category><![CDATA[Supply Chain Security]]></category>
		<category><![CDATA[Sysadmin]]></category>
		<category><![CDATA[Troubleshooting]]></category>
		<category><![CDATA[VPS]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=66</guid>

					<description><![CDATA[<p>A deploy key in GitHub secrets is a shell on your production server, handed to a container you don't control, running code from maintainers you've never met. Here's how to build the pipeline so a leaked key isn't worth much: forced commands, scoped sudo, pinned actions and a real approval gate.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/github-actions-vps-deployment/">Your Deploy Key Is a Root Shell: GitHub Actions VPS Deployment Over SSH, Done Carefully</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Stop and look at what you are about to build. A private SSH key that can log into your production server, pasted into a text box on github.com, handed to a container you do not control, which then runs code from repositories maintained by people you have never met.</p>



<p class="wp-block-paragraph">That is not hypothetical caution. In March 2025 a widely used third-party action was compromised: the attacker got hold of a maintainer&#8217;s token, pushed a malicious commit, and then <em>retroactively repointed every version tag</em> at it. Repositories that referenced the action by tag, which is nearly all of them, picked up the payload automatically. It dumped the runner&#8217;s secrets into the build log. On public repositories, those logs are readable by anyone.</p>



<p class="wp-block-paragraph">Nobody&#8217;s workflow file changed. Nobody clicked anything. The pipeline just quietly handed out its credentials one Friday.</p>



<p class="wp-block-paragraph">A <strong>GitHub Actions VPS deployment</strong> is fundamentally a credential handoff, and most guides on the topic skip straight to the YAML. This post covers the YAML too, but the useful part is what surrounds it: how to narrow the key so it cannot do much, how to verify you are talking to the right server, how to stop an unreviewed commit reaching production, and which of the failures here are silent.</p>



<h2 class="wp-block-heading">Who can actually reach that key</h2>



<p class="wp-block-paragraph">Worth being explicit, because it is a longer list than people expect:</p>



<ul class="wp-block-list">
<li><strong>Anyone with write access to the repository.</strong> A workflow file is code. Push a branch with a step that prints the key somewhere, trigger it, done. Write access to the repo is effectively access to the server.</li>
<li><strong>Every action in the job.</strong> Secrets are exposed to the whole job, not just the step that references them. A compromised action anywhere in that job can read the environment.</li>
<li><strong>Anyone who can influence what runs.</strong> This is why <code>pull_request_target</code> combined with checking out the pull request&#8217;s own code is such a well-known foot-gun: it runs the fork&#8217;s code with the base repository&#8217;s secrets in scope.</li>
</ul>



<p class="wp-block-paragraph">The goal is not to make the key unreachable. It is to make a stolen key not worth much.</p>



<h2 class="wp-block-heading">Decide how the runner authenticates</h2>



<p class="wp-block-paragraph">Three options, and the right one depends on what you already run.</p>



<ol class="wp-block-list">
<li><strong>A dedicated SSH key in repository secrets.</strong> The default. Simple, works everywhere, and it is a long-lived credential sitting in a place several people can reach. Acceptable when the key is narrowed properly, which is most of the rest of this post.</li>
<li><strong>An overlay network.</strong> Put the runner and the server on a private network with Tailscale or WireGuard for the duration of the job, using an ephemeral auth key, and close SSH to the public internet entirely. More moving parts, and it removes port 22 from the internet rather than defending it. If you already run an overlay for other reasons, this is clearly better.</li>
<li><strong>A self-hosted runner on the server.</strong> No inbound SSH at all, since the runner polls outward. The trade-off is real and often understated: you have just installed a general-purpose code execution service on your production box, and for a public repository that is close to indefensible. On a private repository with a trusted team it is reasonable.</li>
</ol>



<p class="wp-block-paragraph">One thing that sounds appealing and is not practical: allowlisting GitHub&#8217;s runner IP ranges at your firewall. The ranges are published through GitHub&#8217;s meta API, but they are enormous and they change, and every other GitHub customer&#8217;s job runs from inside them. It buys you very little for a lot of maintenance.</p>



<h2 class="wp-block-heading">Narrow the key until it is boring</h2>



<p class="wp-block-paragraph">Assume the key leaks. What can the holder do? Work backwards from there.</p>



<p class="wp-block-paragraph"><strong>Give it its own user.</strong> A <code>deploy</code> account that owns the application directory and nothing else. Not your admin account, not root, and definitely not a key you also use from your laptop.</p>



<p class="wp-block-paragraph"><strong>Restrict the key in <code>authorized_keys</code>.</strong> SSH lets you attach options to an individual key, and they apply regardless of what the client asks for:</p>



<pre class="wp-block-code"><code># ~deploy/.ssh/authorized_keys
#
# restrict = no port forwarding, no agent forwarding, no X11,
#            no PTY, no user rc file. Deny everything, then
#            re-enable only what you need.
# command  = ignore whatever the client asked to run and run
#            this instead. The client's command is available
#            to the script as $SSH_ORIGINAL_COMMAND.

restrict,command="/usr/local/bin/deploy-guard" ssh-ed25519 AAAA...  deploy@github-actions</code></pre>



<p class="wp-block-paragraph">The guard script decides what the key is allowed to do. Keep it short enough to read in one go:</p>



<pre class="wp-block-code"><code>#!/usr/bin/env bash
# /usr/local/bin/deploy-guard
set -euo pipefail

case "${SSH_ORIGINAL_COMMAND:-}" in
  # rsync sends its own server-side invocation. Match it loosely
  # enough to work, tightly enough to pin the destination.
  "rsync --server "*" /var/www/app/")
    exec $SSH_ORIGINAL_COMMAND
    ;;
  "deploy:reload")
    exec sudo /bin/systemctl reload nginx
    ;;
  *)
    echo "refused: $SSH_ORIGINAL_COMMAND" &gt;&amp;2
    exit 1
    ;;
esac</code></pre>



<p class="wp-block-paragraph">Now a stolen key can write to one directory and reload one service. It cannot open a shell, cannot forward a port into your private network, and cannot read anything else on the box.</p>



<p class="wp-block-paragraph"><strong>Scope the sudo entry too.</strong> The deploy user needs to reload a service, not run arbitrary commands as root:</p>



<pre class="wp-block-code"><code># visudo -f /etc/sudoers.d/deploy
# One command, no password, no wildcards. A wildcard here
# often turns back into a general-purpose root shell.
deploy ALL=(root) NOPASSWD: /bin/systemctl reload nginx</code></pre>



<p class="wp-block-paragraph">Be honest about the cost: the guard script is one more thing to maintain, and the day you add a new deploy step it will refuse it and you will spend ten minutes confused. That is the trade you are making, and on anything handling customer data it is worth it. On a hobby project, a plain unrestricted <code>deploy</code> user with no sudo is a reasonable stopping point.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Verify the server, not just the key</h2>



<p class="wp-block-paragraph">Almost every tutorial on this topic contains <code>StrictHostKeyChecking=no</code>, which turns off the check that stops you handing your credentials to whatever machine answered. It is there because the alternative is mildly annoying, and the alternative takes one command.</p>



<pre class="wp-block-code"><code># Run this once, from a machine you trust, and compare the
# fingerprint against what the server reports locally.
ssh-keyscan -t ed25519 example.com</code></pre>



<p class="wp-block-paragraph">Store that output as a repository variable, not a secret. It is public information, and keeping it visible means you can actually see it when it changes, which matters because it will change the first time you rebuild the server.</p>



<h2 class="wp-block-heading">The workflow</h2>



<pre class="wp-block-code"><code>name: Deploy

on:
  push:
    tags: ['v*']          # production ships from tags, not branches

# This job reads code and talks to a server. It has no business
# writing to the repository, so take that away.
permissions:
  contents: read

concurrency:
  group: deploy-production
  # Deliberately NOT cancel-in-progress. Killing a deploy halfway
  # leaves the server in a state nobody designed.
  cancel-in-progress: false

jobs:
  deploy:
    runs-on: ubuntu-latest
    timeout-minutes: 15
    environment:
      name: production      # scoped secrets and required reviewers live here
      url: https://example.com

    steps:
      # Pin every action to a full 40-character commit SHA, with the
      # version in a trailing comment so Dependabot can still bump it.
      # Get the SHA from the action's releases page.
      - uses: actions/checkout@PUT_THE_FULL_COMMIT_SHA_HERE  # pinned

      - name: Build
        run: |
          npm ci
          npm run build

      - name: Configure SSH
        # Pass secrets through env, never interpolate ${{ }} directly
        # into a run block. Interpolation happens before the shell
        # sees the script, so a value with the wrong characters
        # either breaks the script or becomes part of it.
        env:
          DEPLOY_KEY: ${{ secrets.DEPLOY_SSH_KEY }}
          KNOWN_HOSTS: ${{ vars.DEPLOY_KNOWN_HOSTS }}
        run: |
          install -m 700 -d ~/.ssh
          printf '%sn' "$DEPLOY_KEY" &gt; ~/.ssh/id_ed25519
          chmod 600 ~/.ssh/id_ed25519
          printf '%sn' "$KNOWN_HOSTS" &gt; ~/.ssh/known_hosts

      - name: Ship
        run: |
          rsync -az --delete --exclude-from=.deployignore 
            -e "ssh -o BatchMode=yes" 
            ./dist/ deploy@example.com:/var/www/app/

      - name: Reload and verify
        run: |
          ssh -o BatchMode=yes deploy@example.com 'deploy:reload'
          curl --fail --silent --show-error https://example.com/healthz &gt; /dev/null</code></pre>



<p class="wp-block-paragraph">Three details in there worth calling out, because they are the ones people leave out.</p>



<p class="wp-block-paragraph"><code>-o BatchMode=yes</code> stops SSH prompting for anything. Without it, a missing key or an unknown host produces a prompt that nobody is there to answer, and the job hangs until the timeout instead of failing in three seconds.</p>



<p class="wp-block-paragraph"><code>--delete</code> is what makes the target match the source, which is the point of a deploy, and also what will erase your uploads directory if <code>.deployignore</code> is wrong. Run it once with <code>--dry-run</code> against staging and read the list before you point it at production.</p>



<p class="wp-block-paragraph">The final <code>curl --fail</code> is the difference between &#8220;the files copied&#8221; and &#8220;the site works&#8221;. Files landing on disk is not a successful deploy, and a job that goes green while the site returns 502 is worse than one that fails.</p>



<h2 class="wp-block-heading">Third-party actions</h2>



<p class="wp-block-paragraph">There are convenient actions that wrap SSH deployment, and they save maybe eight lines. In the one job on your account that holds a production credential, eight lines is a bad trade for another maintainer in your trust chain. The workflow above uses plain <code>ssh</code> and <code>rsync</code> for that reason.</p>



<p class="wp-block-paragraph">Where you do use third-party actions, pin them to a full commit SHA. A tag is a pointer and pointers can be moved, which is precisely what happened in the incident at the top of this post. A SHA is content-addressed: it cannot be repointed at different code.</p>



<p class="wp-block-paragraph">Two things make this sustainable rather than miserable. Dependabot understands SHA pins with a version comment and will open pull requests to bump them. And you can restrict which actions are allowed to run at all, in the repository or organisation settings, which is worth doing once and forgetting about.</p>



<h2 class="wp-block-heading">Gate production properly</h2>



<p class="wp-block-paragraph">The <code>environment:</code> key in that workflow is doing more work than it appears to. Environments give you three things you cannot get from repository secrets alone:</p>



<ul class="wp-block-list">
<li><strong>Scoped secrets.</strong> The production key is only available to jobs that declare the production environment. A branch that adds a job without it gets nothing.</li>
<li><strong>Required reviewers.</strong> The job pauses and waits for a named human to approve it. This is the single control that turns &#8220;anyone with write access can deploy&#8221; into &#8220;anyone with write access can request a deploy&#8221;.</li>
<li><strong>Deployment branch and tag rules.</strong> Restrict the environment so only protected tags or a specific branch can use it, so a feature branch cannot reach production even with the right job definition.</li>
</ul>



<p class="wp-block-paragraph">Deploying from tags rather than branch pushes is worth the small friction. It makes a release a deliberate act, and it gives you an obvious rollback: redeploy the previous tag.</p>



<p class="wp-block-paragraph">For anything with real traffic, put the files somewhere new and switch atomically rather than rsyncing over a live directory. Copy into a timestamped release directory, then move a symlink. The site changes version instantly instead of spending several seconds as a mix of old and new files, and rollback becomes a symlink change rather than a redeploy.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting</h2>



<h3 class="wp-block-heading">Permission denied (publickey)</h3>



<p class="wp-block-paragraph">The most common cause by far is a missing trailing newline. Private keys must end with one, and pasting into a web form frequently drops it. That is why the workflow uses <code>printf '%sn'</code> rather than <code>echo</code>: it guarantees the newline regardless of what got stored.</p>



<p class="wp-block-paragraph">After that, check permissions on the server: <code>~/.ssh</code> must be 700 and <code>authorized_keys</code> 600, owned by the deploy user, and the home directory must not be group-writable. Add <code>-vvv</code> to the ssh command to see how far the handshake gets, and read <code>journalctl -u ssh</code> on the server for the reason, since the client is deliberately vague.</p>



<h3 class="wp-block-heading">Host key verification failed</h3>



<p class="wp-block-paragraph">Either the <code>known_hosts</code> variable is empty, or the server&#8217;s host key genuinely changed. If you rebuilt the server, regenerate it with <code>ssh-keyscan</code> and update the variable. If you did not rebuild the server, stop and find out why the key changed before you update anything.</p>



<h3 class="wp-block-heading">The job hangs until the timeout</h3>



<p class="wp-block-paragraph">SSH is waiting on a prompt nobody will answer. Add <code>-o BatchMode=yes</code> and it fails immediately with a readable reason instead.</p>



<h3 class="wp-block-heading">rsync works from my laptop, fails from the runner</h3>



<p class="wp-block-paragraph">If you set a forced command, it is rejecting the invocation. The refusal message goes to stderr on the server side, so add a log line to the guard script that records <code>$SSH_ORIGINAL_COMMAND</code>, run the job once, and read what rsync actually sent.</p>



<h3 class="wp-block-heading">Works on push, fails on pull requests</h3>



<p class="wp-block-paragraph">Working as intended. Secrets are not exposed to workflows triggered by pull requests from forks. Do not reach for <code>pull_request_target</code> to work around it. Split the workflow so that pull requests build and test without secrets, and only pushes or tags deploy.</p>



<h3 class="wp-block-heading">A secret appeared in the logs</h3>



<p class="wp-block-paragraph">Masking matches the exact stored string. Transform it in any way and the mask stops working: base64-encoded, JSON-escaped, split across lines, or with whitespace trimmed. Do not treat masking as a control. Rotate the key, delete the run logs, and remove whatever produced the output.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Using an existing personal SSH key instead of generating a dedicated one for the runner.</li>
<li>Giving the deploy user full sudo, or a sudo rule with a wildcard in it.</li>
<li>Disabling host key checking with <code>StrictHostKeyChecking=no</code>.</li>
<li>Referencing third-party actions by tag in the job that holds your production key.</li>
<li>Interpolating <code>${{ secrets.X }}</code> directly into a <code>run:</code> script instead of passing it through <code>env:</code>.</li>
<li>Leaving default write permissions on a job that only needs to read.</li>
<li>Using <code>cancel-in-progress: true</code> on a deploy, so a second push interrupts the first mid-copy.</li>
<li>Running <code>rsync --delete</code> at production without testing the exclude list.</li>
<li>Treating &#8220;the files copied&#8221; as a successful deploy, with no health check.</li>
<li>Deploying straight from branch pushes with no approval step.</li>
<li>Relying on log masking to keep a leaked secret out of the output.</li>
<li>Never rotating the deploy key, including after someone leaves the team.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Dedicated deploy user, dedicated key, used for nothing else.</li>
<li>Restrict the key in <code>authorized_keys</code> with <code>restrict</code> and a forced command.</li>
<li>Scope sudo to the exact commands the deploy needs.</li>
<li>Pin every action to a full commit SHA and let Dependabot bump them.</li>
<li>Keep the deploy job small: fewer steps means fewer things that can read the secret.</li>
<li>Set <code>permissions:</code> explicitly, defaulting to read-only.</li>
<li>Verify host keys from a stored <code>known_hosts</code> value.</li>
<li>Use environments with required reviewers and deployment rules for production.</li>
<li>Deploy from tags, and make rollback a redeploy of the previous tag or a symlink swap.</li>
<li>End every deploy with a real HTTP request against the public URL.</li>
<li>Rotate the deploy key on a schedule and whenever someone leaves.</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Is it safe to store an SSH private key in GitHub secrets?</h3>



<p class="wp-block-paragraph">Safe enough, if the key is worth little on its own. Secrets are encrypted at rest and masked in logs, but they are readable by any job that references them and by anyone who can modify a workflow. Assume it can leak and make sure a leaked key only grants write access to one directory and one service reload.</p>



<h3 class="wp-block-heading">Should I use a ready-made SSH deploy action?</h3>



<p class="wp-block-paragraph">They work and they are convenient. My preference is plain <code>ssh</code> and <code>rsync</code> in the job that touches production, because it removes a maintainer from the trust chain for very little added effort. If you do use one, pin it to a SHA like anything else.</p>



<h3 class="wp-block-heading">Can I use OIDC instead of a stored key?</h3>



<p class="wp-block-paragraph">Not directly against sshd, which has no concept of an OIDC token. OIDC removes long-lived secrets when the target is a cloud provider that can verify GitHub&#8217;s token. To get the same benefit for a plain VPS you need something in between that trades the token for short-lived access: a secrets manager, a bastion that supports it, or an overlay network with ephemeral auth keys.</p>



<h3 class="wp-block-heading">Should I allowlist GitHub&#8217;s IP ranges on my firewall?</h3>



<p class="wp-block-paragraph">Generally not worth it. The published ranges are large and change over time, so you are signing up for ongoing maintenance, and every other GitHub customer&#8217;s runners share those ranges. Narrowing what the key can do achieves more.</p>



<h3 class="wp-block-heading">Self-hosted runner or hosted runner?</h3>



<p class="wp-block-paragraph">Hosted for public repositories, without exception, because a self-hosted runner will execute code from pull requests. Self-hosted is reasonable on a private repository with a trusted team, and it removes inbound SSH entirely. Just be clear that you have moved the risk rather than removed it.</p>



<h3 class="wp-block-heading">How do I roll back a bad deploy?</h3>



<p class="wp-block-paragraph">Redeploy the previous tag, or swap the symlink back if you use atomic releases. The part that does not roll back is the database, so if a release ran a migration, know what it did before you revert the files. Files going backwards while the schema stays forwards is often worse than the bug you were fixing.</p>



<h3 class="wp-block-heading">How often should the deploy key be rotated?</h3>



<p class="wp-block-paragraph">On a schedule you will actually keep, and immediately whenever someone with repository access leaves or a run log looks suspicious. Rotation is two steps, adding the new public key to <code>authorized_keys</code> and updating the secret, so there is no good reason for it to be a project.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The one thing to remember</h2>



<p class="wp-block-paragraph">The YAML is the easy half. A working GitHub Actions VPS deployment takes an afternoon; the part that decides whether it is a good idea is what that key can do once it is out of your hands, and it will eventually be out of your hands.</p>



<p class="wp-block-paragraph">So build it assuming the key leaks. Dedicated user, forced command, scoped sudo, pinned actions, an approval gate on production. Then the worst case is an inconvenient rotation rather than an incident, and you will not be reading a build log at midnight trying to work out what a stranger&#8217;s code did with your credentials.</p>



<h2 class="wp-block-heading">Want this built or reviewed?</h2>



<p class="wp-block-paragraph">Most deploy pipelines I get asked to look at work fine and hand out far more access than they need. Work I take on:</p>



<ul class="wp-block-list">
<li>Building a GitHub Actions deploy to a VPS end to end: build, ship, reload, health check, rollback path.</li>
<li>Auditing an existing pipeline for what a leaked runner secret would actually grant, and closing the gap.</li>
<li>Locking down the server side: deploy user, forced commands, scoped sudo, atomic release directories.</li>
<li>Moving SSH off the public internet onto an overlay network, or setting up a self-hosted runner safely.</li>
<li>Pinning and auditing third-party actions, plus repository settings that limit what can run.</li>
<li>Environments, required reviewers and tag-based releases for teams that currently deploy on every push to main.</li>
</ul>



<p class="wp-block-paragraph">Send me your workflow file and the output of <code>ssh deploy@yourhost 'id'</code>, and I will tell you what a stolen key would get.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/github-actions-vps-deployment/">Your Deploy Key Is a Root Shell: GitHub Actions VPS Deployment Over SSH, Done Carefully</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
