<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>n8n | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/n8n/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/n8n/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Sun, 20 Sep 2026 16:57:45 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>n8n | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/n8n/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Six Signs GoHighLevel Needs a Developer, Not Another Workflow</title>
		<link>https://john-nessime.com/blog/workflow-automation/gohighlevel-custom-development-vs-workflows/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 17 Sep 2026 06:00:00 +0000</pubDate>
				<category><![CDATA[Enterprise Integration]]></category>
		<category><![CDATA[SaaS Engineering]]></category>
		<category><![CDATA[Workflow Automation]]></category>
		<category><![CDATA[Architecture Debt]]></category>
		<category><![CDATA[CRM Audit]]></category>
		<category><![CDATA[Error Handling]]></category>
		<category><![CDATA[Execution Logs]]></category>
		<category><![CDATA[Exponential Backoff]]></category>
		<category><![CDATA[GoHighLevel]]></category>
		<category><![CDATA[Idempotency]]></category>
		<category><![CDATA[iPaaS]]></category>
		<category><![CDATA[Marketing Automation]]></category>
		<category><![CDATA[n8n]]></category>
		<category><![CDATA[OAuth]]></category>
		<category><![CDATA[Over-Engineering]]></category>
		<category><![CDATA[Private Integration]]></category>
		<category><![CDATA[Project Scoping]]></category>
		<category><![CDATA[Race Conditions]]></category>
		<category><![CDATA[Rate Limiting]]></category>
		<category><![CDATA[Reconciliation]]></category>
		<category><![CDATA[REST API]]></category>
		<category><![CDATA[Sub-Account Management]]></category>
		<category><![CDATA[Webhooks]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=647</guid>

					<description><![CDATA[<p>Adding another workflow is usually the cheap answer, until it isn't. Six signals that a GoHighLevel account has outgrown the builder, the honest case for staying in it, and the three levels of code to try before committing to a marketplace app.</p>
<p>The post <a href="https://john-nessime.com/blog/workflow-automation/gohighlevel-custom-development-vs-workflows/">Six Signs GoHighLevel Needs a Developer, Not Another Workflow</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The account had forty-one workflows. Two were named &#8220;Lead Follow-Up v3&#8221; and &#8220;Lead Follow-Up v3 (USE THIS ONE)&#8221;. When someone asked what happens to a contact who submits the form twice in one afternoon, nobody could answer. Not the agency owner, not the person who built it.</p>



<p class="wp-block-paragraph">That is the moment. Not a crash, not an error banner. A plain question about the system that nobody can answer from the canvas.</p>



<p class="wp-block-paragraph">GoHighLevel is good at what it was built for and most accounts never need anything else. But there is a point where the next workflow makes the account worse instead of better, and past it, <strong>GoHighLevel custom development</strong> costs less than another six months of patching. Below: the failure mode hiding under all the others, six signals you have crossed that line, the honest case for staying in the builder, and three levels of code to try before committing to a full app.</p>



<h2 class="wp-block-heading">The failure hiding under all the others</h2>



<p class="wp-block-paragraph">The workflow builder has no failure semantics. Everything below is a consequence of that.</p>



<p class="wp-block-paragraph">When a step works, the contact moves on. When a step fails, the contact usually also moves on. No transaction, no rollback, no retry budget you control, no way to say &#8220;this contact is in a bad state, stop and tell someone&#8221;. A Custom Webhook that gets a 500 back does not park the contact for review. It becomes a line in an execution log nobody reads, and the contact continues as though the other system had answered.</p>



<p class="wp-block-paragraph">Fine when the step sends a text message. Not fine when the step is the only thing keeping two systems in agreement. So the useful test is not &#8220;is this hard to build in the builder&#8221;, because plenty of hard things build fine. The test is: <em>what happens when this step fails, and would I find out?</em></p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Signal one: you are storing state in tags</h2>



<p class="wp-block-paragraph">Tags start as labels. Then someone needs to know whether a lead was called, so there is a <code>called</code> tag. Then whether they were called twice, so <code>called-2</code>. Then a workflow that removes <code>called</code> when a new form arrives, and another that puts it back if an appointment was booked first.</p>



<p class="wp-block-paragraph">Those are no longer labels. That is a state machine with no diagram, no validation, and no guarantee that only one workflow writes to it at a time. Two workflows fired by the same event will both read the old value and both write, and whichever finishes last wins. That is a race condition, and it hides during testing because you test one contact at a time. The tell is a tag name with a number in it, or a tag whose meaning needs the word &#8220;unless&#8221; to explain.</p>



<h2 class="wp-block-heading">Signal two: the data has to be right, not eventually right</h2>



<p class="wp-block-paragraph">Marketing automation tolerates drift. One extra nurture email hurts nobody. The moment GoHighLevel holds something that must reconcile with another system, that tolerance is gone. Order status from a store, invoice state from an accounting package, availability shared with a clinic system.</p>



<p class="wp-block-paragraph">Those need idempotency, meaning a repeated message produces the same result as a single one, and reconciliation, meaning something periodically compares both sides and reports the difference. The builder gives you neither: no natural home for an idempotency key, no scheduled job that reads both systems and complains. You can fake reconciliation with a workflow on a timer, but that is a batch job written in a UI designed for follow-up sequences, and the first time it half-runs you will not hear about it.</p>



<h2 class="wp-block-heading">Signal three: per-execution billing became a per-lead tax</h2>



<p class="wp-block-paragraph">The actions that make workflows powerful are premium actions, billed per execution against the agency wallet. Custom Webhook, Inbound Webhook, Custom Code and the workflow AI actions all sit in that category. HighLevel gives each sub-account a small free allowance when premium actions are first switched on, publishes volume tiers above that, and lets you rebill sub-accounts individually instead of absorbing the charge. Take the current rate and tiers from HighLevel&#8217;s billing documentation rather than any blog post, including this one.</p>



<p class="wp-block-paragraph">The mechanism is the point. You pay per execution, not per useful outcome. A workflow calling out three times to enrich, score and push a lead is three executions per lead. Multiply by volume and sub-accounts, and the plumbing line can pass the cost of a small service doing all three in one request.</p>



<p class="wp-block-paragraph">Two caveats. Consolidating three calls into one endpoint is usually cheaper than moving the whole thing to code. And whatever you move still costs something to run, whether that is a VPS from a provider like Contabo or InterServer, a hosted platform such as Make or n8n, or a serverless function. Compare both sides.</p>



<h2 class="wp-block-heading">Signal four: the same logic lives in twenty sub-accounts</h2>



<p class="wp-block-paragraph">Snapshots are the platform&#8217;s answer and better than people expect. You can refresh a snapshot and push selected assets to sub-accounts that previously imported it, choosing both the assets and the recipients, and the push is selective rather than destructive.</p>



<p class="wp-block-paragraph">Two limits. Push Update only reaches sub-accounts inside your own agency, so anything shared externally has to be re-imported through a new link. And HighLevel documents that when a snapshot refresh deletes steps, contacts waiting on those steps in workflows created from that snapshot are removed. Fine for a copy change, destructive for a structural one.</p>



<p class="wp-block-paragraph">Snapshots distribute structure. They do not give you versioning, a rollback path, a diff between intended and running state, or a test that fails before the change reaches a client. Once a change has to land correctly in twenty accounts on one day, you want those, and they come from putting the logic behind an API you control and calling it from a thin step in each account.</p>



<h2 class="wp-block-heading">Signal five: the integration must survive the other side being down</h2>



<p class="wp-block-paragraph">A Custom Webhook fires the moment the contact reaches the step. If your endpoint is slow, restarting or rate limited, that single firing is what you get. No queue in front, no backoff behind.</p>



<p class="wp-block-paragraph">The API side has numbers worth knowing. HighLevel documents a burst limit of 100 requests per 10 seconds and a daily limit of 200,000 requests, counted per marketplace app per resource, where a resource is a location or the company. Every response carries headers telling you where you stand, so backoff can follow what the server says instead of a guessed sleep.</p>



<pre class="wp-block-code"><code>curl -sS -D - -o /dev/null 
  -H "Authorization: Bearer $GHL_TOKEN" 
  -H "Version: $GHL_API_VERSION" 
  "$GHL_API_BASE/contacts/" 
  | grep -i '^x-ratelimit'</code></pre>



<p class="wp-block-paragraph"><code>-D -</code> writes response headers to stdout while <code>-o /dev/null</code> discards the body, so you read the rate limit state without dumping contact data into your shell history. Set the base URL and the required <code>Version</code> value from the current API reference. Watch <code>X-RateLimit-Max</code> and <code>X-RateLimit-Remaining</code> for the burst window, <code>X-RateLimit-Interval-Milliseconds</code> for its length, and <code>X-RateLimit-Limit-Daily</code> with <code>X-RateLimit-Daily-Remaining</code> for the day.</p>



<p class="wp-block-paragraph">The daily budget sounds enormous until a migration. Reading a contact, upserting it, then writing custom fields is three calls each, so forty thousand contacts is 120,000 calls before live traffic touches the account. Batch the reads, skip writes where nothing changed, run backfills overnight. Watch for loops too: writes made through the API can themselves fire workflow triggers, so a receiver that writes back has built a cycle.</p>



<h2 class="wp-block-heading">Signal six: someone needs a screen, not a form</h2>



<p class="wp-block-paragraph">Forms collect. They do not display. Once a client asks to see a filtered view of their own data, approve something in two clicks, or work a queue that is not a pipeline, you are looking at an interface.</p>



<p class="wp-block-paragraph">The platform supports this. Marketplace apps can add Custom Pages, which render inside HighLevel in an embedded iframe loading a URL you host, appearing on the app details page or in the left navigation. HighLevel publishes an app template to start from, plus an SSO mechanism so your page can identify the user instead of asking them to log in again. That is still a real application with hosting, auth and deployment: the biggest step on this list, the one I would defer longest, and the only honest answer when the requirement is a screen.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Where the workflow builder genuinely wins</h2>



<p class="wp-block-paragraph">A one-sided argument is not worth reading, so here is the case against writing code.</p>



<ul class="wp-block-list">
<li><strong>The person who owns the outcome can change it.</strong> An account manager adjusts send timing on a Friday with no deploy and no developer awake.</li>



<li><strong>Messaging and scheduling are solved.</strong> SMS, email, calendars, reminders and the delivery infrastructure behind them are the platform&#8217;s core. Rebuilding any of it is a bad trade.</li>



<li><strong>Timezone handling is built in.</strong> Time-based steps run in the account timezone or the contact&#8217;s own, falling back to the account when a contact has none.</li>



<li><strong>No pipeline to maintain.</strong> No repository, no secrets rotation, no dependency upgrades, no server needing patched at an awkward hour.</li>



<li><strong>Execution history is right there.</strong> For a linear sequence, reading what happened to a contact beats correlating logs across two systems.</li>
</ul>



<p class="wp-block-paragraph">If the automation is a follow-up sequence with branching, the builder is correct and code is the expensive mistake.</p>



<h2 class="wp-block-heading">Three levels before you build an app</h2>



<p class="wp-block-paragraph">&#8220;Needs a developer&#8221; is not one decision. It is three, and most accounts only need the first.</p>



<h3 class="wp-block-heading">Level one: the Custom Code action</h3>



<p class="wp-block-paragraph">The builder has a Custom Code premium action that runs JavaScript inside the workflow. Values from earlier steps are mapped in as named input properties and arrive in an <code>InputData</code> object. The action returns a JavaScript object or an array of objects for later steps to reference, and <code>console.log</code> output is captured so you can see what ran.</p>



<pre class="wp-block-code"><code>// Normalise a phone number before it reaches a branch step.
// rawPhone and country are input properties mapped from earlier steps.
const digits = String(InputData.rawPhone || "").replace(/D/g, "");
const cc = InputData.country === "US" ? "1" : "";

if (!digits) {
  console.log("no phone on contact, routing to manual branch");
  return { e164: "", usable: false };
}

return {
  e164: "+" + (digits.startsWith(cc) ? digits : cc + digits),
  usable: digits.length &gt;= 10
};</code></pre>



<p class="wp-block-paragraph">This kills a lot of branch sprawl. Ten If/Else steps existing only to reshape a value collapse into one action returning a clean value and a boolean. Still inside the platform, still billed per execution, still no retries. But logic smeared across a canvas now reads in one place.</p>



<h3 class="wp-block-heading">Level two: a private integration and a script you own</h3>



<p class="wp-block-paragraph">For one account you do not need a marketplace app. Create a Private Integration inside the sub-account, select only the scopes the job needs, and use the token as a bearer credential. No OAuth handshake, no refresh cycle, no listing.</p>



<p class="wp-block-paragraph">This is where reconciliation jobs, nightly audits, migrations and bulk cleanups belong. Run it as a cron job on a small VPS, behind a Cloudflare Tunnel if it needs an inbound endpoint, with the token in a secrets store rather than a config file. Treat that token as a credential with a rotation plan, not a string you paste once and forget.</p>



<h3 class="wp-block-heading">Level three: a marketplace app</h3>



<p class="wp-block-paragraph">You need this when the thing spans many accounts, has a user interface, or is a product you intend to sell. It means OAuth 2.0 on the authorization code grant, token storage and refresh per installed location, webhook handling, and somewhere to run it. Note the version boundary: the v1 API is end of life and unmaintained, and new builds target the OAuth-based v2. If a tutorial hands you a permanent location API key, it is describing the old world.</p>



<p class="wp-block-paragraph">Marketplace apps can also register their own workflow triggers and actions, which appear in the builder for sub-accounts that installed the app. That is the pattern I reach for first: the hard logic lives in your service, and the person who owns the campaign still drags a step onto a canvas. It requires the workflows read scope, and those custom actions are themselves premium actions billed per execution.</p>



<h2 class="wp-block-heading">Arguments that do not survive contact</h2>



<ul class="wp-block-list">
<li><strong>&#8220;We will just add one more workflow.&#8221;</strong> The cost is not building it. It is that every future workflow now has to be reasoned about alongside it. Ten workflows listening for one tag is not ten problems, it is the interactions between them.</li>



<li><strong>&#8220;No-code means no developer.&#8221;</strong> It means no deployment. Someone still holds the whole model in their head, and when they leave, an undocumented canvas is harder to inherit than a repository with a README.</li>



<li><strong>&#8220;Zapier or Make will handle it.&#8221;</strong> Often true and worth trying first. What they do not give you is one place where business logic lives. Split rules across three tools and debugging means three log formats.</li>



<li><strong>&#8220;It works, so it is fine.&#8221;</strong> Every silent failure worked right up until someone counted. Absence of errors in a system with no error reporting is not evidence.</li>



<li><strong>&#8220;Custom code is lock-in.&#8221;</strong> Backwards. Logic on a canvas is bound to one vendor&#8217;s UI and exports only as a snapshot. Logic in a repository moves.</li>
</ul>



<h2 class="wp-block-heading">How I would decide</h2>



<ol class="wp-block-list">
<li><strong>Write down what happens when each external call fails.</strong> If the answer anywhere is &#8220;the contact continues as if it succeeded&#8221; and that matters, you have found your requirement.</li>



<li><strong>Count the workflows writing to the same tag or field.</strong> More than two means you have a state machine. Draw it before touching anything.</li>



<li><strong>Count premium executions per lead, then multiply by monthly volume.</strong> Compare against a few hours a month maintaining a small service. Sometimes the builder still wins. Do the arithmetic instead of assuming.</li>



<li><strong>Ask how a change reaches every account.</strong> If it involves opening sub-accounts one at a time, that is the real cost, and it grows with every client.</li>



<li><strong>Try the smallest level that fits, and leave the trigger in the builder.</strong> Custom Code action, then private integration script, then marketplace app. Whatever moves out, keep the entry point as a workflow step so the marketing side can still change timing and audience without you.</li>
</ol>



<p class="wp-block-paragraph">Whatever you build, monitor it. An uptime check on your receiver and an alert when the reconciliation job reports a non-zero difference will catch more real problems than any amount of careful workflow naming.</p>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Do I need a marketplace app to use the GoHighLevel API?</h3>



<p class="wp-block-paragraph">No. For a single sub-account, create a Private Integration inside that account, grant only the scopes you need, and use the token as a bearer credential. Marketplace apps and OAuth are for anything spanning accounts or distributed to other people.</p>



<h3 class="wp-block-heading">How much automation should stay in workflows?</h3>



<p class="wp-block-paragraph">The parts a non-developer should be able to change: timing, audience, message copy, branching on obvious contact attributes. Move out the parts where being wrong is expensive and being wrong is silent.</p>



<h3 class="wp-block-heading">Why does my workflow show as successful when the integration clearly failed?</h3>



<p class="wp-block-paragraph">A step firing and a step achieving something are different events, and the builder records the first. A webhook returning an error still counts as having run. If the outcome matters, verify it explicitly rather than trusting the log.</p>



<h3 class="wp-block-heading">Should I avoid premium workflow actions on cost grounds?</h3>



<p class="wp-block-paragraph">Not on principle. They cost per execution because they are the expensive verbs. The problem is fragmentation: three separate calls where one would do triples the bill for the same outcome. Consolidate before considering a move off the platform.</p>



<h3 class="wp-block-heading">What does a GoHighLevel custom development project usually start with?</h3>



<p class="wp-block-paragraph">An audit, not a build. Map which workflows write to which fields, where external calls happen, and what the failure path is for each. Often that document alone removes the need for the project.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">Ask what happens when a step fails. That question separates automation belonging in the builder from automation needing code, more reliably than counting steps, counting workflows, or judging how clever the branching has become.</p>



<p class="wp-block-paragraph">If failure is visible and cheap, stay in the builder and enjoy that your client can edit it themselves. If failure is silent and expensive, GoHighLevel custom development stops being an upgrade and becomes maintenance you already deferred. Then start small: a Custom Code action beats a marketplace app you will not want to maintain.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Working on this with me</h2>



<p class="wp-block-paragraph">I work on the engineering side of GoHighLevel accounts, usually the parts failing quietly rather than the parts that look broken.</p>



<ul class="wp-block-list">
<li>Auditing an account for silent failures: which workflows write to which tags and fields, and what happens on every external call that returns an error.</li>



<li>Replacing branch sprawl with Custom Code actions, so logic reads as one block instead of fifteen steps on a canvas.</li>



<li>Webhook receivers and sync services that retry, deduplicate and reconcile, with the rate limit headers driving backoff instead of a fixed sleep.</li>



<li>Migrations and backfills planned against the daily API budget, batched and run in the account&#8217;s timezone so live traffic keeps working.</li>



<li>Private integration setup with least-privilege scopes, token storage and a rotation plan that is written down.</li>



<li>Marketplace app work when genuinely warranted: OAuth handling, custom workflow actions, and Custom Pages hosted somewhere you control.</li>
</ul>



<p class="wp-block-paragraph">If you want a second opinion, send something concrete: a screenshot of the workflow you are least sure about, an execution log entry that makes no sense, or the response body your webhook endpoint is returning. That is a faster conversation than a scoping call.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/workflow-automation/gohighlevel-custom-development-vs-workflows/">Six Signs GoHighLevel Needs a Developer, Not Another Workflow</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Monitoring n8n Workflow Failures in Grafana: Why Your Green Dashboard Is Lying</title>
		<link>https://john-nessime.com/blog/devops/monitoring-n8n-workflow-failures-grafana/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Tue, 25 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Troubleshooting]]></category>
		<category><![CDATA[Alerting]]></category>
		<category><![CDATA[Automation]]></category>
		<category><![CDATA[Cardinality]]></category>
		<category><![CDATA[Docker Compose]]></category>
		<category><![CDATA[Grafana]]></category>
		<category><![CDATA[Monitoring]]></category>
		<category><![CDATA[n8n]]></category>
		<category><![CDATA[Observability]]></category>
		<category><![CDATA[Production]]></category>
		<category><![CDATA[Prometheus]]></category>
		<category><![CDATA[PromQL]]></category>
		<category><![CDATA[Pushgateway]]></category>
		<category><![CDATA[Self Hosting]]></category>
		<category><![CDATA[SRE]]></category>
		<category><![CDATA[Webhooks]]></category>
		<category><![CDATA[Workflow Automation]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=276</guid>

					<description><![CDATA[<p>n8n's Prometheus endpoint tells you the process is alive and busy. It does not tell you a workflow stopped producing correct output. This post walks through the three failure families that slip past a default n8n dashboard, the PromQL that catches each one, and how to close the gap with a second source of truth from the executions API.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/monitoring-n8n-workflow-failures-grafana/">Monitoring n8n Workflow Failures in Grafana: Why Your Green Dashboard Is Lying</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Someone asks why the nightly invoice sync hasn&#8217;t posted anything since Tuesday. You open Grafana. The n8n scrape target is UP, the instance has been running for weeks without a restart, every panel is green. Nothing on the dashboard says a thing is wrong, and yet four days of records are missing.</p>



<p class="wp-block-paragraph">That gap is the entire problem. Monitoring n8n workflow failures in Grafana is not hard because the tooling is missing. It&#8217;s hard because the built-in metrics endpoint answers a question you didn&#8217;t ask. It tells you the process is alive and busy. It does not tell you that a workflow stopped producing correct output.</p>



<p class="wp-block-paragraph">This post covers what n8n&#8217;s Prometheus endpoint actually exposes, the three families of failure it handles badly, and the specific PromQL and alert rules that close each gap. There&#8217;s also a troubleshooting section for the case where <code>/metrics</code> returns nothing at all, which is the first wall most people hit.</p>



<h2 class="wp-block-heading">What the n8n metrics endpoint actually gives you</h2>



<p class="wp-block-paragraph">n8n uses the <code>prom-client</code> library and serves metrics on the same port as the editor, at <code>/metrics</code>. The endpoint is off by default, and almost everything interesting inside it is off by default too, behind its own environment variable.</p>



<pre class="wp-block-code"><code>N8N_METRICS=true
N8N_METRICS_INCLUDE_MESSAGE_EVENT_BUS_METRICS=true
N8N_METRICS_INCLUDE_WORKFLOW_ID_LABEL=true
N8N_METRICS_INCLUDE_NODE_TYPE_LABEL=true
N8N_METRICS_INCLUDE_QUEUE_METRICS=true</code></pre>



<p class="wp-block-paragraph">The first line opens the endpoint. The second turns on the workflow and node event counters, which is the part people assume is included and is not. The third attaches a <code>workflow_id</code> label to workflow metrics, without which you get a single instance-wide number and no way to tell which automation broke.</p>



<p class="wp-block-paragraph">Those labels are opt-in for a reason worth understanding rather than working around. Every distinct label value creates a separate time series. Turn on <code>workflow_id</code> and you get one series per workflow per counter; add <code>node_type</code> and node counters fan out across every integration you use. On a few dozen workflows that&#8217;s fine. On several hundred with per-node labels, you&#8217;re signing up for a cardinality problem that surfaces later as slow queries and a Prometheus process that keeps getting OOM-killed.</p>



<p class="wp-block-paragraph">With the event bus metrics enabled, the counters you get include <code>n8n_workflow_started_total</code>, <code>n8n_workflow_success_total</code>, <code>n8n_node_started_total</code> and <code>n8n_node_finished_total</code>. Queue mode adds <code>n8n_scaling_mode_queue_jobs_active</code>, <code>_completed</code>, <code>_failed</code> and <code>_waiting</code>, gathered from Bull and exposed on the main instance only.</p>



<p class="wp-block-paragraph">Prometheus side, the scrape job is unremarkable:</p>



<pre class="wp-block-code"><code>scrape_configs:
  - job_name: n8n
    static_configs:
      - targets:
          - n8n-main:5678
          - n8n-worker-1:5678
    metrics_path: /metrics</code></pre>



<p class="wp-block-paragraph">Note both main and workers. Each process exposes its own endpoint and its own counters. Scrape only the main and you lose everything the workers did, which in queue mode is nearly all the real execution work.</p>



<p class="wp-block-paragraph">One thing the n8n docs are emphatic about, and they&#8217;re right: don&#8217;t put <code>/metrics</code> on the public internet. It leaks workflow IDs, execution volumes and instance topology to anyone who asks. Keep the scrape path on a private network segment, a WireGuard tunnel, or your provider&#8217;s internal networking. If you&#8217;re running the whole stack on a single VPS from somewhere like Contabo or InterServer, binding n8n to the Docker network and scraping it by container name is enough. Publishing port 5678 to the host and firewalling it afterwards is not, because you will forget.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Failure family one: the execution that never happened</h2>



<p class="wp-block-paragraph">This is the one that costs the most and shows up the least. A schedule trigger gets deactivated during a deploy. A credential expires and the workflow is switched off rather than fixed. An upstream service stops calling a webhook. Nothing errors. Nothing increments. The workflow simply stops existing as far as your metrics are concerned.</p>



<p class="wp-block-paragraph">Every error-rate alert you write is blind to this, because an error rate needs a denominator. Zero executions divided by zero executions is not an alert, it&#8217;s a NaN.</p>



<p class="wp-block-paragraph">The instinct is to write this:</p>



<pre class="wp-block-code"><code>sum by (workflow_id) (increase(n8n_workflow_started_total[6h])) == 0</code></pre>



<p class="wp-block-paragraph">It looks correct and it does not work. When a workflow stops running, its series stops being exported. Prometheus marks it stale after a few missed scrapes and instant queries stop returning it. There is no series left for <code>== 0</code> to match against. The alert never fires, because the thing you wanted to alert on is exactly the thing that removed the data.</p>



<p class="wp-block-paragraph">What you want is a function that returns something when a series is absent:</p>



<pre class="wp-block-code"><code>absent_over_time(
  n8n_workflow_started_total{workflow_id="AbC123XyZ"}[6h]
)</code></pre>



<p class="wp-block-paragraph"><code>absent_over_time</code> returns 1 if the matched series produced no samples in the window and nothing at all if it did. That inversion is the point. The alert fires on silence.</p>



<p class="wp-block-paragraph">The cost is that <code>absent_over_time</code> needs an exact matcher, so you can&#8217;t write it once for everything. That&#8217;s more acceptable than it sounds. You don&#8217;t need a heartbeat on all two hundred workflows, you need one on the six a human would notice within an hour. Write those six by hand and treat the list as part of the runbook.</p>



<p class="wp-block-paragraph">One Grafana-specific trap. When a query returns nothing, Grafana&#8217;s alert rule enters a <strong>No Data</strong> state, which is configurable separately from the alert condition and defaults to firing in most versions. Set it deliberately. For an <code>absent_over_time</code> rule, No Data means the whole scrape target is gone, which is a different incident from one workflow going quiet, and you probably want it routed differently.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Failure family two: the failure you can see but can&#8217;t attribute</h2>



<p class="wp-block-paragraph">You have the counters. Something is failing. Which workflow?</p>



<p class="wp-block-paragraph">Without <code>N8N_METRICS_INCLUDE_WORKFLOW_ID_LABEL</code>, the answer is: no idea. The counters are instance-wide totals. You know the number moved and that&#8217;s it. That&#8217;s a fine signal for capacity planning and useless during an incident.</p>



<p class="wp-block-paragraph">Turn the label on and you get IDs, which are opaque strings like <code>AbC123XyZ</code>. A dashboard full of those is technically attributable and practically unreadable at 3am. Newer n8n versions solve this with a gauge that maps IDs to names, enabled with <code>N8N_METRICS_INCLUDE_WORKFLOW_INFO</code>:</p>



<pre class="wp-block-code"><code>n8n_workflow_info{workflow_id="VaQPuPmx9tPpo6BX",workflow_name="Invoice sync"} 1</code></pre>



<p class="wp-block-paragraph">It&#8217;s a constant-1 gauge that exists only to be joined against. The pattern is a standard Prometheus vector match:</p>



<pre class="wp-block-code"><code>sum by (workflow_name) (
  rate(n8n_workflow_started_total[5m])
  * on(workflow_id) group_left(workflow_name)
  n8n_workflow_info
)</code></pre>



<p class="wp-block-paragraph">The multiplication is arithmetically a no-op because the gauge is always 1. Its only job is to carry the <code>workflow_name</code> label across with <code>group_left</code>. Do this once in a Grafana variable or a recording rule rather than in every panel, or you&#8217;ll be copy-pasting the join for the rest of your life.</p>



<p class="wp-block-paragraph">On an older build without the info gauge, the fallback is a hand-maintained relabel config or a Grafana value mapping. Both drift, and the relabel version drifts silently.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Failure family three: the failure that counts as a success</h2>



<p class="wp-block-paragraph">This is the invisible one, and it&#8217;s why a green n8n dashboard is not evidence of anything.</p>



<p class="wp-block-paragraph">n8n gives you several deliberate ways to make a node&#8217;s failure not be an execution failure. They&#8217;re all reasonable features. Together they mean the success counter and &#8220;the automation worked&#8221; are different statements.</p>



<ul class="wp-block-list">
<li><strong>Continue on error.</strong> A node set to continue passes its error downstream instead of stopping the run. The execution finishes. It finishes as a success.</li>

<li><strong>Error output branches.</strong> A node with a separate error output routes failures to a different path. That path might log and stop. The execution still completes normally.</li>

<li><strong>Never Error on HTTP responses.</strong> The HTTP Request node can be configured to treat any status code as a valid response. A run of 500s from an upstream API becomes a workflow that processed zero useful records, successfully.</li>

<li><strong>Empty result sets.</strong> A query that returns nothing is not an error. A sync that syncs nothing, every night, looks exactly like a sync that has nothing to do.</li>
</ul>



<p class="wp-block-paragraph">There are also user reports on the n8n community forum of started and success counters matching even when a node failed outright, in queue mode. I have not reproduced that and I&#8217;d treat it as unconfirmed rather than a known behavior. But it&#8217;s the right instinct: don&#8217;t assume the success counter is a strict complement of failure. Verify it on your own instance before you build alerting on the assumption.</p>



<p class="wp-block-paragraph">For webhook-triggered workflows, newer n8n versions give you a real answer here. The <code>n8n_webhook_request_duration_seconds</code> histogram carries a <code>status_code</code> label, so you can alert on what the caller actually received rather than on what n8n thought about the run:</p>



<pre class="wp-block-code"><code>sum by (workflow_id) (
  rate(n8n_webhook_request_duration_seconds_count{status_code!~"2.."}[5m])
)</code></pre>



<p class="wp-block-paragraph">Worth flagging: the n8n documentation shows this example with <code>status_code!="2.."</code>, using the equality operator. In PromQL, <code>!=</code> compares against a literal string, so that matcher excludes only a status code whose literal value is the three characters <code>2..</code>, which never occurs. It matches everything, including your 200s. Use <code>!~</code> for the regex form. Test any label matcher you copy from documentation by running it in Prometheus and checking the series count against what you expect.</p>



<h3 class="wp-block-heading">Getting a second source of truth</h3>



<p class="wp-block-paragraph">For everything that isn&#8217;t a webhook, the metrics endpoint alone won&#8217;t close this gap. You need a second signal that reads execution outcomes from n8n&#8217;s own store.</p>



<p class="wp-block-paragraph">The public REST API exposes exactly that. Executions can be filtered by status, and the failed ones are one request away:</p>



<pre class="wp-block-code"><code>curl -s -H "X-N8N-API-KEY: $N8N_API_KEY" 
  "https://n8n.example.com/api/v1/executions?status=error&amp;limit=50"</code></pre>



<p class="wp-block-paragraph">Two ways to get that into Grafana. Write a small exporter that polls the API on a timer and serves its own <code>/metrics</code>, or run a scheduled job that pushes counts to a Prometheus Pushgateway. The exporter is more work and behaves correctly. Pushgateway is faster to build and carries the usual caveat: pushed metrics persist until deleted, so a job that stops running leaves a stale value sitting there looking healthy. That reintroduces failure family one through the back door.</p>



<p class="wp-block-paragraph">Two caveats on the API. There&#8217;s an open report that the <code>status=crashed</code> list filter returns executions whose stored status is something else entirely when re-fetched by ID, which would make crash alerting built on the list endpoint unreliable. Check the behavior on your version before depending on it. And n8n&#8217;s log streaming feature, which forwards execution events to syslog or a webhook in real time, is an Enterprise capability. If you&#8217;re on the community edition, the API poll is your path.</p>



<p class="wp-block-paragraph">The third option, and often the best one for a small setup, is an Error Trigger workflow. n8n lets you designate a workflow that runs whenever another workflow fails. Point it at Pushgateway, at Loki, or at a Grafana webhook contact point. It&#8217;s native, it needs no API key, and it gives you the error message rather than just a count. The trade-off is that it only fires on real execution failures, so it inherits every blind spot in this section.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The Grafana panels that earn their space</h2>



<p class="wp-block-paragraph">Four panels cover most of what you need. Everything past that is usually decoration.</p>



<ol class="wp-block-list">
<li><strong>Executions per workflow, by name.</strong> A time series of <code>rate(n8n_workflow_started_total[5m])</code> joined to <code>n8n_workflow_info</code>. Its real job is showing you the lines that flatten to zero.</li>

<li><strong>Success ratio.</strong> <code>rate(n8n_workflow_success_total[15m]) / rate(n8n_workflow_started_total[15m])</code>. Anything below 1 is worth a look. Anything above 1 means your window is catching a restart and the counters reset.</li>

<li><strong>Queue depth and failed jobs.</strong> <code>n8n_scaling_mode_queue_jobs_waiting</code> next to <code>increase(n8n_scaling_mode_queue_jobs_failed[1h])</code>. A waiting count that climbs and never drains means workers are dead or starved, which looks identical to &#8220;no failures&#8221; on every other panel.</li>

<li><strong>Webhook non-2xx rate.</strong> The query from the previous section. This is the only panel that reports what the outside world experienced.</li>
</ol>



<p class="wp-block-paragraph">Skip the node-level breakdown unless you have a specific reason. It is the most expensive thing you can enable in cardinality terms and it answers a question you can answer faster by opening the execution in the n8n UI.</p>



<h3 class="wp-block-heading">Alert rules worth paging on</h3>



<ol class="wp-block-list">
<li><strong>Named workflow silent.</strong> <code>absent_over_time</code> on each business-critical workflow, window set to roughly twice its expected interval. This is the rule that catches the invoice sync.</li>

<li><strong>Success ratio below threshold.</strong> Sustained for 15 minutes, not instantaneous. A single failed run is not a page.</li>

<li><strong>Queue waiting count rising.</strong> Compare the current value to fifteen minutes ago rather than to a fixed number, so the threshold survives traffic growth.</li>

<li><strong>Webhook error rate.</strong> Per workflow, so a single noisy endpoint doesn&#8217;t mask the rest.</li>

<li><strong>Scrape target down.</strong> The <code>up</code> metric. Obvious, and routinely forgotten, and it invalidates every other rule on this list.</li>
</ol>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting: the endpoint returns nothing useful</h2>



<ul class="wp-block-list">
<li><strong>&#8220;Cannot GET /metrics&#8221;.</strong> <code>N8N_METRICS</code> isn&#8217;t set on the process you&#8217;re hitting, or it&#8217;s set as a string that n8n doesn&#8217;t read as true. In Docker Compose, quote it as <code>"true"</code> and confirm with <code>docker compose exec n8n env | grep N8N_METRICS</code>.</li>

<li><strong>Endpoint responds but has no n8n_ counters.</strong> You have the default Node.js process metrics and nothing else. Set <code>N8N_METRICS_INCLUDE_MESSAGE_EVENT_BUS_METRICS=true</code> and restart.</li>

<li><strong>Counters exist but no workflow_id label.</strong> <code>N8N_METRICS_INCLUDE_WORKFLOW_ID_LABEL</code> is off. Restart is required; it&#8217;s read at startup.</li>

<li><strong>Queue metrics are zero or missing.</strong> They&#8217;re gathered from Bull and exposed on the main instance, not the workers. Querying the worker endpoint gives you nothing. Confirm you&#8217;re looking at the right target.</li>

<li><strong>Counters look wrong after a deploy.</strong> They reset to zero when the process restarts. Always wrap them in <code>rate()</code> or <code>increase()</code>, which handle counter resets. Raw counter values on a dashboard will lie to you every time you deploy.</li>

<li><strong>Prometheus can&#8217;t reach the target.</strong> Usually a Docker network mismatch. Scrape by container name on the shared network rather than by published host port, and check the target page in Prometheus for the actual error rather than guessing.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating the success counter as proof the automation worked. It proves the run finished.</li>

<li>Building only error-rate alerts, so the workflows that stop entirely go unmonitored.</li>

<li>Enabling every <code>N8N_METRICS_INCLUDE_*</code> variable on day one and discovering the cardinality bill a month later.</li>

<li>Scraping only the main instance in queue mode and losing all worker-side execution data.</li>

<li>Copying label matchers out of documentation without running them and checking the returned series count.</li>

<li>Exposing <code>/metrics</code> publicly because it&#8217;s &#8220;just numbers&#8221;.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Decide which workflows are business-critical and write a silence alert for each one by name. Six good heartbeats beat two hundred generic panels.</li>

<li>Run a second source of truth. Metrics tell you about volume, the executions API tells you about outcomes. You need both.</li>

<li>Push the ID-to-name join into a recording rule so panels stay readable and cheap.</li>

<li>Break a staging workflow on purpose and confirm the alert fires. An untested alert is a decoration.</li>

<li>If you&#8217;d rather not run Prometheus and Grafana yourself, Grafana Cloud will scrape a remote endpoint and gives you alerting without a control plane to maintain. Weigh that against your metrics leaving your network.</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Does n8n expose a failed-executions counter?</h3>



<p class="wp-block-paragraph">Not a reliable per-workflow one. You get started and success counters from the event bus, and a failed-jobs counter for queue mode. Deriving failures by subtracting success from started works only if you&#8217;ve verified on your own instance that a failing run genuinely doesn&#8217;t increment success, which is exactly the assumption that breaks when nodes are set to continue on error.</p>



<h3 class="wp-block-heading">Can I monitor n8n Cloud this way?</h3>



<p class="wp-block-paragraph">No. The <code>/metrics</code> endpoint isn&#8217;t available on n8n Cloud. On Cloud you&#8217;re limited to the public API and the built-in insights views, so the executions-API approach in this post is the only one that transfers.</p>



<h3 class="wp-block-heading">Why does my workflow error rate look like zero when I know things are failing?</h3>



<p class="wp-block-paragraph">Almost always one of three things: the event bus metrics aren&#8217;t enabled so you&#8217;re only seeing Node.js process metrics, the failures are being absorbed by continue-on-error settings and recorded as successes, or you&#8217;re scraping the main instance while the work happens on workers.</p>



<h3 class="wp-block-heading">How do I alert when a scheduled workflow stops running entirely?</h3>



<p class="wp-block-paragraph">Use <code>absent_over_time</code> against that workflow&#8217;s started counter with an exact <code>workflow_id</code> matcher, and set the window to about twice the expected interval. A threshold comparison won&#8217;t work, because a workflow that stops running stops producing the series you&#8217;d compare against.</p>



<h3 class="wp-block-heading">Is enabling the workflow_id label safe on a large instance?</h3>



<p class="wp-block-paragraph">It multiplies your n8n series count by the number of workflows. On a few dozen it&#8217;s negligible. Past a few hundred, combined with node-type labels, watch your Prometheus memory and head series count before and after. Enable the workflow label, leave the node label off, and only turn the node one on when you have a specific question it answers.</p>



<h3 class="wp-block-heading">Should I use OpenTelemetry instead of Prometheus for n8n?</h3>



<p class="wp-block-paragraph">They solve different problems. Metrics answer &#8220;how many, how often, how fast&#8221; cheaply and forever. Traces answer &#8220;where did this run spend its time and what did it call&#8221;, which is better for debugging a slow multi-service workflow. Tracing n8n usually means a custom image with the OpenTelemetry SDK loaded before startup, so it&#8217;s a project rather than an environment variable. Start with metrics; add tracing when you have a latency question metrics can&#8217;t answer.</p>



<h3 class="wp-block-heading">Where do the n8n counters go when the container restarts?</h3>



<p class="wp-block-paragraph">They reset to zero. They&#8217;re in-process counters with no persistence. Prometheus handles this correctly as long as your queries use <code>rate()</code> or <code>increase()</code>. Any panel showing a raw counter value will appear to lose all its history on every deploy.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">Monitoring n8n workflow failures in Grafana works, but only once you stop asking the metrics endpoint a question it can&#8217;t answer. It reports activity. Activity and correctness are different things, and the space between them is where the expensive outages live.</p>



<p class="wp-block-paragraph">So build for absence first. A silence alert on the handful of workflows a human would miss will catch more real incidents than any error-rate panel, because errors are loud and stopped schedules are not. Then add attribution with the workflow label, then close the false-success gap with the executions API or an Error Trigger workflow. In that order, because each one is cheap and the value drops off fast after the third.</p>



<h2 class="wp-block-heading">Need help wiring this up?</h2>



<p class="wp-block-paragraph">I build and maintain observability stacks for self-hosted automation platforms, and n8n is a common one. If any of the above sounds like a week you&#8217;d rather not spend, here&#8217;s what I can take off your plate:</p>



<ul class="wp-block-list">
<li>Enabling and securing the n8n metrics endpoint on a private network path, including queue-mode setups with multiple workers</li>

<li>Building Prometheus scrape configs, recording rules and the ID-to-name join so your Grafana panels are readable rather than a wall of hashes</li>

<li>Writing silence and error-rate alert rules that survive counter resets, Grafana No Data states and traffic growth</li>

<li>Adding a second source of truth from the n8n executions API, either as a small exporter or an Error Trigger workflow feeding Loki or Pushgateway</li>

<li>Auditing an existing setup for cardinality problems, stale Pushgateway series and alerts that quietly stopped being able to fire</li>

<li>Running a failure drill against staging so you know which alerts actually work before you need them</li>
</ul>



<p class="wp-block-paragraph">Send me your Compose file, your <code>prometheus.yml</code>, or a screenshot of the panel that looks fine while things are on fire, and I&#8217;ll tell you what&#8217;s missing.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/monitoring-n8n-workflow-failures-grafana/">Monitoring n8n Workflow Failures in Grafana: Why Your Green Dashboard Is Lying</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
