<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>GraphQL | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/graphql/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/graphql/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Sun, 20 Sep 2026 11:12:39 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>GraphQL | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/graphql/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Building a GraphQL Data Ingestion Pipeline on AWS That Doesn&#8217;t Lie to You</title>
		<link>https://john-nessime.com/blog/devops/graphql-data-ingestion-pipeline-aws/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Mon, 17 Aug 2026 18:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Athena]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Glue]]></category>
		<category><![CDATA[AWS Lambda]]></category>
		<category><![CDATA[Cursor Pagination]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[Data Lake]]></category>
		<category><![CDATA[Data Quality]]></category>
		<category><![CDATA[ETL]]></category>
		<category><![CDATA[EventBridge]]></category>
		<category><![CDATA[GraphQL]]></category>
		<category><![CDATA[Idempotency]]></category>
		<category><![CDATA[Incremental Sync]]></category>
		<category><![CDATA[Orchestration]]></category>
		<category><![CDATA[Pipeline Design]]></category>
		<category><![CDATA[Rate Limiting]]></category>
		<category><![CDATA[REST API]]></category>
		<category><![CDATA[Schema Drift]]></category>
		<category><![CDATA[Secrets Management]]></category>
		<category><![CDATA[Serverless]]></category>
		<category><![CDATA[Shopify]]></category>
		<category><![CDATA[Step Functions]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=215</guid>

					<description><![CDATA[<p>A GraphQL source can hand you a 200 OK, a populated data block, and a quietly broken column in the same response. Here is how to build a GraphQL data ingestion pipeline on AWS that catches partial errors, respects cost-based rate limits, resumes cleanly from a cursor, and notices when the schema moves under you.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/graphql-data-ingestion-pipeline-aws/">Building a GraphQL Data Ingestion Pipeline on AWS That Doesn&#8217;t Lie to You</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The row counts looked fine. That was the problem.</p>



<p class="wp-block-paragraph">A nightly sync from a partner&#8217;s GraphQL API had run for weeks without a single failed execution. Green across the board in the console. Then someone asked why one column had been empty since the previous Tuesday. Not sparse. Empty. Every row, every night, for six nights.</p>



<p class="wp-block-paragraph">The API had started returning a field-level error on that one field. Status code 200. A populated <code>data</code> block. An <code>errors</code> array sitting right next to it. The fetcher parsed <code>data</code>, wrote the records to S3, exited zero, and the orchestrator logged a success. Nothing in the stack was wrong, exactly. Everything in the stack was checking the wrong thing.</p>



<p class="wp-block-paragraph">This post covers how to build a GraphQL data ingestion pipeline on AWS that fails honestly. It is organised by failure family rather than by service, because the AWS wiring is the easy part. The hard part is that GraphQL breaks four assumptions that REST ingestion quietly relies on, and every one of them fails silently by default.</p>



<h2 class="wp-block-heading">Why GraphQL ingestion breaks the habits REST taught you</h2>



<p class="wp-block-paragraph">Most ETL code carries three inherited reflexes from years of REST work: the status code tells you whether it worked, the response shape is the server&#8217;s business, and rate limits are counted in requests. GraphQL invalidates all three.</p>



<ul class="wp-block-list">
<li><strong>The status code is not the contract.</strong> The GraphQL over HTTP specification says a server should return 200 even when field errors are raised during execution, because a partial response is still a successful execution.</li>

<li><strong>You own the response shape.</strong> The selection set is written by you, which means schema drift shows up as a query that stops being valid, not as a payload that changes underneath you.</li>

<li><strong>Cost is query-shaped, not endpoint-shaped.</strong> One endpoint can cost almost nothing or blow your entire budget depending on how deeply you nested it.</li>
</ul>



<p class="wp-block-paragraph">Each of those becomes a distinct failure family. Let&#8217;s take them in the order they bite.</p>



<h2 class="wp-block-heading">Failure one: the 200 OK that quietly ate a column</h2>



<p class="wp-block-paragraph">This is the one that costs the most and shows up the latest. GraphQL splits errors into two categories, and they behave completely differently on the wire.</p>



<ul class="wp-block-list">
<li><strong>Request errors</strong> happen before execution starts: syntax errors, validation failures, bad variable coercion. No <code>data</code> key is returned at all. Nothing executed.</li>

<li><strong>Field errors</strong> happen during execution: a resolver threw, a downstream service timed out, a permission check failed. Execution continues, the failed field goes null, and you get <code>data</code> and <code>errors</code> together.</li>
</ul>



<p class="wp-block-paragraph">Field errors are the dangerous ones for ingestion, because they look exactly like success to anything that only inspects the transport layer. Worse, GraphQL&#8217;s null propagation rules mean a single failing non-null field bubbles up to the nearest nullable ancestor, so one broken resolver can hollow out an entire nested object while the surrounding rows land normally.</p>



<p class="wp-block-paragraph">The fix is to make the response body the contract. Classify before you commit anything:</p>



<pre class="wp-block-code"><code>import json, urllib.request

class RequestError(Exception): pass
class FieldError(Exception): pass

def graphql(endpoint, token, query, variables):
    body = json.dumps({"query": query, "variables": variables}).encode()
    req = urllib.request.Request(
        endpoint,
        data=body,
        headers={
            "Content-Type": "application/json",
            "Accept": "application/graphql-response+json, application/json",
            "Authorization": f"Bearer {token}",
        },
    )
    with urllib.request.urlopen(req, timeout=30) as resp:
        payload = json.loads(resp.read())

    # The status code told us nothing. The body is the contract.
    if "data" not in payload:
        # Nothing executed. Query or auth problem. Do not retry blindly.
        raise RequestError(payload.get("errors"))

    if payload.get("errors"):
        # Partial response. data is populated but incomplete.
        raise FieldError(payload["errors"])

    return payload["data"]</code></pre>



<p class="wp-block-paragraph">Two things worth explaining. The <code>Accept</code> header asks for <code>application/graphql-response+json</code>, the media type the GraphQL over HTTP spec recommends, which lets a compliant server return a real 4xx for pre-execution failures instead of burying them in a 200. Servers that don&#8217;t support it fall back to <code>application/json</code>, so including both is free.</p>



<p class="wp-block-paragraph">The second is the split between the two exception types. They need different handling. A request error is a bug in your query or a credential problem, and retrying it just burns budget on the same failure. A field error is transient often enough to be worth retrying, but never worth committing.</p>



<h3 class="wp-block-heading">Deciding what &#8220;partial&#8221; means for your data</h3>



<p class="wp-block-paragraph">Failing the whole page on any error is the safe default and the one I reach for first. But it&#8217;s not always right. If you are pulling a hundred fields and one optional enrichment field is flaky, refusing every page forever is its own outage.</p>



<p class="wp-block-paragraph">The workable middle ground is to classify by path. Every error object carries a <code>path</code> array pointing at the exact field that failed. Maintain a small allowlist of paths that are permitted to be null, fail the page on anything outside it, and emit the error count as a CloudWatch metric so a rising trend is visible before someone in finance notices.</p>



<h2 class="wp-block-heading">Failure two: rate limits that never send a 429</h2>



<p class="wp-block-paragraph">Your retry logic almost certainly triggers on 429 and 5xx. Several major GraphQL APIs will throttle you without ever sending one.</p>



<p class="wp-block-paragraph">GitHub&#8217;s GraphQL API is explicit about this: exceeding the primary rate limit returns a 200 response with an error message in the body and <code>x-ratelimit-remaining</code> set to zero. Your Step Functions retrier sees a success. Your Lambda sees no data. The run completes, having ingested nothing, and reports green.</p>



<p class="wp-block-paragraph">The other half of the problem is that request counting is the wrong unit. Shopify&#8217;s GraphQL Admin API and several others price queries by <em>calculated cost</em>, using a leaky bucket. The server statically analyses your query before executing it, charges the bucket the worst-case <em>requested</em> cost, executes, then refunds the difference between requested and <em>actual</em> cost. That has a consequence people miss: a query asking for the maximum connection size gets charged as if every page were full, even when the last page returns three records.</p>



<p class="wp-block-paragraph">The good news is that these APIs tell you the state of the bucket on every response, under the <code>extensions</code> key:</p>



<pre class="wp-block-code"><code>{
  "extensions": {
    "cost": {
      "requestedQueryCost": 72,
      "actualQueryCost": 38,
      "throttleStatus": {
        "maximumAvailable": 1000.0,
        "currentlyAvailable": 962.0,
        "restoreRate": 100.0
      }
    }
  }
}</code></pre>



<p class="wp-block-paragraph">Read those numbers, never hardcode them. Bucket size and restore rate vary by plan tier, and a query that works fine against a large merchant&#8217;s store will fail instantly against a small one. The pattern that holds up is a pre-flight check: before dispatching, compare your estimated cost against <code>currentlyAvailable</code>, and if you&#8217;re short, sleep for the shortfall divided by <code>restoreRate</code>.</p>



<p class="wp-block-paragraph">On APIs that expose their own budget as a queryable field, ask for it in the same round trip rather than spending a second call to find out:</p>



<pre class="wp-block-code"><code>query BudgetCheck {
  rateLimit {
    limit
    cost
    remaining
    resetAt
  }
}</code></pre>



<p class="wp-block-paragraph">Two more things about concurrency. First, if you fan out across Lambda invocations, each worker&#8217;s local view of the bucket is wrong, because the bucket is shared. Multiple workers independently deciding they have headroom is the classic way to get throttled while every individual worker believes it&#8217;s behaving. Either centralise the token accounting or cap concurrency low enough that the arithmetic can&#8217;t go wrong.</p>



<p class="wp-block-paragraph">Second, if the source offers a bulk or async export operation, use it for backfills. Paginating fifty thousand records through the interactive endpoint burns the same budget your incremental syncs need, and it competes with itself.</p>



<h2 class="wp-block-heading">Failure three: cursors that don&#8217;t survive a restart</h2>



<p class="wp-block-paragraph">Most production GraphQL schemas follow the Relay cursor connections convention: <code>first</code> and <code>after</code> arguments, a <code>pageInfo</code> object with <code>hasNextPage</code> and <code>endCursor</code>, and edges carrying opaque per-item cursors.</p>



<pre class="wp-block-code"><code>query Orders($cursor: String) {
  orders(first: 100, after: $cursor) {
    pageInfo {
      hasNextPage
      endCursor
    }
    edges {
      cursor
      node {
        id
        updatedAt
      }
    }
  }
}</code></pre>



<p class="wp-block-paragraph">The filter argument for incremental sync is not standardised. Some schemas take a query-string filter, some take a typed timestamp argument, some expose nothing at all. Run introspection against the schema and find out rather than guessing, because a wrong argument name is a request error and your run will fail before it starts.</p>



<p class="wp-block-paragraph">The failure here is subtle. Cursors are opaque and their validity is entirely up to the server. Some are stable positional pointers. Some encode a sort key and a snapshot identifier and expire. If a page-47 fetch fails and you retry an hour later with the stored cursor, you may get an error, or you may get results from a shifted window with a hole in the middle. The second outcome is worse, because it looks like it worked.</p>



<p class="wp-block-paragraph">Three habits make restarts safe:</p>



<ol class="wp-block-list">
<li><strong>Checkpoint after the write, not after the fetch.</strong> A cursor recorded before the page lands in S3 is a promise you can&#8217;t keep.</li>

<li><strong>Store a high watermark alongside the cursor.</strong> The cursor is your fast path. The watermark, taken from the newest record you actually committed, is your fallback when the cursor is rejected.</li>

<li><strong>Overlap the watermark deliberately.</strong> Restart a few minutes before the last committed timestamp and deduplicate on the primary key downstream. Re-reading a handful of rows is cheap. Missing rows that arrived during a clock skew is not.</li>
</ol>



<p class="wp-block-paragraph">A DynamoDB item per source stream is enough state for all of this:</p>



<pre class="wp-block-code"><code>{
  "source": "orders",
  "run_id": "01hq7k...",
  "cursor": "eyJsYXN0X2lkIjoxMjM0fQ==",
  "high_watermark": "&lt;updatedAt of newest committed record&gt;",
  "pages_committed": 47,
  "status": "IN_PROGRESS"
}</code></pre>



<p class="wp-block-paragraph">Overlap only works if downstream deduplication is real. Write raw pages with an ingest-time partition and a run identifier in the object key, then let the curated layer resolve duplicates by primary key and latest <code>updatedAt</code>. If you write straight into a merged table with no dedupe step, overlapping restarts will double-count and you&#8217;ll have traded silent loss for silent inflation.</p>



<h2 class="wp-block-heading">Failure four: the schema moved and nobody told you</h2>



<p class="wp-block-paragraph">GraphQL&#8217;s deprecation story is genuinely better than REST&#8217;s. Fields carry a <code>@deprecated</code> directive with a reason string, and introspection exposes it. The problem is that nothing forces you to look.</p>



<p class="wp-block-paragraph">The failure mode is a field that gets deprecated, stops being populated, then eventually gets removed. Stage one is invisible: your query still validates, the field returns null, and your warehouse fills with nulls. Stage two is loud but late: the query becomes invalid and the whole ingest breaks at 3am.</p>



<p class="wp-block-paragraph">Put an introspection diff in CI. Fetch the schema on a schedule, store it as an artifact, and diff against the last known good version. Fail the build on removals or type changes to fields your queries actually select, and warn on new deprecations. Tools in the Apollo and Hasura ecosystems ship schema-diff tooling for this, and rolling your own against the introspection result is not much work either. Either way, the point is that the check runs on a schedule rather than when the pager goes off.</p>



<p class="wp-block-paragraph">The complementary habit is to keep queries narrow. Every field you select is a field that can break you, a field that costs budget, and a field that widens your Glue table. Selecting a whole object because it was convenient during development is a liability you carry indefinitely.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Wiring the GraphQL data ingestion pipeline on AWS</h2>



<p class="wp-block-paragraph">With the failure families understood, the AWS assembly is mostly a set of trade-offs about where the fetch loop lives.</p>



<h3 class="wp-block-heading">Where the fetch loop runs</h3>



<p class="wp-block-paragraph">Step Functions can call an HTTPS endpoint directly with an HTTP Task, no Lambda involved. It uses an EventBridge connection to hold the credentials, which means your token lives in a Secrets Manager secret managed by EventBridge rather than in an environment variable:</p>



<pre class="wp-block-code"><code>"FetchPage": {
  "Type": "Task",
  "Resource": "arn:aws:states:::http:invoke",
  "Parameters": {
    "ApiEndpoint": "https://api.example.com/graphql",
    "Method": "POST",
    "InvocationConfig": {
      "ConnectionArn": "arn:aws:events:region:account-id:connection/name/id"
    },
    "Headers": { "Content-Type": "application/json" },
    "RequestBody": {
      "query.$": "$.query",
      "variables.$": "$.variables"
    }
  },
  "Retry": [{
    "ErrorEquals": [
      "States.Http.StatusCode.429",
      "States.Http.StatusCode.502",
      "States.Http.StatusCode.503",
      "States.Http.StatusCode.504"
    ],
    "IntervalSeconds": 1,
    "BackoffRate": 2,
    "MaxAttempts": 3,
    "JitterStrategy": "FULL"
  }],
  "Next": "InspectBody"
}</code></pre>



<p class="wp-block-paragraph">The execution role needs four permissions: <code>states:InvokeHTTPEndpoint</code>, <code>events:RetrieveConnectionCredentials</code>, and both <code>secretsmanager:GetSecretValue</code> and <code>secretsmanager:DescribeSecret</code> scoped to the EventBridge-managed secret.</p>



<p class="wp-block-paragraph">It&#8217;s a clean pattern, and I like it for small, high-value syncs. Two constraints decide whether it fits. HTTP Task requests time out after 60 seconds, which a deeply nested GraphQL query can exceed without being pathological. And Step Functions reads the response within its standard state payload ceiling of roughly 256 KB, so a page of a hundred rich records will not fit. Also note the retrier above: it fires on HTTP status codes, so it will not catch a throttle that arrives as a 200 with an error in the body. That check has to happen in the state after it.</p>



<p class="wp-block-paragraph">The alternative is a Lambda driver that paginates internally and writes pages straight to S3, returning only the cursor and a count to the state machine. This dodges both limits, gives you a real HTTP client with real backoff, and keeps large payloads out of the workflow&#8217;s state. It costs you per-page retry granularity, which you buy back with the checkpoint record.</p>



<p class="wp-block-paragraph">Distributed Map is worth mentioning for the fan-out case, such as one child execution per tenant or per shard. It gives you high concurrency and a per-item execution history, which is genuinely useful for debugging. Be aware of two things before committing: express child workflows cap at five minutes rather than Lambda&#8217;s fifteen, and practitioners have published benchmarks showing throughput well below what the headline concurrency numbers suggest for small, fast items. For a lot of ingestion work, SQS plus a DynamoDB job table remains the simpler answer.</p>



<p class="wp-block-paragraph">One case genuinely falls outside serverless: sources that require a long-lived session, a persistent websocket subscription, or a synchronisation window longer than fifteen minutes with no resumable cursor. A small always-on worker on a modest VPS from a provider like InterServer or Hetzner, feeding the same S3 landing zone, is less clever and more honest than contorting Lambda around the constraint.</p>



<h3 class="wp-block-heading">The landing zone</h3>



<p class="wp-block-paragraph">Land raw responses first, unmodified, including the <code>errors</code> and <code>extensions</code> keys. Storage is the cheapest thing in this architecture and the raw layer is what lets you reprocess after you discover a parsing bug six weeks later.</p>



<ul class="wp-block-list">
<li><strong>Partition by ingest time, not event time.</strong> Event-time partitioning requires reading the record before you know where to put it, which means you can&#8217;t write until you&#8217;ve parsed. Ingest-time keys let the raw write be dumb and fast.</li>

<li><strong>Put the run identifier in the key.</strong> When you need to unwind one bad run, being able to delete or ignore by prefix beats writing a surgical delete.</li>

<li><strong>Define Glue tables explicitly rather than crawling.</strong> You wrote the selection set, so you already know the schema. A crawler inferring types from JSON will surprise you the first time an optional field is absent from an entire page.</li>

<li><strong>Convert to columnar in the curated layer.</strong> Athena over raw JSON works and is fine for debugging. Parquet is what you want for anything anyone queries repeatedly.</li>
</ul>



<p class="wp-block-paragraph">If your source pushes rather than being polled, such as webhook deliveries carrying GraphQL payloads, Amazon Data Firehose (renamed from Kinesis Data Firehose, with APIs and IAM policies unchanged) will buffer, convert to Parquet and partition on the way to S3 without you running a consumer. For scheduled pull-based syncs it adds a hop you don&#8217;t need.</p>



<h2 class="wp-block-heading">Troubleshooting</h2>



<ul class="wp-block-list">
<li><strong>Runs succeed but a column is all null.</strong> Field errors. Pull a raw response from the landing zone and look at the <code>errors</code> array. The <code>path</code> tells you exactly which field.</li>

<li><strong>Runs succeed but ingest zero rows.</strong> Likely a throttle returned as a 200. Log the response body and the rate-limit headers on empty results, always.</li>

<li><strong>Intermittent failures partway through pagination.</strong> Cursor expiry. Fall back to the watermark and check whether the source documents a cursor lifetime.</li>

<li><strong>Query fails immediately with no <code>data</code> key.</strong> Request error. Validate against current introspection; something in the schema moved.</li>

<li><strong>Throttled despite staying under the documented limit.</strong> Either concurrent workers are sharing a bucket they each think they own, or you&#8217;re being charged requested cost on near-empty pages.</li>

<li><strong>HTTP Task fails on large pages.</strong> The response exceeds the state payload ceiling. Reduce page size or move the fetch into Lambda.</li>

<li><strong>Row counts drift upward over time.</strong> Overlap without deduplication. Check that the curated layer actually resolves on the primary key.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating a 200 as success and never reading the body.</li>

<li>Retrying request errors. They are deterministic; retrying just spends budget to fail identically.</li>

<li>Hardcoding bucket sizes and restore rates that differ per plan tier and change without notice.</li>

<li>Checkpointing the cursor before the data is durable.</li>

<li>Requesting the maximum page size on every call while being charged the worst-case cost for it.</li>

<li>Selecting entire objects because it was easier during development.</li>

<li>Alerting only on execution failure, so a job that succeeds while ingesting nothing never pages anyone.</li>

<li>Overlapping restart windows with no downstream deduplication.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Classify every response into clean, partial, or request-failed before anything downstream sees it.</li>

<li>Persist raw responses with <code>errors</code> and <code>extensions</code> intact, then parse from the raw layer.</li>

<li>Alarm on row-count deltas and partial-error rate, not just on execution status.</li>

<li>Read the throttle state the API hands you and back off before you hit the wall, not after.</li>

<li>Keep both a cursor and a watermark, and know which one is authoritative on restart.</li>

<li>Diff the schema on a schedule so deprecations reach you before they reach your data.</li>

<li>Hold credentials in Secrets Manager or an EventBridge connection, never in a Lambda environment variable.</li>

<li>Trace the pipeline end to end. CloudWatch covers the basics; Grafana or Datadog earn their keep once you&#8217;re correlating child executions across a fan-out.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Can Step Functions call a GraphQL API without a Lambda function?</h3>



<p class="wp-block-paragraph">Yes. An HTTP Task using the <code>arn:aws:states:::http:invoke</code> resource posts your query directly, with an EventBridge connection handling authentication. It works well for small pages. The 60-second request timeout and the state payload size ceiling are what push most real ingestion work back into Lambda.</p>



<h3 class="wp-block-heading">How do I detect GraphQL errors when the status code is always 200?</h3>



<p class="wp-block-paragraph">Inspect the body. If there&#8217;s no <code>data</code> key, nothing executed and you have a request error. If <code>data</code> is present alongside a non-empty <code>errors</code> array, you have a partial response and the data is incomplete. Only a response with <code>data</code> and no errors is unambiguously clean.</p>



<h3 class="wp-block-heading">Does AWS AppSync fit into this?</h3>



<p class="wp-block-paragraph">Only if you&#8217;re on the serving side. AppSync is a managed GraphQL API for exposing your own data, so it&#8217;s what you&#8217;d reach for if you wanted clients to push events in via mutations. For pulling data out of somebody else&#8217;s GraphQL API, AppSync isn&#8217;t part of the picture.</p>



<h3 class="wp-block-heading">Should I store raw GraphQL responses or parsed records in S3?</h3>



<p class="wp-block-paragraph">Both, in separate layers. Raw responses are your audit trail and your reprocessing path, and they&#8217;re the only way to answer &#8220;was this field null in the source, or did my parser drop it?&#8221; weeks after the fact. Parsed and columnar output goes in a curated prefix that analysts query.</p>



<h3 class="wp-block-heading">How do I run an incremental sync from a GraphQL API?</h3>



<p class="wp-block-paragraph">Find the schema&#8217;s filter argument for a modification timestamp using introspection, pass your stored watermark into it, and page forward with cursors. Overlap the window slightly on each run and deduplicate on the primary key downstream. If the schema exposes no such filter, you&#8217;re limited to full pulls or webhook-driven change capture.</p>



<h3 class="wp-block-heading">Why am I throttled when I&#8217;m making very few requests?</h3>



<p class="wp-block-paragraph">Cost-based APIs charge for query complexity, not call count. A single deeply nested query pulling a connection inside a connection can consume a large fraction of the bucket by itself. Read the cost figures the API returns and flatten the query if the requested cost is consistently much higher than the actual cost.</p>



<h3 class="wp-block-heading">Is GraphQL a worse ingestion source than REST?</h3>



<p class="wp-block-paragraph">Different, not worse. You control the payload, which cuts transfer and lets you avoid the N+1 fan-out REST forces on nested data. Deprecation metadata is machine-readable, which REST rarely offers. The cost is that error signalling and rate limiting both moved into the body, so naive clients fail silently instead of loudly.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">A GraphQL data ingestion pipeline on AWS fails differently from a REST one, and the difference is almost entirely about where the truth lives. In REST, the transport tells you what happened. In GraphQL, the transport tells you the request arrived, and the body tells you what happened. Every failure family in this post traces back to that: partial errors hidden behind a 200, throttles hidden behind a 200, cursors that fail by returning the wrong rows rather than an error.</p>



<p class="wp-block-paragraph">So build the pipeline so that a green run means something. Parse before you commit, alarm on row deltas rather than exit codes, and keep the raw responses so you can prove what the source actually said. Everything else is ordinary AWS plumbing.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help with a GraphQL ingestion pipeline?</h2>



<p class="wp-block-paragraph">This is the kind of work I take on regularly. Specifically:</p>



<ul class="wp-block-list">
<li>Auditing an existing GraphQL sync for silent partial-error loss and telling you which columns are already affected</li>

<li>Building the fetch, checkpoint and land loop in Step Functions and Lambda with cursors that survive restarts</li>

<li>Fitting a cost-aware throttle to whatever budget model your source actually uses, including multi-worker coordination</li>

<li>Designing the raw and curated S3 layout, Glue table definitions and Athena partitioning so queries stay cheap</li>

<li>Adding introspection diffing to CI so schema deprecations surface in a pull request, not in a dashboard</li>

<li>Wiring CloudWatch metrics and alarms on row-count deltas and partial-error rates, so a green run is actually green</li>
</ul>



<p class="wp-block-paragraph">Send me a run log, a state machine definition, or a raw response with an <code>errors</code> array in it, and I&#8217;ll tell you what it&#8217;s costing you.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/graphql-data-ingestion-pipeline-aws/">Building a GraphQL Data Ingestion Pipeline on AWS That Doesn&#8217;t Lie to You</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>AWS Amplify vs Firebase: Choosing a Backend You Can Still Leave</title>
		<link>https://john-nessime.com/blog/devops/aws-amplify-vs-firebase/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Mon, 10 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[AppSync]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Amplify]]></category>
		<category><![CDATA[BaaS]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Cognito]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[Deployment]]></category>
		<category><![CDATA[DynamoDB]]></category>
		<category><![CDATA[Firebase]]></category>
		<category><![CDATA[Firestore]]></category>
		<category><![CDATA[GraphQL]]></category>
		<category><![CDATA[Schema Design]]></category>
		<category><![CDATA[Serverless]]></category>
		<category><![CDATA[Vendor Lock-In]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=177</guid>

					<description><![CDATA[<p>Both platforms look identical on day one: a schema file, a typed client, a deploy that works. The difference shows up months later, in the queries you cannot afford and the layer you cannot leave. A practical comparison of data models, billing shape, auth stickiness and exit cost.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/aws-amplify-vs-firebase/">AWS Amplify vs Firebase: Choosing a Backend You Can Still Leave</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The ticket that ends the honeymoon usually looks harmless. Something like: <em>show me last quarter&#8217;s orders, grouped by region, filtered to three statuses, sorted by value</em>. Someone picks it up, opens the backend, and works out that there is no way to answer it without either pulling the whole collection down to the client or standing up a second database next to the first one.</p>



<p class="wp-block-paragraph">That is the moment the <strong>AWS Amplify vs Firebase</strong> decision actually gets made. Not on day one, when both platforms look like a schema file, a typed client and a deploy that just works. On day two hundred, when the read path you committed to without noticing starts charging rent.</p>



<p class="wp-block-paragraph">This post is a comparison, but not a feature tour. Both vendors publish good feature tables and neither of them is lying. What they do not tell you is which parts of the platform you can walk away from cheaply and which parts you are stuck with. So this is organised around the four things that decide that: the data model, the billing shape, the auth layer, and the exit cost.</p>



<h2 class="wp-block-heading">The failure mode that bites: your read path is decided on day one</h2>



<p class="wp-block-paragraph">Both platforms hand you a pleasant abstraction over a NoSQL store. Firebase gives you Firestore, a document database. Amplify&#8217;s data layer, by default, gives you AppSync sitting in front of DynamoDB. In both cases you write a schema, get a typed client, and start shipping.</p>



<p class="wp-block-paragraph">Here is what neither schema file makes obvious: it is not a description of your data. It is a bet on the queries you will need. Change the bet later and you are not editing a schema, you are backfilling.</p>



<p class="wp-block-paragraph">In Firestore, every query shape needs an index that supports it. Filter on two fields and sort on a third and you need a composite index. The console will helpfully offer to create it for you, which is why most teams never notice they are accumulating a per-query-shape index budget. There are no joins. If an order needs its customer&#8217;s name, either you denormalise the name into the order document or you do a second fetch per row.</p>



<p class="wp-block-paragraph">In Amplify, relationships between models are resolved by AppSync, which usually means an extra round trip per relationship. Secondary indexes have to be declared in the schema so the underlying table can be built to serve them. Same constraint, different vocabulary.</p>



<p class="wp-block-paragraph">The good news is that both vendors have finally admitted this is a problem. Firebase Data Connect puts a managed Cloud SQL for PostgreSQL instance behind a GraphQL schema and generates typed SDKs against it. On the AWS side, Amplify&#8217;s data layer can be pointed at a SQL database rather than DynamoDB, and the newer Blocks approach lets you compose Postgres and other capabilities into an existing Amplify backend. If your application has anything resembling reporting, joins or ad-hoc filtering in its future, start there rather than starting on documents and migrating under pressure.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">AWS Amplify: where it wins and where it doesn&#8217;t</h2>



<p class="wp-block-paragraph">Amplify Gen 2 defines the whole backend in TypeScript. Auth, data, storage and functions all live as code in an <code>amplify/</code> directory, get reviewed in pull requests, and compile down through CDK to CloudFormation.</p>



<pre class="wp-block-code"><code>// amplify/data/resource.ts
import { type ClientSchema, a, defineData } from '@aws-amplify/backend';

const schema = a.schema({
  Order: a
    .model({
      region: a.string().required(),
      status: a.enum(['PENDING', 'SHIPPED', 'CANCELLED']),
      total: a.float(),
    })
    .authorization((allow) =&gt; [allow.owner()]),
});

export type Schema = ClientSchema&lt;typeof schema&gt;;

export const data = defineData({ schema });</code></pre>



<p class="wp-block-paragraph">Two things in that snippet matter more than they look. The <code>authorization</code> call is not a comment about intent, it generates the actual resolver-level access rules, so an access change is a code change with a diff. And <code>ClientSchema</code> flows the backend types into the frontend, which means removing a field breaks the build rather than breaking production at 3am.</p>



<p class="wp-block-paragraph">Development happens in a per-developer cloud sandbox. It watches the <code>amplify/</code> folder and redeploys on save:</p>



<pre class="wp-block-code"><code># Deploy and watch your own isolated backend stack
npx ampx sandbox

# Tear it down when you're finished - it is real infrastructure
npx ampx sandbox delete</code></pre>



<h3 class="wp-block-heading">Where Amplify wins</h3>



<ul class="wp-block-list">
<li><strong>You are already on AWS.</strong> If your organisation has accounts, IAM boundaries, a VPC design and a compliance story, Amplify slots into all of it. Firebase would mean a second cloud to audit.</li>

<li><strong>The escape hatch is real.</strong> Because the backend compiles to CDK, you can drop into raw constructs and attach any AWS service without leaving the framework. That ceiling is much higher than Firebase&#8217;s.</li>

<li><strong>Environments are branches.</strong> Git branch to deployed environment is the native model, so staging is not a thing you build, it is a thing you push.</li>

<li><strong>Type safety end to end.</strong> This genuinely catches a class of bug that Firebase projects tend to find in production.</li>
</ul>



<h3 class="wp-block-heading">Where Amplify doesn&#8217;t</h3>



<ul class="wp-block-list">
<li><strong>Gen 1 is on a clock.</strong> Amplify Gen 1 is in maintenance mode and reaches end of life on 1 May 2027. This matters because a lot of the tutorials, Stack Overflow answers and agency codebases you will encounter are Gen 1, and Gen 1 to Gen 2 is a migration, not an upgrade. AWS ships tooling for it, but the guide describes a blue/green run with an irreversible refactor step. Budget for it properly.</li>

<li><strong>&#8220;Local&#8221; development is not local.</strong> The sandbox is a real CloudFormation stack in a real account. It is high fidelity, which is the point, but your inner loop now includes deploy time and your dev environment costs money.</li>

<li><strong>Errors surface as AWS errors.</strong> A mistake in a TypeScript schema can come back as a CloudFormation rollback referencing a resource you never named. Debugging Amplify means being comfortable reading AppSync, DynamoDB and CloudWatch.</li>

<li><strong>Smaller community.</strong> Fewer answers exist, and a meaningful share of the ones that do exist are for the previous generation.</li>
</ul>



<h2 class="wp-block-heading">Google Firebase: where it wins and where it doesn&#8217;t</h2>



<p class="wp-block-paragraph">Firebase&#8217;s advantage is not that it is simpler. It is that the whole loop, from writing a rule to seeing a client behave, is shorter. The emulator suite runs the pieces on your machine:</p>



<pre class="wp-block-code"><code># Run auth, Firestore and functions locally, no cloud project touched
firebase emulators:start --only auth,firestore,functions</code></pre>



<p class="wp-block-paragraph">The thing you will spend the most time on is security rules, and it is worth understanding why they are shaped the way they are:</p>



<pre class="wp-block-code"><code>rules_version = '2';
service cloud.firestore {
  match /databases/{database}/documents {
    match /orders/{orderId} {
      allow read: if request.auth != null
                  &amp;&amp; request.auth.uid == resource.data.ownerUid;

      allow create: if request.auth != null
                    &amp;&amp; request.auth.uid == request.resource.data.ownerUid;

      allow update, delete: if false;
    }
  }
}</code></pre>



<p class="wp-block-paragraph">The distinction people get wrong is <code>resource.data</code> versus <code>request.resource.data</code>. The first is the document as it exists in the database, so it protects the read path. The second is the document the client is trying to write, so it protects the write path. Use the wrong one in a create rule and you are checking ownership on a document that does not exist yet, which evaluates to nothing and quietly lets the write through. That is the kind of bug that ships.</p>



<h3 class="wp-block-heading">Where Firebase wins</h3>



<ul class="wp-block-list">
<li><strong>Mobile.</strong> Crashlytics, Cloud Messaging, Remote Config, A/B testing and analytics are one SDK away and genuinely good. AWS has equivalents, but not assembled like this.</li>

<li><strong>Offline and realtime.</strong> Firestore&#8217;s offline persistence and listener model handle flaky connectivity without you designing for it. If your users are on trains or in warehouses, this is a serious argument.</li>

<li><strong>Emulators.</strong> Being able to run the backend on a laptop, in CI, with no cloud project, is a real productivity difference against Amplify&#8217;s cloud sandbox.</li>

<li><strong>Time to first screen.</strong> For a prototype or an MVP that needs to exist by Friday, Firebase usually wins on speed alone.</li>
</ul>



<h3 class="wp-block-heading">Where Firebase doesn&#8217;t</h3>



<ul class="wp-block-list">
<li><strong>Configuration drifts out of git.</strong> The console is convenient and it is also a second source of truth. Indexes get created from an error link, a provider gets enabled by hand, and six months later nobody can rebuild the project from the repository. Terraform and the Firebase CLI both help, but you have to decide to use them.</li>

<li><strong>Security rules are a language, not a checkbox.</strong> They are also evaluated on the server for every access, which has cost implications when rules perform document lookups.</li>

<li><strong>Features you assume are included are an upgrade.</strong> Multi-factor auth, SAML and OIDC, blocking functions, audit logging and multi-tenancy live behind the Identity Platform upgrade, which comes with a different pricing model. On the free Spark plan, upgrading also introduces a daily active user cap. Find this out during design, not during an enterprise sales cycle.</li>

<li><strong>The ceiling is lower.</strong> When you outgrow Firebase you are usually moving to Google Cloud proper, which is a different set of tools and a different mental model.</li>
</ul>



<h2 class="wp-block-heading">What actually generates the bill</h2>



<p class="wp-block-paragraph">Rates change, so treat published numbers as current-at-time-of-reading and go to the vendor pricing pages for the arithmetic. What is stable, and what you should actually reason about, is the <em>shape</em> of the meters.</p>



<p class="wp-block-paragraph">Firestore charges per document, and the details are where teams get caught:</p>



<ul class="wp-block-list">
<li>A query that returns 200 documents is 200 reads, whether you use two fields from each one or forty.</li>

<li>A realtime listener is charged a read each time a document in its result set is added or updated. Attach a listener to a busy collection and you have built a meter that runs while nobody is looking at the screen.</li>

<li>Where your SDK exposes query offsets, skipped documents are still billed. Paginate with cursors, not offsets.</li>

<li>Aggregations such as <code>count()</code> are billed per batch of index entries read, with a minimum of one document read. Cheap, but not free, and not zero when you run them per page load.</li>

<li>Stored bytes include indexes and metadata, so index sprawl shows up twice: once in write amplification, once in storage.</li>
</ul>



<p class="wp-block-paragraph">Amplify has no single meter, because Amplify is not a service. The bill is assembled from AppSync (query and mutation operations, realtime update messages, and connection-minutes for open WebSockets), DynamoDB, Lambda invocations and duration, CloudWatch log ingestion and retention, Cognito monthly active users, and Amplify Hosting build minutes plus data served.</p>



<p class="wp-block-paragraph">That difference is the practical one. Firebase gives you a bill you can read in the console in thirty seconds. AWS gives you eight line items that will not tell you which feature caused them unless you tagged resources properly on the way in. If you go the AWS route, set up cost allocation tags and a budget alarm before launch, not after the first invoice. Pushing CloudWatch metrics into <a href="https://grafana.com/" target="_blank" rel="noreferrer noopener">Grafana</a> alongside your application metrics is worth the afternoon it takes, because cost anomalies and traffic anomalies are usually the same incident viewed twice.</p>



<h2 class="wp-block-heading">Auth is the component you cannot cheaply replace</h2>



<p class="wp-block-paragraph">Firebase Authentication and Amazon Cognito are both fine. Neither is the reason to pick a platform. But auth is the layer where switching hurts most, because you are not migrating data, you are migrating <em>sessions and identities</em>, and every user notices.</p>



<p class="wp-block-paragraph">Both let you export users, including password hashes, so a migration is technically possible. What is hard is everything hanging off the user ID: rules that compare against it, resolver authorisation that assumes it, third-party integrations keyed on it, and support tooling that looks users up by it.</p>



<p class="wp-block-paragraph">One habit removes most of that pain regardless of which platform you pick: store your own internal user record from the first commit, with your own primary key, and treat the provider&#8217;s UID as an external identifier on that record. It costs you one table and one lookup. It converts an auth migration from a rewrite into a mapping exercise.</p>



<h2 class="wp-block-heading">Hosting is the layer nobody should agonise over</h2>



<p class="wp-block-paragraph">Amplify Hosting deploys from a git branch and supports server-rendered frameworks. Firebase now has two: the original Hosting for static sites, and App Hosting for modern server-rendered apps, which is generally available and runs on Cloud Run with Cloud CDN in front.</p>



<p class="wp-block-paragraph">Both are good. Neither should influence your decision much, because hosting is the one layer with genuine competition and near-zero switching cost. A Next.js or Nuxt app moves to Vercel, Netlify, Cloudflare Pages or a container on a VPS from a provider like <a href="https://www.interserver.net/" target="_blank" rel="noreferrer noopener">InterServer</a> or DigitalOcean in an afternoon. Put a CDN such as <a href="https://www.cloudflare.com/" target="_blank" rel="noreferrer noopener">Cloudflare</a> in front and the origin becomes an implementation detail your users never see.</p>



<p class="wp-block-paragraph">Spend your deliberation budget on the data layer instead. That is where it pays.</p>



<h2 class="wp-block-heading">Exit cost, ranked</h2>



<p class="wp-block-paragraph">If you rank the layers by what it costs to leave, the ordering is the same on both platforms and it is the opposite of the ordering people use when choosing:</p>



<ol class="wp-block-list">
<li><strong>Hosting.</strong> Swap it in a day. Barely counts as lock-in.</li>

<li><strong>Functions.</strong> Cloud Functions and Lambda handlers are mostly your own code with a different signature wrapped around it. A rewrite, but a bounded one.</li>

<li><strong>Auth.</strong> A user migration project with a communications plan attached.</li>

<li><strong>The data model.</strong> You do not migrate this. You rebuild the read path. Firestore&#8217;s managed export produces Firestore-format files in Cloud Storage, and DynamoDB exports to S3, and in both cases what lands is denormalised documents shaped by decisions you made two years ago, not a relational dataset you can point a reporting tool at.</li>
</ol>



<p class="wp-block-paragraph">There is a fifth item with no clean export at all: realtime. Firestore listeners and AppSync subscriptions have no portable equivalent, so if realtime is core to your product, whichever you choose is close to permanent. That is not automatically an argument against either. It is an argument for knowing you are making that commitment when you make it.</p>



<h2 class="wp-block-heading">AWS Amplify vs Firebase: how I&#8217;d actually decide</h2>



<p class="wp-block-paragraph">Not a scorecard. A procedure, in order, stopping at the first clear answer:</p>



<ol class="wp-block-list">
<li><strong>Write down your three hardest read queries.</strong> Not the CRUD. The reporting screen, the search, the admin filter. If you cannot write them, you are not ready to choose a backend.</li>

<li><strong>Do any of them need joins, aggregates or filters you cannot predict?</strong> If yes, start relational: Firebase Data Connect, Amplify over a SQL data source, or plain Postgres behind an API you own. Do not start on documents planning to fix it later.</li>

<li><strong>Is this mobile-first with real offline requirements?</strong> Firebase, and it is not close.</li>

<li><strong>Does your organisation already run on AWS?</strong> Existing accounts, IAM, VPCs, audit posture and a security team who will ask questions about a second cloud all point to Amplify, even if Firebase would be faster this quarter.</li>

<li><strong>Who is maintaining this in two years?</strong> Amplify Gen 2 expects TypeScript fluency and some tolerance for CDK and CloudFormation. Firebase expects less. Match the platform to the team you will actually have.</li>

<li><strong>Any data residency or regulatory constraints?</strong> Check region availability for every service you plan to use, not just the headline one, before you commit.</li>

<li><strong>Still tied?</strong> It is a prototype. Pick the one your team can ship this week, and write down explicitly that the data model is the part you will have to revisit.</li>
</ol>



<h2 class="wp-block-heading">Mistakes I see repeatedly</h2>



<ul class="wp-block-list">
<li><strong>Choosing on SDK ergonomics.</strong> Both SDKs are pleasant. You are picking a data model and a billing model, and the SDK is the layer you would replace most easily.</li>

<li><strong>Building reporting on the transactional store.</strong> Both platforms punish this. Stream to a warehouse or a read replica and let the operational store do one job.</li>

<li><strong>Launching without a budget alarm.</strong> A missing <code>limit()</code>, a listener on a busy collection, or one post that gets traction can turn a hobby bill into a real one overnight. Set the alarm before you need it.</li>

<li><strong>Treating authorisation as a later task.</strong> On both platforms, access control is coupled to the data model. Retrofitting it usually means reshaping documents.</li>

<li><strong>Following Amplify Gen 1 material.</strong> The commands, the directives and the mental model are all different. Check which generation a tutorial targets before you follow it.</li>

<li><strong>Letting console clicks become infrastructure.</strong> Anything you enabled by hand is something you cannot rebuild. Get it into the CLI config or into Terraform.</li>

<li><strong>Assuming the free tier is the plan.</strong> It is a trial of the plan. Model your costs at ten times current traffic and see whether you still like the answer.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Is AWS Amplify or Firebase cheaper?</h3>



<p class="wp-block-paragraph">It depends entirely on your access patterns, and anyone who answers this without seeing them is guessing. Firebase tends to be cheaper and far more predictable at small scale, partly because the free allowances are generous and partly because there is one meter to watch. AWS tends to win when you have steady, high-volume traffic and the engineering discipline to tune DynamoDB capacity and Lambda sizing. The bigger practical difference is legibility: a Firebase bill tells you what happened, an AWS bill tells you what was consumed.</p>



<h3 class="wp-block-heading">Can I move from Firebase to AWS Amplify later?</h3>



<p class="wp-block-paragraph">Yes, but price it by layer rather than as one project. Hosting and functions move cheaply. Auth is a real migration with user impact. The data model is usually a rebuild rather than a move, because a Firestore export gives you denormalised documents, not a relational dataset. Teams that manage this well are the ones that kept business logic out of security rules and out of client code from the beginning.</p>



<h3 class="wp-block-heading">Is Amplify Gen 1 still supported?</h3>



<p class="wp-block-paragraph">Gen 1 is in maintenance mode and reaches end of life on 1 May 2027. It receives critical fixes and security patches, not new features. If you are starting something new, use Gen 2. If you have a Gen 1 application in production, schedule the migration deliberately using the official tooling and guide, and test thoroughly before the refactor step, which is difficult to reverse.</p>



<h3 class="wp-block-heading">Do I need Firebase Data Connect, or is Firestore enough?</h3>



<p class="wp-block-paragraph">Firestore is enough when your reads are predictable, mostly key-based, and denormalise cleanly. The moment you need joins, aggregates across collections, or filters that product will keep changing, Data Connect&#8217;s managed Postgres is the better foundation. It is also a lot easier to adopt at the start than to migrate onto after a year of document design.</p>



<h3 class="wp-block-heading">Which one is better for a mobile app?</h3>



<p class="wp-block-paragraph">Firebase, in most cases. The offline persistence model, the crash reporting, the push messaging and the remote configuration are all mature and designed to work together. Amplify has mobile libraries and they work, but you are assembling the surrounding pieces from separate AWS services rather than getting them as a set.</p>



<h3 class="wp-block-heading">What about a self-hosted alternative?</h3>



<p class="wp-block-paragraph">Worth considering if lock-in is your main worry. Supabase and Appwrite both offer a comparable developer experience over Postgres, and either can run on your own infrastructure. Understand what you are taking on, though: backups, restore testing, upgrades, patching and monitoring all become yours. That is a real operational load, and it is the trade most teams are implicitly paying a managed vendor to avoid.</p>



<h3 class="wp-block-heading">Can I use both together?</h3>



<p class="wp-block-paragraph">You can, and some teams do run Firebase for analytics and messaging alongside an AWS backend. Be honest about the cost: two identity models, two billing accounts, two audit trails and two on-call runbooks. Do it because one platform is clearly better at a specific job, not because it was easier than making a decision.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">The <strong>AWS Amplify vs Firebase</strong> question is almost never settled by features, and it is never settled by which SDK feels nicer in a tutorial. Both platforms will get you to a working application quickly, and both will hold up in production for the workloads they were designed for.</p>



<p class="wp-block-paragraph">What separates them, months later, is the read path you committed to on day one and the layer you cannot afford to leave. Write down your three hardest queries before you write any schema. Decide whether you need relational storage while it is still a decision rather than a migration. Keep your own user record from the first commit. Set a budget alarm before launch.</p>



<p class="wp-block-paragraph">Do those four things and either platform is a reasonable choice. Skip them and neither one saves you.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second opinion before you commit?</h2>



<p class="wp-block-paragraph">Backend platform decisions are cheap to review and expensive to reverse. Things I help with in this area:</p>



<ul class="wp-block-list">
<li>Reviewing a Firestore or DynamoDB data model against the queries you actually need, before the design hardens</li>

<li>Cost modelling both platforms against your real traffic assumptions, with the meters broken out so you can see what drives the bill</li>

<li>Amplify Gen 1 to Gen 2 migration planning, including the blue/green sequence and what to verify before the irreversible step</li>

<li>Auditing Firestore security rules and Amplify authorisation rules for gaps between the read path and the write path</li>

<li>Setting up cost allocation tags, budget alarms and CloudWatch or Grafana dashboards so a runaway query shows up in hours rather than on the invoice</li>

<li>Exit planning: working out what a migration off your current platform would genuinely cost, layer by layer</li>
</ul>



<p class="wp-block-paragraph">If you want a concrete answer rather than a general one, send me the thing itself: your schema file, your security rules, a screenshot of last month&#8217;s billing breakdown, or the three queries you are worried about. That is usually enough to tell you something useful.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/aws-amplify-vs-firebase/">AWS Amplify vs Firebase: Choosing a Backend You Can Still Leave</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
