The frustrating part is that the pipeline is fine. Webhooks are arriving, Lambda is running clean, S3 has the files, Athena returns rows. Nothing is broken in the way monitoring understands “broken”. The pipeline is faithfully recording something that is no longer true.

This post is about building a Shopify sales dashboard with AWS that survives that conversation. Not the wiring, which is well documented and mostly straightforward, but the design decisions that determine whether your numbers still hold up six months in. I’ll cover the three ingestion paths and when each one is the right call, why append-only pipelines drift, how to lay out S3 and Athena so recomputation is cheap, and what to do when the totals are already wrong.

Why a Shopify sales dashboard with AWS drifts from the admin

Here’s the thing that catches almost everyone: a Shopify order is not an event, it’s a mutable record.

An event pipeline assumes facts are immutable once written. A payment happened. A shipment left. You append it, you never touch it again, and the sum of the log is the truth. That model is why streaming architectures are so clean, and it’s exactly wrong for order data.

An order created on Monday can be edited on Tuesday, partially refunded on Friday, and fully refunded three weeks later. Every one of those changes belongs, financially, to Monday. If your pipeline appends the orders/create payload and never revisits it, Monday’s revenue is frozen at the moment of checkout and it will only ever be too high.

This is the invisible failure. Nothing alerts. No queue backs up. Your dashboard is confidently wrong, and the gap widens roughly in proportion to your return rate. A store with a two percent return rate takes a long time to notice. A fashion store running thirty percent returns notices in about a month, usually via an angry accountant.

The four adjustments that move historical numbers

• Refunds. Full or partial. A refund carries its own created_at, which is when the money moved back. The order it belongs to has a different, earlier date. You need both, and which one you attribute to depends on whether finance wants cash-basis or order-basis reporting. Ask before you build.
• Order edits. A merchant adds a line item or adjusts a quantity after the fact. The original payload is now stale. Shopify exposes both the original and the current totals precisely because of this.
• Cancellations. A cancelled order keeps existing in the API. If you filter only on payment status you will happily keep counting it.
• Test and draft orders. Test orders carry a flag marking them as such. Nobody remembers to filter these until a QA run during a quiet week produces a suspicious spike.

The design consequence is simple to state and annoying to implement: your pipeline must be able to recompute any past day. Every storage and partitioning decision below follows from that one requirement.

Getting data out of Shopify: three paths, three trade-offs

Before anything else: new Shopify apps are built on the GraphQL Admin API. The REST Admin API has been designated a legacy API and new public apps must use GraphQL. If you’re starting fresh, start there. If you inherited a REST integration, it probably still runs, but you’re on borrowed time and you should plan the migration rather than discover the deadline.

Path 1: EventBridge partner event source

Shopify can deliver webhooks straight into an Amazon EventBridge partner event bus in your account. No public endpoint, no API Gateway, no HMAC verification code, because verification only applies to HTTPS deliveries. Shopify’s own docs confirm EventBridge and Pub/Sub deliveries skip it.

You create the source in the Shopify app configuration using your AWS account ID, region and a source name, then associate it with an event bus in the EventBridge console and write rules to route it. The address you register with Shopify is the partner event source ARN, not the event bus ARN. That distinction accounts for a large share of the “I set it up and nothing arrives” threads on the Shopify forums.

A rule matching everything from the Shopify partner source looks like this. Start broad, then narrow once you’ve seen the real shape of an event:

{
  "source": [ { "prefix": "aws.partner/shopify.com" } ]
}

Send that to an SQS queue with a dead-letter queue attached rather than straight to Lambda. Buffering gives you a replay buffer when a downstream deploy goes wrong, and the DLQ means a bad payload parks itself instead of poisoning the whole rule. This is the path I reach for first for anything already on AWS.

Path 2: HTTPS webhooks into API Gateway and Lambda

The conventional route, and the right one if you need webhook delivery outside AWS too, or you want the payloads to pass through something you fully control. The cost is that you now own an internet-facing endpoint and the HMAC verification on it.

Verify against the raw request body, before any JSON parsing. Re-serialising the payload changes byte-for-byte content and the signature will never match. Use a constant-time comparison so the check doesn’t leak timing information:

import base64, hashlib, hmac

def verify(raw_body: bytes, header_hmac: str, secret: str) -> bool:
    digest = hmac.new(secret.encode(), raw_body, hashlib.sha256).digest()
    computed = base64.b64encode(digest).decode()
    return hmac.compare_digest(computed, header_hmac)

Shopify sends the signature in the X-Shopify-Hmac-SHA256 header, base64-encoded, computed with your app’s client secret over the raw body. Store that secret in Secrets Manager or as an SSM SecureString parameter, not in a Lambda environment variable.

Acknowledge fast. Shopify’s timeout is short and it retries with backoff over a finite window, so a handler that does real work inline will generate a wall of duplicate deliveries during a flash sale, exactly when you can least afford it. Return 2xx immediately, do the work asynchronously.

Path 3: scheduled GraphQL bulk pull

Webhooks give you low latency. They do not give you completeness. Anything that fails past its retry window is gone, and Shopify will eventually remove a subscription that keeps failing. That’s a silent data loss mode with no local symptom at all.

So run a scheduled reconciliation pull alongside the stream. Shopify’s GraphQL bulk operations are built for this: you submit a query, it runs asynchronously, and you fetch a JSONL result file when it finishes. That’s the right tool for backfills and nightly catch-up, rather than paginating thousands of pages against a points-based rate limiter and getting throttled halfway through.

A nightly job that re-pulls the last seven to fourteen days and overwrites those partitions costs almost nothing and quietly fixes every category of drift described above. If you build one thing from this post, build that.

Decide what “revenue” means before you write a line of SQL

This is where most reconciliation arguments actually live, and it isn’t an engineering problem at all until you’ve had the conversation.

Shopify’s own sales reporting builds total sales from gross sales, minus discounts, minus returns, plus taxes and shipping. Gift card sales sit outside that in a separate finance report. If your dashboard sums order totals and calls it revenue, you have built a different metric with the same name, and it will disagree with the admin forever no matter how good your pipeline is.

Write the definition down. Put it in the dashboard as a tooltip. When someone challenges a number, you want the argument to be about the definition, not about whether your infrastructure works.

The currency trap

If the store sells in more than one currency, the money fields split in two. Shopify exposes totals as a set containing both shop_money and presentment_money: the amount in the store’s base currency, and the amount the customer actually saw and paid.

Sum the presentment amounts across a multi-currency store and you get a number with no meaning at all, euros and yen added together as if they were the same unit. For a single reporting figure you want the shop-currency side. Keep the presentment amount and its currency code in the table anyway, because the day someone asks “how much did we actually sell in Germany”, you’ll want it and it is painful to backfill.

One caveat worth knowing: orders created through the API rather than through checkout can behave differently from native multi-currency checkout orders. If your store takes orders from an ERP or a marketplace integration, spot-check a few of those specifically.

Storage layout: partition by order date, never by arrival date

Two layers in S3. Keep them separate and keep them honest about what they are.

1. Raw. Every payload exactly as received, partitioned by ingestion date. Append-only, never edited. This is your audit trail and your rebuild source. Lifecycle it to a colder storage class after a few months, don’t delete it.
2. Curated. One row per order representing current state, in Parquet, partitioned by order date. This is what the dashboard queries. It is derived, disposable and rewritable.

The partitioning choice on the curated layer is the load-bearing decision in the whole design. If you partition by arrival date, which is what Amazon Data Firehose does by default because it buckets on the moment it writes the file, then a refund that arrives three weeks late lands in today’s partition. Correcting Monday now means finding and rewriting fragments scattered across twenty other partitions. Partitioned by order date, correcting Monday means overwriting exactly one prefix.

Firehose can do this with dynamic partitioning, which routes records by keys inside the payload rather than by write time. If you’re not using Firehose, extract the order date in your Lambda and write the prefix yourself.

Use partition projection so Athena stops guessing

The default Glue Data Catalog approach means running a crawler or issuing MSCK REPAIR TABLE to register new partitions. Forget one and you get a query that silently returns nothing for recent days. Nobody notices until Monday.

Partition projection removes the metastore lookup entirely. You tell Athena the shape of the partition keys and it calculates the prefixes at query time:

CREATE EXTERNAL TABLE shop_orders (
  order_id             bigint,
  order_number         string,
  created_at           timestamp,
  financial_status     string,
  cancelled_at         timestamp,
  is_test              boolean,
  total_shop           decimal(12,2),
  shop_currency        string,
  total_presentment    decimal(12,2),
  presentment_currency string
)
PARTITIONED BY (order_date string)
STORED AS PARQUET
LOCATION 's3://your-bucket/curated/orders/'
TBLPROPERTIES (
  'projection.enabled' = 'true',
  'projection.order_date.type' = 'date',
  'projection.order_date.format' = 'yyyy-MM-dd',
  'projection.order_date.range' = '2019-01-01,NOW',
  'projection.order_date.interval' = '1',
  'projection.order_date.interval.unit' = 'DAYS',
  'storage.location.template' =
    's3://your-bucket/curated/orders/order_date=${order_date}/'
);

Set the range start to your store’s actual first order month. Projection generates every prefix in the range, so a range starting a decade too early makes wide scans slower for no benefit.

Parquet matters here for the same reason. Athena bills on bytes scanned, so a columnar format with good compression cuts the bill directly, and a dashboard that only ever selects six columns from a forty-column table never touches the rest.

Net sales in one query

With refunds in their own table keyed by order and carrying their own date, attributing them back to the original order day is a left join and a subtraction:

SELECT
    o.order_date,
    SUM(o.total_shop)                                AS gross_shop,
    SUM(COALESCE(r.refunded_shop, 0))                AS refunded_shop,
    SUM(o.total_shop - COALESCE(r.refunded_shop, 0)) AS net_shop
FROM shop_orders o
LEFT JOIN (
    SELECT order_id, SUM(amount_shop) AS refunded_shop
    FROM shop_refunds
    GROUP BY order_id
) r ON r.order_id = o.order_id
WHERE o.order_date BETWEEN '2025-01-01' AND '2025-01-31'
  AND o.is_test = false
  AND o.cancelled_at IS NULL
GROUP BY o.order_date
ORDER BY o.order_date;

Note the two filters doing quiet work at the bottom. Those two lines are the difference between a number finance accepts and a number they don’t.

Choosing the dashboard layer

Once the data is correct, this part is genuinely a preference. All of these work.

• Amazon QuickSight, now delivered as part of Amazon Quick Suite, is the least-friction option if you’re already in AWS. Its in-memory SPICE layer means viewers aren’t firing an Athena query per chart interaction, which controls both latency and scan cost. Per-viewer pricing tends to be the deciding factor either way, so model it for your actual audience size before committing.
• Grafana with the Athena data source is a good fit if you’re already running Grafana for infrastructure and want commercial and operational panels on one screen. Grafana Cloud removes the hosting question if you’d rather not run it.
• Power BI makes sense when the finance team already lives in Microsoft 365 and models in DAX. The cross-cloud hop is real but manageable.
• Metabase or a self-hosted alternative on a small VPS from a provider like InterServer or Hetzner is the pragmatic answer for a handful of internal viewers, where per-seat BI licensing costs more than the entire pipeline.

The honest trade-off: managed BI costs more per month and saves you from becoming the person who patches the reporting server. Self-hosting inverts that. Neither is wrong, but pick deliberately rather than by inertia.

Troubleshooting: symptom to cause

Totals are consistently higher than the Shopify admin

Almost always refunds, cancellations or test orders. Check in that order. If the gap grows with the age of the reporting window, it’s refunds. If it’s a fixed offset on specific days, look for test orders or a QA run.

Totals are lower, and recent days are missing rows

Either partitions aren’t registered, which projection fixes permanently, or the webhook subscription has been dropped after repeated delivery failures. Check the subscription still exists before you go digging through Lambda logs. A nightly bulk pull would have masked this, which is another argument for having one.

Orders appear twice

Shopify’s delivery model is at-least-once, not exactly-once, and you may also have more than one subscription on the same topic. Deduplicate on the delivery ID header before you touch anything else, and make the write itself idempotent so a duplicate is a no-op rather than a second row.

Numbers are right on the daily view, wrong on the monthly

Timezone. Order timestamps carry an offset; your partition key is a date string. If you derive the date in UTC and the store reports in a local timezone, orders near midnight land on the wrong day. That averages out over a month, which is exactly why the discrepancy hides until month boundaries.

Athena costs jumped without more data

Someone built a dashboard with a filter that doesn’t hit the partition column, so every panel refresh scans the full table. Look at bytes scanned per query and check whether the BI tool is caching results or re-querying on every interaction.

Common mistakes

• Treating orders as immutable events and never revisiting a past day.
• Partitioning on arrival time because that’s the default, then discovering corrections are expensive.
• Verifying the HMAC against a re-serialised body instead of the raw bytes.
• Doing real work inside the webhook handler, generating duplicates under load.
• Summing presentment amounts across currencies.
• Registering the event bus ARN with Shopify instead of the partner event source ARN.
• Relying on webhooks alone with no scheduled reconciliation.
• Shipping a “revenue” number without ever defining what it includes.

Best practices

• Keep raw and curated layers separate. Raw is append-only; curated is rewritable.
• Make every partition idempotently rebuildable from raw. Test that path deliberately, before you need it.
• Buffer through SQS with a dead-letter queue. Free replay, free isolation of bad payloads.
• Run a nightly bulk pull over a rolling window and overwrite those partitions.
• Use partition projection. It removes an entire category of silent failure.
• Alarm on the absence of events, not just on errors. A CloudWatch alarm on zero orders processed in an hour during business hours catches broken subscriptions the same day.
• Store the API secret in Secrets Manager and scope the Lambda role to the exact prefixes it writes.
• Publish a reconciliation panel comparing your total to the admin’s for the same window. Surfacing the gap builds more trust than hiding it.

Frequently asked questions

Do I need a data warehouse, or is S3 and Athena enough?

For a single store’s order data, S3 with Athena is almost certainly enough, and it’s cheaper because you pay per query rather than for a running cluster. Redshift starts to earn its place when you’re joining Shopify data against several other large sources, or when concurrent query load makes Athena’s queue times noticeable.

How near-real-time can this be?

Events land within seconds. The practical floor is your buffering window, since writing one tiny file per order gives you a small-files problem that ruins query performance. A few minutes of buffering is the usual compromise. If you genuinely need sub-minute order counts, put a live counter in DynamoDB alongside the analytical pipeline rather than trying to make the data lake do both jobs.

Which webhook topics should I subscribe to?

At minimum, order creation, order update, order cancellation and refund creation. Update and refund topics are the ones people skip, and they’re exactly the ones carrying the corrections. Subscribe to fewer topics than you think you need and add rather than subscribing to everything, since every extra topic is volume you pay to store and process.

Can I skip AWS and use a connector tool?

Yes, and for many stores that’s the right answer. A managed connector into a hosted warehouse gets you a working dashboard in an afternoon. You’re paying a monthly fee to avoid owning any of this, and trading away control over the data model. Building it on AWS wins when you need Shopify data joined to systems the connector doesn’t cover, or when row-based connector pricing outgrows the infrastructure cost.

How do I backfill historical orders?

Use a GraphQL bulk operation rather than paginating the API. Submit the query, poll for completion, then stream the JSONL result into your raw bucket and run the same transformation your live pipeline uses. If backfill and live processing use different code paths, they will diverge, and you’ll spend an afternoon working out which one is lying.

What does a setup like this cost to run?

For a typical single store, the pipeline itself is small money: Lambda invocations, a few gigabytes in S3, and Athena billed on bytes scanned, which partitioning and Parquet keep low. The BI seats are usually the largest line item, which is why the dashboard layer decision deserves more thought than the ingestion one. Model it against current published rates rather than trusting any figure you read in a blog post, including this one.

The one thing to take away

A Shopify sales dashboard with AWS doesn’t fail because the pipeline breaks. It fails because the pipeline keeps working perfectly on data that has since changed underneath it.

Design for correction from the first commit. Partition by order date, keep the raw layer so you can always rebuild, run a scheduled pull to catch what the stream missed, and agree on what revenue means before anyone builds a chart. Do that and the Monday morning screenshot comparison becomes a non-event, which is the highest praise a reporting pipeline ever gets.

Need help with your Shopify data pipeline on AWS?

I design and build ecommerce data pipelines and reporting stacks on AWS. Typical engagements look like:

• Working out why an existing Shopify dashboard disagrees with the admin, and fixing the root cause rather than patching the query
• Building the ingestion layer end to end: EventBridge or API Gateway, Lambda, SQS with dead-letter handling, and a scheduled GraphQL bulk reconciliation job
• Designing the S3 layout, Glue schema and Athena tables so past days can be recomputed cheaply and partitions never go missing
• Migrating REST Admin API integrations to GraphQL before the deadline forces the issue
• Building the dashboard itself in QuickSight, Grafana or Metabase, including the metric definitions finance will actually sign off on
• Cutting Athena scan costs and BI licensing on a reporting stack that has grown more expensive than anyone planned

If you’re in the middle of one of these, send me the actual thing: the Athena query, the S3 prefix layout, the two totals that don’t match. It’s a much faster conversation than describing it in the abstract.

The post Shopify Sales Dashboard with AWS: Build One That Actually Reconciles appeared first on John Nessime.

That is where embedded analytics on AWS stops being a front-end task and becomes an architecture decision. The awkward part is that the choice you make first, how the viewer is identified, quietly decides which isolation mechanisms remain available to you afterwards. Get that order backwards and you rebuild the data layer, not the iframe.

This post covers the four decisions that determine whether the build holds: identity model, tenant isolation, query mode, and session economics. Then the embed handshake itself, the errors you will actually see in the browser console, and what to check first when it fails.

Before anything else: the product got renamed

Amazon QuickSight was folded into a broader platform called Amazon Quick Suite, and the BI product inside it is now called Amazon Quick Sight. AWS documentation has since moved again under an “Amazon Quick” umbrella. You will land on all three naming conventions depending on which search result you click, which makes finding the right doc page genuinely annoying.

The practical upshot: the APIs, SDKs and IAM action names did not change. You are still calling quicksight:GenerateEmbedUrlForRegisteredUser against ARNs in the quicksight namespace, and the JavaScript SDK is still published as amazon-quicksight-embedding-sdk. Nothing in your code breaks. Only your bookmarks do. I mention it because half the confusion in a first embedded build comes from following a doc page that describes a UI menu that has since been reorganised.

Decision one: registered users or anonymous sessions

Two API operations generate embed URLs. GenerateEmbedUrlForRegisteredUser issues a session for a user who exists inside the BI account. GenerateEmbedUrlForAnonymousUser issues a session for someone who does not, and never will.

This reads like a convenience choice. It is not. Row-level security using session tags, the mechanism most SaaS products want, is supported only for anonymous embedding. It does not work with GenerateEmbedUrlForRegisteredUser, it does not work with the older GetDashboardEmbedUrl operation, and it is not supported with the IAM identity type. That constraint is documented, easy to miss, and it is the single most expensive thing to discover late.

So the fork is really this. If you register every viewer, you get per-user features (bookmarks, threshold alerts, scheduled snapshots) and you enforce isolation with username or group rules on the dataset. You also inherit the job of provisioning, deprovisioning and reconciling a user directory that mirrors your own. If you go anonymous, you skip all of that and filter with session tags at embed time, but per-user features are off the table because there is no persistent user to hang them on.

The registered-user request body is small. Everything interesting is in ExperienceConfiguration:

POST /accounts/<aws-account-id>/embed-url/registered-user

{
  "UserArn": "arn:aws:quicksight:<region>:<account>:user/default/<user>",
  "SessionLifetimeInMinutes": 60,
  "AllowedDomains": ["https://app.example.com"],
  "ExperienceConfiguration": {
    "Dashboard": {
      "InitialDashboardId": "<dashboard-id>",
      "FeatureConfigurations": {
        "Bookmarks": { "Enabled": true }
      }
    }
  }
}

One trap on the anonymous path that deserves its own sentence. Anonymous sessions belong to a namespace, and any dashboard shared with that namespace is reachable by a session in it, whether or not you listed the dashboard in AuthorizedResourceArns. If you were treating that parameter as your allowlist, it is not. Namespace membership is the real boundary.

Decision two: where tenant isolation actually lives

There are three places you can put the filter, and only one of them scales.

• A dashboard per tenant. Works for five customers. Becomes a deployment problem at fifty and a change-management disaster at five hundred, because every visual fix is now a fan-out.
• A dataset per tenant, filtered in SQL. Better isolation guarantees, genuinely defensible in a compliance review, but you multiply refresh jobs and in-memory footprint by tenant count.
• One dashboard, one dataset, row-level security. The standard answer. One artifact to maintain, filtering applied per session.

With anonymous embedding, RLS is driven by tags. You declare tag keys against columns on the dataset, then supply values at embed time. The filter is evaluated server-side against the session, so a viewer poking at the iframe cannot lift it.

POST /accounts/<aws-account-id>/embed-url/anonymous-user

{
  "Namespace": "default",
  "SessionLifetimeInMinutes": 60,
  "AuthorizedResourceArns": [
    "arn:aws:quicksight:<region>:<account>:dashboard/<dashboard-id>"
  ],
  "SessionTags": [
    { "Key": "tenant_id", "Value": "acme-corp" },
    { "Key": "region",    "Value": "emea" }
  ],
  "AllowedDomains": ["https://app.example.com"],
  "ExperienceConfiguration": {
    "Dashboard": { "InitialDashboardId": "<dashboard-id>" }
  }
}

The value in SessionTags must come from your server-side session, never from a request parameter, a cookie your client can write, or a JWT claim you have not verified. This is the whole security boundary. Tag rules support combining conditions, so a manager who should see several sites is expressible without a second dashboard.

One quiet limit worth knowing before it bites: when RLS is applied to in-memory datasets, each field has a maximum length in Unicode characters, and fields exceeding it are truncated during ingestion rather than rejected. If your tenant identifiers are long opaque strings, test that a truncated value cannot collide with another tenant’s. Silent truncation plus a prefix collision is exactly the kind of bug that produces a cross-tenant data leak with no error anywhere in the logs.

Decision three: SPICE or direct query against your AWS data

Every dataset runs in one of two modes, and the difference shows up on a bill somewhere else in your account.

Direct query sends a live query to the source each time a visual renders. Against Amazon Athena that means an S3 scan per dashboard open, billed by bytes scanned. Against Amazon Redshift it means a concurrent query slot per viewer. Freshness is perfect. The failure mode is that dashboard load is now coupled to warehouse load, and your analytics traffic competes with everything else running there. Two hundred people opening a dashboard at 9am is two hundred queries, and Redshift concurrency is finite.

SPICE imports a snapshot into an in-memory engine and serves every viewer from it. One scan on refresh, then arbitrarily many reads. For an embedded product where the same aggregate is served to thousands of sessions, this is usually the right call, and the Athena cost difference between “scan once per refresh” and “scan once per pageview” is not subtle. What you give up is freshness, bounded by your refresh schedule, plus a capacity dimension to manage and incremental refresh to configure if the dataset is large.

The pattern I reach for first on a data-lake backend is a hybrid: recent partitions in SPICE with an incremental refresh on a look-back window, historical data left on direct query for the rare deep query. It costs more design effort up front and it is the thing most teams skip, but it is the only shape that keeps both the bill and the load time flat as history grows.

Whichever you pick, note that visual generation has a timeout, and data-source-specific timeouts apply on top of it. A query that is merely slow in a console tab renders as a broken visual in a customer’s browser. Model your worst partition, not your average one.

Decision four: what a session actually costs

I am not going to quote figures, because AWS changes them and you should read the current pricing page. The mechanism is what matters, and it is genuinely different from seat-based BI licensing.

• A reader session is a fixed 30-minute window. Not a pageview, not a query. Reopening the dashboard twenty minutes later is still the same session.
• Per-user pricing charges per session with a monthly cap per reader. Predictable when the same people return daily.
• Capacity pricing buys sessions in bulk with no user provisioning at all. This is the model built for embedding, and it is the one that pairs with anonymous sessions.
• Capacity pricing is also the prerequisite for programmatic dashboard refresh, so if near-real-time rendering is a product requirement, that decision is already made for you.
• Annual commitments to capacity unlock removing the “Powered by” attribution footer. If white-labelling is a contractual requirement, factor that in early rather than discovering it during a customer demo.
• Enabling certain Pro-tier and generative Q&A capabilities triggers an account-level monthly infrastructure fee that exists whether or not anyone uses the feature.

The cost failure mode nobody plans for is architectural rather than commercial. If you embed the dashboard on a tab that loads by default, you bill a session for every user who lands on that page and looks at something else. Lazy-load the iframe on explicit interaction. That one change is often the largest single lever on the bill, and it costs an afternoon.

The embed handshake, and the three things that break it

The flow is short. Your backend authenticates the user with your own identity system, calls the embed URL API with the right tags or user ARN, returns the URL to the browser, and the SDK mounts an iframe against it.

1. The generated URL carries a temporary bearer token valid for five minutes, and it is single use once redeemed. Generate it per page load from your backend. Never cache it, never put it in a build artifact, never log it.
2. Session lifetime is separate from URL validity, set with SessionLifetimeInMinutes, and ranges from fifteen minutes to ten hours with ten hours as the default. Ten hours is almost never what you want for a customer-facing product. Match it to your own session, or shorter.
3. Domains must be allowed explicitly. An administrator configures static domains in the admin menu, and AllowedDomains on the API call can override that with up to three domains or subdomains per request. Add an AllowedEmbeddingDomains condition to the IAM policy of the calling role, or any developer with that permission can list any domain on the internet.

On the browser side, the v2 SDK creates an embedding context (which appends its own zero-pixel iframe to body for message passing) and then mounts the experience:

import { createEmbeddingContext } from 'amazon-quicksight-embedding-sdk';

const context = await createEmbeddingContext();

await context.embedDashboard(
  {
    url: embedUrl,                        // fetched from your backend, just now
    container: '#analytics',
    height: '600px',                      // acts as loading height below
    resizeHeightOnSizeChangedEvent: true,
  },
  {
    toolbarOptions: { export: false, undoRedo: false, reset: false },
    attributionOptions: { overlayContent: true },
    onMessage: async (event) => {
      if (event.eventName === 'ERROR_OCCURRED') {
        console.error(event.message.errorCode);
      }
    },
  }
);

Two details in there earn their place. resizeHeightOnSizeChangedEvent turns the height value into a loading placeholder and lets the frame grow to fit content, which is what stops the dashboard rendering into a 600px letterbox with an inner scrollbar. And overlayContent tells the layout to overlay the attribution footer rather than reserve extra height at the bottom for it.

Troubleshooting embedded analytics on AWS

Almost every failure lands in one of these. Read the error code out of the ERROR_OCCURRED message before doing anything else.

• Forbidden means the URL’s authentication code expired. You held the URL longer than five minutes, or you served it from a cache, or a retry redeemed it twice. Fix the generation path, not the permissions.
• Unauthorized means the session obtained from that code expired. Different problem, different fix: your SessionLifetimeInMinutes is shorter than how long people keep the tab open. Handle it by re-fetching a fresh URL and re-mounting rather than letting the frame sit there dead.
• Frame never appears at all. Check the onChange handler for NO_CONTAINER or INVALID_CONTAINER, which usually means you mounted before your target element existed, and for INVALID_URL, which means the URL shape does not match the experience method you called.
• Frame appears, dashboard does not. Nine times out of ten this is the domain allowlist. The request domain has to match what was allowed, including scheme and any subdomain, and a staging hostname that nobody added is the usual culprit.
• Modals render off-screen. A known consequence of auto-resizing height: an export dialog can open above the visible viewport. Listen for MODAL_OPENED and scroll the parent page to the frame position.
• Toolbar features silently missing. Bookmarks, threshold alerts and scheduling require both the SDK toolbar flag and the matching entry under FeatureConfigurations in the embed URL request, and they only exist on the registered-user path. Setting the client flag alone does nothing.
• First render is slow, later ones are fine. Direct query against a cold warehouse. Compare the same query in Athena or Redshift directly to confirm before blaming the BI layer.

Common mistakes

• Choosing registered-user embedding for the identity story, then discovering session-tag RLS is unavailable on that path.
• Treating AuthorizedResourceArns as the security boundary instead of namespace membership.
• Deriving a session tag value from anything the client can influence.
• Generating the embed URL at build time, or caching it in a CDN, and then not understanding the Forbidden errors.
• Leaving session lifetime at the ten-hour default in a customer-facing app.
• Putting the dashboard on a default-loaded tab and paying for sessions nobody asked for.
• Building the first version on direct query against Athena because it is quicker to wire up, then meeting the scan bill.
• Forgetting that embedding and row-level security sit in the Enterprise tier, so a Standard-tier proof of concept proves nothing.

Best practices

• Decide the identity model before you build a single dataset. Everything downstream inherits it.
• Put the embed URL call behind one server-side endpoint that reads tenant scope from your own session and nowhere else. One function, one place to audit.
• Constrain the calling IAM role with an AllowedEmbeddingDomains condition and scope resources to specific namespaces rather than a wildcard.
• Write an automated test that requests tenant A’s embed URL and asserts tenant B’s rows are absent. Run it on every dataset change, because RLS breaks silently.
• Default to SPICE with a refresh schedule matched to a stated freshness SLA, and only reach for direct query where the SLA genuinely demands it.
• Track refresh failures as a first-class alert in CloudWatch or whatever you already run, whether that is Grafana, Datadog or something in-house. A stale dashboard that still renders is worse than one that errors, because nobody notices.
• Lazy-load the iframe on user intent, not on page mount.
• Keep the embedded surface read-only unless authoring is a real product requirement. Console embedding is a much larger permissions surface than dashboard embedding.

Is managed BI even the right call?

Worth asking honestly, because the answer is not always yes. The case for the AWS-native route is real: no connector layer to maintain against Athena, Redshift, S3 and Aurora, IAM you already understand, and a usage-based cost model that beats per-seat licensing when your viewers are bursty. If most of your data already sits in AWS, that adds up.

The case against is equally real. Visual customisation is limited compared to charting directly against your own API, the attribution footer needs a commitment to remove, and if you want full control of the front end you may be better served by Apache Superset or Metabase self-hosted, or by Grafana where the workload is closer to operational metrics than customer-facing BI. Those come with an operational burden you now own. That is the trade: you either run the BI layer or you rent it, and renting it means living inside its constraints.

Frequently asked questions

Do my users need AWS accounts to view an embedded dashboard?

No. With anonymous embedding they need no AWS account and no BI user record at all. Your application authenticates them however you already do, and your backend maps that identity to session tags when it requests the embed URL.

Can I use row-level security with registered-user embedding?

Yes, but only with username or group based rules, not with session tags. Tag-based RLS is restricted to the anonymous embedding path. If you need tags, you need anonymous sessions.

How long does an embed URL stay valid?

The URL itself carries a bearer token valid for five minutes and usable once. The session it opens is separate and lasts between fifteen minutes and ten hours depending on SessionLifetimeInMinutes, defaulting to ten hours.

Should I use SPICE or direct query for embedded analytics on AWS?

SPICE for anything with many viewers per refresh, which describes most embedded products. Direct query where the data must be current to the second, or where the dataset exceeds what you want to hold in memory. A hybrid split by data age is often the right answer and is under-used.

Why do I get a Forbidden error when the dashboard worked yesterday?

Forbidden points at the URL, not at permissions. The most common causes are caching the URL, generating it more than five minutes before use, or a client retry redeeming the same single-use token twice. If it is Unauthorized instead, the session expired and you need a fresh URL.

Can I white-label the embedded dashboard completely?

Largely. Themes control colours and typography, the SDK hides toolbar controls, and parameters let your own UI drive the dashboard. Removing the attribution footer entirely is tied to an annual capacity commitment, so confirm that against current terms before you promise it to a customer.

Does natural-language querying work in an embedded context?

Yes. The SDK exposes a generative Q&A experience alongside dashboards and visuals, driven by curated topics rather than raw tables. It is billed on its own capacity dimension and gates behind the Pro tiers, so treat it as a separate cost decision rather than a free addition.

The one thing worth remembering

Embedded analytics on AWS is not a rendering problem. The iframe is the last five percent. The part that decides whether the build survives contact with a second customer is the identity model, because it silently determines which isolation mechanism you are allowed to use, and that in turn shapes your dataset design, your refresh strategy and your bill.

Pick that first. Write the cross-tenant test before you write the dashboard. Everything else is recoverable in an afternoon.

Need help with an embedded analytics build?

This is the kind of work I do. Things I can help with directly:

• Reviewing an existing embed integration for cross-tenant leakage, including the session-tag path and the namespace boundary.
• Designing the identity and row-level security model before you commit to a dataset layout.
• Cutting Athena scan and Redshift concurrency cost by moving the right datasets into SPICE with incremental refresh.
• Building the backend embed-URL service with scoped IAM roles, domain conditions and sane session lifetimes.
• Setting up refresh failure alerting so a stale dashboard does not quietly serve last week’s numbers.
• Automated cross-tenant isolation tests wired into CI, so an RLS regression fails the build instead of the customer.

Send me the actual thing: your embed URL request payload with secrets stripped, the browser console error, or the dataset RLS rules. It is much faster to reason about a real payload than a description of one.

The post Embedded Analytics on AWS: The Four Decisions That Bite Later appeared first on John Nessime.