<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>PrivateLink | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/privatelink/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/privatelink/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Thu, 29 Oct 2026 06:05:05 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>PrivateLink | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/privatelink/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Salesforce AWS PrivateLink: The Failures Nobody Warns You About</title>
		<link>https://john-nessime.com/blog/technical-guides/salesforce-aws-privatelink/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 29 Oct 2026 06:05:05 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Networking]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Web Security]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[DNS]]></category>
		<category><![CDATA[Hyperforce]]></category>
		<category><![CDATA[Interface Endpoint]]></category>
		<category><![CDATA[Named Credentials]]></category>
		<category><![CDATA[Network Isolation]]></category>
		<category><![CDATA[Private Hosted Zone]]></category>
		<category><![CDATA[PrivateLink]]></category>
		<category><![CDATA[Rate Limiting]]></category>
		<category><![CDATA[Route 53]]></category>
		<category><![CDATA[Salesforce]]></category>
		<category><![CDATA[Salesforce Private Connect]]></category>
		<category><![CDATA[SNI]]></category>
		<category><![CDATA[TLS]]></category>
		<category><![CDATA[Transit Gateway]]></category>
		<category><![CDATA[VPC]]></category>
		<category><![CDATA[VPC Flow Logs]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=330</guid>

					<description><![CDATA[<p>Salesforce Private Connect over AWS PrivateLink does not fail loudly. It fails by working: green statuses, successful callouts, and packets still crossing the public internet because one DNS record was wrong. Here are the six failure families that actually bite, and how to prove your traffic is private.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/salesforce-aws-privatelink/">Salesforce AWS PrivateLink: The Failures Nobody Warns You About</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The ticket said the integration was &#8220;already private.&#8221; Someone had bought the license, clicked through the setup wizard, seen a green status in the org, and closed the epic. Six weeks later a security review pulled VPC flow logs and asked a simple question: why is this traffic still leaving through the NAT gateway?</p>



<p class="wp-block-paragraph">That is the failure mode worth understanding before you touch anything. A Salesforce AWS PrivateLink setup does not fail loudly. It fails by working. The callout succeeds, the payload arrives, the dashboard is green, and the packets went over the public internet anyway because one DNS record was wrong. Nothing in either console tells you.</p>



<p class="wp-block-paragraph">This post covers what actually breaks when you wire Salesforce to AWS over private connectivity: the DNS trap above, the throughput ceiling nobody budgets for, the TLS and port constraints, why your sandbox cannot rehearse the change, and how to prove after the fact that traffic really is staying on the AWS network.</p>



<h2 class="wp-block-heading">What Private Connect actually gives you</h2>



<p class="wp-block-paragraph">Salesforce Private Connect is the product name. AWS PrivateLink is the underlying mechanism. Salesforce runs a managed transit VPC in each supported AWS region, and you attach to it: either your VPC creates an interface endpoint pointing at their endpoint service, or their transit VPC creates an endpoint pointing at yours.</p>



<p class="wp-block-paragraph">Two things people consistently get wrong here.</p>



<p class="wp-block-paragraph">First, the direction naming is from the org&#8217;s point of view, not yours. <strong>Inbound</strong> means traffic entering Salesforce, so your AWS workload calling the Salesforce REST or Bulk API. <strong>Outbound</strong> means Salesforce calling out to something you host. If you are an AWS engineer, &#8220;inbound&#8221; is the one that feels like outbound to you. Half the confused meetings I have seen on this trace back to that single word.</p>



<p class="wp-block-paragraph">Second, Private Connect only builds the pipe. It does not move data. You still need Apex callouts, External Services, Platform Events, MuleSoft, Amazon AppFlow, or whatever else was already doing the work. If your integration is broken today, a private path will carry the broken requests privately.</p>



<blockquote class="is-layout-flow wp-block-quote-is-layout-flow"><p>Private connectivity is a network property, not an application feature. Nothing above layer 4 changes. Plan the rollout as a routing change, not an integration project.</p></blockquote>



<h2 class="wp-block-heading">Failure family one: DNS that quietly resolves to the public internet</h2>



<p class="wp-block-paragraph">This is the one that produced the ticket at the top, and it is by far the most common way a Salesforce AWS PrivateLink deployment ends up decorative.</p>



<p class="wp-block-paragraph">For the inbound case, your application calls a hostname like <code>yourorg.my.salesforce.com</code>. That hostname resolves publicly. Creating the interface endpoint does not change that. Unless something inside your VPC overrides the answer, the resolver returns Salesforce&#8217;s public addresses and your traffic takes the normal internet path. The endpoint sits there costing you money and carrying nothing.</p>



<p class="wp-block-paragraph">The override is a Route 53 private hosted zone, associated with the VPC that holds the endpoint, containing a record that points the org hostname at the endpoint. Salesforce&#8217;s own walkthrough names the zone <code>my.salesforce.com</code> and puts the org-specific record inside it.</p>



<p class="wp-block-paragraph">Three things have to line up or the override silently does nothing:</p>



<ul class="wp-block-list"><li><strong>The zone is associated with the right VPC.</strong> A private hosted zone answers only for VPCs it is attached to. Query it from anywhere else and the request falls through to public recursion.</li><li><strong>The VPC has DNS support and DNS hostnames enabled.</strong> Both attributes must be on, or the VPC resolver will not serve the zone at all.</li><li><strong>Callers actually use the VPC resolver.</strong> Workloads pinned to an external resolver, containers with a hardcoded <code>nameserver</code> line, or on-premises clients arriving over Direct Connect will bypass the zone entirely unless you front it with a Route 53 Resolver inbound endpoint.</li></ul>



<p class="wp-block-paragraph">There is a sharp edge in the other direction too. If a private hosted zone matches the queried name but contains no record of the right type, Route 53 returns NXDOMAIN rather than falling through to public DNS. So a half-finished zone does not degrade gracefully, it takes the hostname down. Creating the zone and the record in separate change windows is how teams turn a routing improvement into an outage.</p>



<p class="wp-block-paragraph">Verify from inside the VPC, not from your laptop:</p>



<pre class="wp-block-code"><code># Run this on an instance in the VPC associated with the private zone.
dig +short yourorg.my.salesforce.com

# You want RFC 1918 addresses from your endpoint subnets.
# Public addresses mean the private zone is not being consulted.

# Confirm which resolver answered:
dig yourorg.my.salesforce.com | grep -i "SERVER:"</code></pre>



<h2 class="wp-block-heading">Failure family two: the throughput ceiling nobody reads</h2>



<p class="wp-block-paragraph">Private Connect is rate limited, and the default entitlement is small enough to catch people out badly.</p>



<p class="wp-block-paragraph">Salesforce documents an initial license entitlement of 225 MB of data per hour, measured separately for inbound and outbound, with an add-on ceiling far higher. The allowance does not roll over. It resets on the hour.</p>



<p class="wp-block-paragraph">Now consider what a nightly Bulk API extract looks like. A few gigabytes pushed in a twenty minute window is entirely normal and completely incompatible with a 225 MB hourly bucket. The two directions behave differently when you hit the wall, which matters more than the number itself:</p>



<ul class="wp-block-list"><li><strong>Outbound is enforced technically.</strong> Exceed it and outbound traffic stops until the counter resets at the top of the next hour. Your Apex callouts start failing. This is a production incident.</li><li><strong>Inbound is enforced contractually.</strong> Salesforce documents that you are responsible for monitoring and enforcing the inbound limit yourself. Nothing throttles you at the network layer, but you are outside your agreement, and standard enterprise API limits still apply on top.</li></ul>



<p class="wp-block-paragraph">The outbound allowance is visible through the Limits API as <code>PrivateConnectOutboundCalloutHourlyLimitMB</code>, which is what makes it monitorable. Poll it, put it on a dashboard next to your other API limit metrics in Grafana or CloudWatch, and alert well before exhaustion rather than after the callouts fail.</p>



<p class="wp-block-paragraph">If your volume genuinely exceeds the entitlement, the honest options are buying more, reshaping the workload to spread across hours, or accepting that bulk movement stays on a different path and Private Connect covers only your latency-sensitive transactional traffic. That last one is a legitimate architecture, not a failure.</p>



<h2 class="wp-block-heading">Failure family three: TLS, SNI, and the port you cannot use</h2>



<p class="wp-block-paragraph">Private Connect inbound supports port 443 only. Port 8443 is explicitly not supported because Salesforce uses it for mutual TLS. If your integration depends on client certificate authentication on that port, this design does not cover it, and finding out during cutover is expensive.</p>



<p class="wp-block-paragraph">The subtler problem is SNI. Salesforce documents that Hyperforce requires callers to include the Server Name Indication extension in the TLS ClientHello so the right certificate can be served, and that clients calling a hostname outside the <code>*.my.salesforce.com</code> or <code>*.sandbox.my.salesforce.com</code> patterns will hit HTTPS errors. Browsers always send SNI. Older HTTP libraries, embedded clients, and some middleware stacks do not.</p>



<p class="wp-block-paragraph">The reason this bites specifically during a PrivateLink migration is that you are changing what the name resolves to while keeping the name identical. A client that was quietly relying on the public path&#8217;s behavior now hits a different frontend. Test the handshake explicitly:</p>



<pre class="wp-block-code"><code># From inside the VPC, forcing SNI to the org hostname.
openssl s_client -connect yourorg.my.salesforce.com:443 
  -servername yourorg.my.salesforce.com &lt;/dev/null

# Compare with SNI omitted. If this one fails and the above
# succeeds, your client library is the problem, not the network.
openssl s_client -connect yourorg.my.salesforce.com:443 &lt;/dev/null</code></pre>



<p class="wp-block-paragraph">On the outbound side the mirror image applies. Salesforce is the client calling your endpoint service, and it will validate your certificate against the hostname it was told to use. If your Network Load Balancer terminates TLS with a certificate whose subject alternative names do not include the private DNS name you registered, the callout fails on hostname verification. Issue the certificate for a name you control and that matches what the named credential targets.</p>



<h2 class="wp-block-heading">Failure family four: you cannot rehearse this properly</h2>



<p class="wp-block-paragraph">This one deserves its own section because it changes how you plan the work.</p>



<p class="wp-block-paragraph">Private connections are not copied from production into sandboxes. You recreate them. And the ability to recreate them depends on sandbox type:</p>



<ul class="wp-block-list"><li><strong>Full and Partial Copy sandboxes:</strong> you can create connections and provision them. This is where genuine rehearsal happens.</li><li><strong>Developer and Developer Pro sandboxes:</strong> you can create connections but cannot provision them.</li><li><strong>Scratch orgs and Developer orgs:</strong> create only, no provisioning.</li></ul>



<p class="wp-block-paragraph">If your organization only has Developer sandboxes, there is no lower environment where the private path can be exercised end to end. That is a procurement conversation, not an engineering one, and it needs to happen at design time. Budget for a Full or Partial Copy sandbox or accept that production is your first real test.</p>



<p class="wp-block-paragraph">Related trap: connections are packageable while unprovisioned, and unprovisioned connections do not consume a license. That is useful for shipping configuration through a deployment pipeline, but it also means a successful deploy proves nothing about connectivity.</p>



<h2 class="wp-block-heading">Failure family five: license arithmetic and connection identity</h2>



<p class="wp-block-paragraph">Each Private Connect license covers one provisioned connection in each direction. Each connection is a one-to-one mapping between an org ID and a VPC endpoint ID.</p>



<p class="wp-block-paragraph">Read that mapping carefully, because it is where multi-account AWS estates get expensive. Separate VPCs mean separate endpoints mean separate connections mean separate licenses. Separate orgs mean the same again. A design with three application VPCs across two orgs is not one connection, it is six.</p>



<p class="wp-block-paragraph">The architectural answer is usually a shared services or egress VPC that owns the endpoint, with the application VPCs reaching it through a Transit Gateway. You pay for one endpoint and route to it. This is the same centralized-endpoint pattern you would already use for interface endpoints to AWS services, and the reasoning is identical: endpoints are billed per attachment, so consolidate them.</p>



<p class="wp-block-paragraph">Four inbound connections consume four licenses. If you hold four licenses, you have used all of them and have none left for outbound. Count both directions before the renewal conversation.</p>



<h2 class="wp-block-heading">Failure family six: the AWS side of an outbound connection</h2>



<p class="wp-block-paragraph">For outbound, you are the service provider. Salesforce&#8217;s transit VPC creates the endpoint into your account, which means you have to build a proper endpoint service.</p>



<p class="wp-block-paragraph">The sequence, in the order that avoids rework:</p>



<ol class="wp-block-list"><li>Stand up a Network Load Balancer in front of your service, across at least two Availability Zones.</li><li>Create the VPC endpoint service backed by that NLB.</li><li>Copy the IAM role ARN from the Private Connect setup page in Salesforce for your region, and add it to the allowed principals on the endpoint service. Without this, Salesforce cannot connect and the failure looks like a generic timeout.</li><li>Register a private DNS name on the endpoint service and complete domain ownership verification.</li><li>Create the outbound connection in Salesforce using the endpoint service name, then sync it to pick up the AWS-side status.</li><li>Associate the connection with a named credential. Callouts only take the private path when they go through a named credential bound to that connection.</li></ol>



<p class="wp-block-paragraph">Step four is where people lose an afternoon. Domain verification requires a TXT record in a <em>public</em> zone, because AWS queries the public name servers registered at your domain&#8217;s registrar. Put the record in a private hosted zone and verification sits in pending forever with no useful error.</p>



<pre class="wp-block-code"><code># Register the private DNS name on your endpoint service.
aws ec2 modify-vpc-endpoint-service-configuration 
  --service-id vpce-svc-EXAMPLE 
  --private-dns-name "sf-integration.example.com"

# Read back the TXT record name and value AWS expects.
aws ec2 describe-vpc-endpoint-service-configurations 
  --service-ids vpce-svc-EXAMPLE 
  --query 'ServiceConfigurations[0].PrivateDnsNameConfiguration'

# After publishing the TXT record in your PUBLIC zone:
aws ec2 start-vpc-endpoint-service-private-dns-verification 
  --service-id vpce-svc-EXAMPLE</code></pre>



<p class="wp-block-paragraph">One detail worth knowing: if verification status later changes away from verified, new connection requests are refused while existing connections keep working. So a lapsed TXT record does not break you immediately. It breaks you the next time something reconnects, which is usually during an unrelated incident.</p>



<h2 class="wp-block-heading">What this costs, mechanically</h2>



<p class="wp-block-paragraph">Rates change, so model the mechanism rather than memorizing figures from a blog post.</p>



<ul class="wp-block-list"><li><strong>Interface endpoint hours.</strong> Billed per endpoint, per Availability Zone, per hour, whether or not traffic flows. A three-AZ deployment is three billable attachments.</li><li><strong>Data processing.</strong> Per GB through the endpoint, on a tiered scale that drops as regional volume rises.</li><li><strong>Cross-AZ transfer through interface endpoints.</strong> AWS removed this charge, so multi-AZ endpoint placement is a resilience decision rather than a cost trade-off.</li><li><strong>Cross-region.</strong> Still charged, in both directions, plus a per-hour fee on the endpoint service for each remote region with attached endpoints. Match your org region to your VPC region if you can.</li><li><strong>Provider-side infrastructure.</strong> For outbound, the NLB and its targets are your bill, not PrivateLink&#8217;s.</li><li><strong>The Salesforce license.</strong> Separate from all of the above, priced per connection.</li></ul>



<p class="wp-block-paragraph">Compare honestly against what you are replacing. If the alternative is a NAT gateway carrying the same traffic, NAT has both an hourly charge and a per-GB processing charge of its own, and the comparison is often closer than people assume. If the alternative is a site-to-site VPN or a commercial network overlay, the calculus changes again. PrivateLink wins on the security posture, not automatically on the invoice.</p>



<h2 class="wp-block-heading">Troubleshooting: proving the traffic is actually private</h2>



<p class="wp-block-paragraph">Do not trust a green status field. Prove it with evidence from the data path.</p>



<ol class="wp-block-list"><li><strong>Resolve the hostname from inside the VPC.</strong> Private addresses from your endpoint subnets, or nothing else matters.</li><li><strong>Check the endpoint state.</strong> <code>aws ec2 describe-vpc-endpoints</code> should show the endpoint available, with an ENI in each subnet you expect.</li><li><strong>Read VPC flow logs.</strong> Filter for your application&#8217;s source addresses and confirm the destination is the endpoint ENI, not a NAT gateway or internet gateway address. This is the actual proof.</li><li><strong>Check the security group on the endpoint.</strong> Interface endpoints carry security groups. If 443 is not permitted from your application subnets, connections hang rather than refuse, which reads as a Salesforce problem in your logs.</li><li><strong>For outbound, sync the connection in Salesforce</strong> before believing its status. The displayed state is the last polled AWS state, not live.</li><li><strong>Test the callout through the named credential.</strong> A direct callout to a hardcoded URL will not use the private path even when everything else is correct.</li></ol>



<p class="wp-block-paragraph">Two symptom patterns worth recognizing. Connections that hang and time out point at security groups or NLB target health. Connections that fail fast with a TLS error point at certificate naming or SNI. They are diagnosed in completely different places.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list"><li>Treating the setup wizard&#8217;s success message as proof that traffic is private. It proves the control plane is configured.</li><li>Creating the private hosted zone in one change and the record in another, producing NXDOMAIN for the org hostname in between.</li><li>Publishing the domain verification TXT record in a private zone instead of the public one.</li><li>Forgetting the Salesforce IAM role in the allowed principals list on the endpoint service, then debugging it as a network problem.</li><li>Sizing the connection for peak batch throughput without checking the hourly data entitlement.</li><li>Assuming sessions locked to their originating IP address will keep working. That behavior is documented as unsupported with Private Connect, along with asynchronous callouts using Apex continuations.</li><li>Building one endpoint per application VPC and discovering the license count at renewal.</li><li>Leaving callouts on hardcoded endpoints instead of named credentials, so outbound traffic never enters the private path.</li></ul>



<h2 class="wp-block-heading">Best practices for a Salesforce AWS PrivateLink rollout</h2>



<ul class="wp-block-list"><li>Inventory every integration touching the org first, with direction, protocol, hostname, and peak volume per hour. The inventory determines the design.</li><li>Centralize endpoints in a shared services VPC and route from application VPCs through Transit Gateway rather than duplicating endpoints and licenses.</li><li>Put the endpoint in the same region as the org. Cross-region private connectivity works but gives up most of the latency benefit and adds transfer charges.</li><li>Enable VPC flow logs before cutover so you have a clean before-and-after picture. Ship them somewhere you can query, whether that is CloudWatch Logs Insights, Athena over S3, or a platform like Grafana Cloud or Datadog.</li><li>Alert on the outbound hourly limit as a percentage of entitlement, not on callout failures. By the time callouts fail you are already down.</li><li>Terraform the AWS side. The endpoint, the endpoint service, the allowed principals, the private hosted zone, and the records all belong in code. The Salesforce side is click-configured, so document it in the same repository as a runbook.</li><li>Keep a documented rollback: the change that reverts you to public routing is removing or disassociating the private hosted zone, and it should be a one-step action you have practiced.</li><li>Review the configuration on a schedule. Private routing is invisible in normal operation and easy to break with an unrelated DNS or VPC change.</li></ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Does Private Connect encrypt my traffic?</h3>



<p class="wp-block-paragraph">Your traffic is already encrypted in transit by TLS. What private connectivity changes is the path, not the cipher. The value is that packets stay on the AWS network rather than traversing the public internet, which removes an entire class of exposure and simplifies a lot of compliance conversations. It is not a substitute for TLS, and it does not authenticate anything.</p>



<h3 class="wp-block-heading">Can I use Private Connect to access the Salesforce user interface privately?</h3>



<p class="wp-block-paragraph">No. This is API-path connectivity for integrations. Browser access to the org is a separate concern, and if that is what you need, you are looking at conditional access policies, login IP ranges, or a corporate network solution rather than PrivateLink.</p>



<h3 class="wp-block-heading">What Salesforce features actually route through an outbound connection?</h3>



<p class="wp-block-paragraph">Salesforce documents outbound support for Apex callouts, External Services, Platform Events, Change Data Capture, CRM Analytics, the OData 4.01 adapter and custom adapter for Salesforce Connect, the Salesforce Connect SQL adapter for Amazon Athena, and Agent Actions backed by External Services. Inbound covers the supported public APIs. Anything not on the current list should be verified against the documentation before you design around it, because that list grows.</p>



<h3 class="wp-block-heading">Do I need Hyperforce for this?</h3>



<p class="wp-block-paragraph">Salesforce states that Private Connect works whether the org sits in first-party data centers or on Hyperforce. That said, Hyperforce brings its own requirements, particularly the SNI behavior described above, so confirm where your org actually runs before you start debugging TLS.</p>



<h3 class="wp-block-heading">How do I connect on-premises systems, not just AWS workloads?</h3>



<p class="wp-block-paragraph">Bring the on-premises network into AWS first with Direct Connect or a site-to-site VPN, then use the interface endpoint from there. The extra work is DNS: on-premises resolvers need a Route 53 Resolver inbound endpoint to see the private hosted zone, otherwise they resolve the org hostname publicly and you are back to the first failure family.</p>



<h3 class="wp-block-heading">Can I use one connection for several Salesforce orgs?</h3>



<p class="wp-block-paragraph">No. A connection is a one-to-one mapping between an org ID and a VPC endpoint ID, so each org needs its own provisioned connection and its own license, even if they all point at the same VPC.</p>



<h3 class="wp-block-heading">Is a VPN a reasonable alternative?</h3>



<p class="wp-block-paragraph">For Salesforce specifically, no. Salesforce does not terminate a customer VPN into the org, so PrivateLink is the mechanism on offer. A VPN or a commercial overlay such as NordLayer solves a different problem, which is getting your own users and sites onto your own network. Where those tools genuinely overlap is the on-premises leg, and even there Direct Connect is usually the better answer for sustained integration traffic.</p>



<h2 class="wp-block-heading">The one thing to remember</h2>



<p class="wp-block-paragraph">A Salesforce AWS PrivateLink deployment that is misconfigured looks exactly like one that works. Same green statuses, same successful callouts, same clean application logs. The only difference is which path the packets took, and the only place that difference is visible is your flow logs.</p>



<p class="wp-block-paragraph">So build the verification step into the project rather than the postmortem. Resolve the hostname from inside the VPC, read the flow logs, confirm the destination is the endpoint ENI, and keep the check running afterwards. Everything else in this post is detail. That one habit is what separates private connectivity from a line item that says private.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help wiring Salesforce to AWS privately?</h2>



<p class="wp-block-paragraph">This is the kind of work where a second pair of eyes on the DNS and the flow logs saves a security review later. I help teams with:</p>



<ul class="wp-block-list"><li>Designing the endpoint topology, including centralized endpoints behind Transit Gateway so you are not buying a connection per VPC.</li><li>Building and verifying the Route 53 private hosted zone override, including Resolver inbound endpoints for on-premises callers.</li><li>Standing up the endpoint service side for outbound connections: NLB, allowed principals, private DNS name registration, and domain verification.</li><li>Proving traffic is genuinely private with VPC flow log analysis, before and after cutover.</li><li>Instrumenting the hourly data entitlement so you find out about throttling from a dashboard rather than from failed callouts.</li><li>Codifying the AWS side in Terraform or OpenTofu, with a documented rollback to public routing.</li></ul>



<p class="wp-block-paragraph">If you are mid-way through one of these and something is not behaving, send me the <code>dig</code> output from inside the VPC, a flow log sample, or your endpoint service configuration and I will tell you what I see.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/salesforce-aws-privatelink/">Salesforce AWS PrivateLink: The Failures Nobody Warns You About</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Legal Document Intelligence on AWS: The Five Boundaries That Have to Hold</title>
		<link>https://john-nessime.com/blog/cloud-computing/legal-document-intelligence-aws/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Tue, 25 Aug 2026 18:00:00 +0000</pubDate>
				<category><![CDATA[Case Studies]]></category>
		<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Web Security]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[Amazon Comprehend]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[Audit Logging]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS KMS]]></category>
		<category><![CDATA[AWS Organizations]]></category>
		<category><![CDATA[Bedrock Guardrails]]></category>
		<category><![CDATA[Bedrock Knowledge Bases]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[Compliance]]></category>
		<category><![CDATA[Data Residency]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Encryption]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[Litigation Hold]]></category>
		<category><![CDATA[Metadata Filtering]]></category>
		<category><![CDATA[Multi-Tenant]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[OpenSearch Serverless]]></category>
		<category><![CDATA[PrivateLink]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[S3 Object Lock]]></category>
		<category><![CDATA[Step Functions]]></category>
		<category><![CDATA[Tenant Isolation]]></category>
		<category><![CDATA[Vector Database]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=272</guid>

					<description><![CDATA[<p>A confident answer with a citation from a matter the reader was walled off from. Nothing crashed, nothing alerted. Building legal document intelligence on AWS with Textract, Bedrock and OpenSearch is the easy half; the hard half is tenant isolation, verified deletion, S3 Object Lock holds, what leaves the account, and an audit trail that names the actual user.</p>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/legal-document-intelligence-aws/">Legal Document Intelligence on AWS: The Five Boundaries That Have to Hold</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The demo was going well until someone asked where the third citation came from.</p>



<p class="wp-block-paragraph">The answer on screen was good. Fluent, specific, correctly hedged, three sources listed underneath it. Then a partner in the room asked a simple question: which matter is source three from? Nobody could answer it in the room, and when we went and looked, it was from a matter that half the people watching the demo were formally walled off from.</p>



<p class="wp-block-paragraph">Nothing had crashed. No alarm fired. The retrieval layer had done exactly what a similarity search does, which is return the nearest vectors, and the nearest vectors did not care about the ethical wall. That is the thing about building a legal document intelligence platform on AWS: the failures that matter almost never look like failures. They look like a confident answer with a citation attached.</p>



<p class="wp-block-paragraph">This post is about the parts of that build that are hard, organised around the five boundaries that actually have to hold. The extraction pipeline is the easy half. I will cover it, but quickly, because the AWS documentation is good and the failure modes are visible. The other four boundaries fail quietly, and that is where the engineering goes.</p>



<h2 class="wp-block-heading">The shape of the thing</h2>



<p class="wp-block-paragraph">Before the boundaries, the skeleton. A legal document intelligence platform on AWS almost always ends up looking like this, whether you plan it or arrive at it:</p>



<ul class="wp-block-list">
<li>Documents land in Amazon S3, one prefix per matter, versioning on.</li>



<li>An S3 event triggers AWS Lambda, which starts an asynchronous Amazon Textract job.</li>



<li>Textract output lands back in S3 as JSON. Something normalises it into text plus page and bounding-box coordinates.</li>



<li>Amazon Comprehend or a foundation model classifies the document and pulls entities: parties, dates, governing law, clause types.</li>



<li>Chunks get embedded and written to a vector store, usually Amazon OpenSearch Serverless behind Amazon Bedrock Knowledge Bases.</li>



<li>An application layer, typically API Gateway plus Lambda, takes a question, retrieves, and calls a model on Amazon Bedrock.</li>
</ul>



<p class="wp-block-paragraph">AWS Step Functions is worth reaching for once you have more than three stages, because Textract jobs are long-running and Lambda timeouts are not a retry strategy. That is the whole architecture. You can stand it up in a fortnight. Then you spend six months on everything below.</p>



<h2 class="wp-block-heading">Boundary one: isolation, and why metadata filtering is not optional</h2>



<p class="wp-block-paragraph">This is the one from the demo, and it is the one that ends engagements.</p>



<p class="wp-block-paragraph">A vector index has no native concept of a matter, a client, or an ethical wall. If you pool every document into one index and rely on the prompt to keep things separate, you have built a system where a well-phrased question can pull privileged material across a wall. Prompt instructions are not an access control. They are a suggestion that usually works.</p>



<p class="wp-block-paragraph">There are two shapes that do work, and the choice between them is a real trade-off rather than a best practice.</p>



<h3 class="wp-block-heading">Pooled index with server-side filtering</h3>



<p class="wp-block-paragraph">Every chunk carries a <code>matter_id</code> and a <code>client_id</code> as metadata. Every retrieval call attaches a filter. With an S3 data source in Bedrock Knowledge Bases, the metadata comes from a sidecar JSON file that sits next to the document and carries a <code>metadataAttributes</code> object.</p>



<pre class="wp-block-code"><code>{
  "metadataAttributes": {
    "matter_id": "M-4417",
    "client_id": "C-108",
    "doc_type": "engagement_letter",
    "privileged": true
  }
}</code></pre>



<p class="wp-block-paragraph">The retrieval call then narrows the search before the model ever sees a chunk:</p>



<pre class="wp-block-code"><code>"retrievalConfiguration": {
  "vectorSearchConfiguration": {
    "filter": {
      "equals": { "key": "matter_id", "value": "M-4417" }
    }
  }
}</code></pre>



<p class="wp-block-paragraph">Two rules make this safe, and both are the kind of thing that gets skipped under deadline. First, the filter value is derived on the server from the authenticated caller&#8217;s claims, never accepted from the client. If the browser can send you a <code>matter_id</code>, the browser can send you a different one. Second, the filter is applied by code that no feature request can bypass. A helper that builds every retrieval request, and a code review rule that no other path may call the retrieve API directly.</p>



<p class="wp-block-paragraph">The trap here is ordering. You cannot filter on an attribute the index does not have. Add <code>matter_id</code> after ingestion and every chunk already in the index is invisible to that filter, which means it either returns for everyone or for no one, depending on how your filter is written. Design the metadata schema before the first ingestion run, not after the first demo.</p>



<h3 class="wp-block-heading">Separate collection per client</h3>



<p class="wp-block-paragraph">The heavier option. A separate OpenSearch Serverless collection per client, which buys you a separate AWS KMS key per client, separate index settings, and a failure mode where a bug in the filtering logic cannot reach across clients at all.</p>



<p class="wp-block-paragraph">It costs you sprawl. Every collection is a resource to provision, monitor, patch policy on, and eventually delete. Ingestion jobs run per data source per knowledge base, so freshness guarantees fragment. For a firm with twelve institutional clients this is fine. For a platform onboarding a hundred small clients it becomes the main operational burden of the product.</p>



<p class="wp-block-paragraph">My default is pooled with strict server-side filtering, and per-client collections only where the client&#8217;s own contract demands a dedicated encryption key. If you cannot say out loud which of those two you are running, you are running neither properly.</p>



<h2 class="wp-block-heading">Boundary two: extraction, and the confidence score everyone ignores</h2>



<p class="wp-block-paragraph">Textract returns a confidence score between 0 and 100 with each prediction. AWS is explicit in its own best-practice guidance that applications sensitive to detection errors should enforce a minimum threshold and route anything below it for human scrutiny, and that the right threshold depends entirely on how the output gets used.</p>



<p class="wp-block-paragraph">Legal documents sit at the harsh end of that. A misread date on an archival scan is a curiosity. A misread date on a limitation period is a problem with a name on it.</p>



<p class="wp-block-paragraph">Practical notes from this layer:</p>



<ul class="wp-block-list">
<li>Use the asynchronous operations for anything multipage. <code>StartDocumentAnalysis</code> and <code>GetDocumentAnalysis</code> handle long PDFs and TIFFs; the synchronous <code>AnalyzeDocument</code> path is for single pages and will fight you on a 400-page bundle.</li>



<li>Textract&#8217;s Queries feature earns its place on structured instruments. Instead of extracting everything and grepping, you ask the document a direct question and get a scoped answer with its own confidence.</li>



<li>Keep the page number and bounding box for every chunk. When a lawyer asks where a sentence came from, &#8220;page 14, second column&#8221; is an answer. &#8220;It&#8217;s in the corpus somewhere&#8221; is not, and the platform loses credibility the first time you say it.</li>



<li>Amazon Augmented AI wires low-confidence predictions into a human review queue with a private workforce, so the reviewers are your people rather than an anonymous pool. For privileged material that distinction is the entire point.</li>
</ul>



<p class="wp-block-paragraph">What I would skip early: building a custom entity recognition model. Comprehend supports custom entity recognition and it is genuinely useful, but you need labelled data you do not have yet, and a foundation model with a decent prompt gets you far enough to find out which entities the users actually care about. Train the custom model once the requirement has stopped moving.</p>



<h2 class="wp-block-heading">Boundary three: retention, where deletion is a distributed problem</h2>



<p class="wp-block-paragraph">Here is the failure that will not show up in any test suite you write.</p>



<p class="wp-block-paragraph">A client asks for a document to be removed. Someone deletes the S3 object. The ticket closes. The document is still fully searchable, because its chunks are sitting in the vector index and will stay there until the data source is re-synced. Meanwhile the Textract JSON output is in a second bucket, the extracted text may be in a database, and if model invocation logging is on, whole passages of the document are sitting in CloudWatch Logs or another S3 bucket entirely.</p>



<p class="wp-block-paragraph">One delete, at least four places holding a copy. Write the deletion path as a real workflow with an assertion at the end, not as a single API call:</p>



<ol class="wp-block-list">
<li>Delete the source object and, if versioning is on, its versions.</li>



<li>Delete the derived artefacts: Textract JSON, normalised text, any thumbnails or page images.</li>



<li>Trigger an ingestion job so the knowledge base drops the orphaned chunks.</li>



<li>Confirm removal by querying the index for the document identifier and expecting nothing back.</li>



<li>Deal with the logs, which means either a retention policy short enough to make the problem expire or a deliberate decision, written down, that logs are out of scope.</li>
</ol>



<p class="wp-block-paragraph">Step four is the one people leave out, and it is the only step that actually proves anything.</p>



<h3 class="wp-block-heading">The opposite problem: things that must not be deleted</h3>



<p class="wp-block-paragraph">Legal work has the reverse requirement too. When a matter goes into litigation hold, the documents need to survive an administrator with delete permissions and a bad afternoon.</p>



<p class="wp-block-paragraph">S3 Object Lock is the mechanism, and it has two independent controls that people routinely conflate. A retention period protects an object version until a fixed date, in either governance mode, which privileged users can override, or compliance mode, which nobody can override and where the period cannot be shortened. A legal hold has no date at all. It stays until someone with <code>s3:PutObjectLegalHold</code> explicitly removes it. The two can be active at once, and while either is active the object version cannot be deleted or overwritten.</p>



<pre class="wp-block-code"><code># Place an indefinite hold on a specific object version
aws s3api put-object-legal-hold 
  --bucket matters-archive 
  --key M-4417/exhibit-c.pdf 
  --version-id 3sL7f2Qz9pXvB1kR 
  --legal-hold Status=ON</code></pre>



<p class="wp-block-paragraph">Two things bite here. Object Lock requires versioning and turns it on automatically, and once enabled on a bucket you cannot turn it off or suspend versioning again. And compliance mode is genuinely permanent: if you set a seven-year retention when you meant seven days, that is the answer, for everyone, including the account root. Test in governance mode first. The bypass path exists and is deliberately awkward, requiring both the <code>s3:BypassGovernanceRetention</code> permission and an explicit <code>x-amz-bypass-governance-retention:true</code> header on the request, which is exactly the level of friction you want on that operation.</p>



<p class="wp-block-paragraph">Design the hold model before you design the deletion model, because holds win. A deletion request that collides with an active hold is a legal question, not an engineering one, and the platform&#8217;s job is to surface the collision clearly rather than resolve it silently.</p>



<h2 class="wp-block-heading">Boundary four: disclosure, or what leaves the account</h2>



<p class="wp-block-paragraph">This is the boundary the client&#8217;s general counsel will ask about, usually in writing, usually before signature. Three separate controls, and they are not interchangeable.</p>



<h3 class="wp-block-heading">The AI services opt-out policy</h3>



<p class="wp-block-paragraph">AWS Organizations has a governance policy type that opts your accounts out of having content processed by certain AI services stored and used for service improvement. The list includes Amazon Textract and Amazon Comprehend, which are precisely the two doing the reading in this architecture. The policy type has to be enabled at the organisation root before you can attach anything:</p>



<pre class="wp-block-code"><code>aws organizations enable-policy-type 
  --root-id r-example 
  --policy-type AISERVICES_OPT_OUT_POLICY</code></pre>



<p class="wp-block-paragraph">Read the AWS documentation on this one carefully rather than taking my summary as gospel, because there is a caveat in it that matters: the services may still need to store your data operationally even when you have opted out of it being used for improvement. Opting out is not the same as the data never existing outside your account. Say that plainly in the client conversation. It is a much better position than being asked about it later.</p>



<h3 class="wp-block-heading">Bedrock&#8217;s data position, and the log that undoes it</h3>



<p class="wp-block-paragraph">Amazon Bedrock&#8217;s published position is that inputs and outputs are not shared with third-party model providers and are not used to train the base models, and that fine-tuning operates on a private copy. That is a strong starting point for privileged content and it is the main reason Bedrock rather than a direct provider API shows up in these builds.</p>



<p class="wp-block-paragraph">Then there is model invocation logging. It captures full prompt and response payloads to CloudWatch Logs or S3, and you want it on, because without it you cannot reconstruct what the system told someone. But understand what you have just built: a log that contains privileged document text, at a per-region setting, in a destination that probably has looser access controls than the document store it came from. Encrypt it with the same KMS key discipline, restrict it harder than you think you need to, and set a retention period on purpose.</p>



<p class="wp-block-paragraph">Worth separating in your head: CloudTrail records that an API call happened and who made it. Model invocation logging records what was in it. You need both, for different questions, and only one of them contains client confidences.</p>



<h3 class="wp-block-heading">Network path</h3>



<p class="wp-block-paragraph">Interface VPC endpoints via AWS PrivateLink keep traffic to Bedrock, Textract and the rest on the AWS network rather than out through an internet gateway or NAT. Add a gateway endpoint for S3 while you are there.</p>



<p class="wp-block-paragraph">Endpoint policies are the part that gets forgotten. An endpoint without a policy is a private path to the whole service, including into accounts that are not yours. Scope it to the operations and resources you actually use, and pair it with an S3 bucket policy that rejects requests not arriving through your endpoint. The first protects what leaves; the second protects what can be reached.</p>



<h3 class="wp-block-heading">The parts that are not on the architecture diagram</h3>



<p class="wp-block-paragraph">Two of these have caught people out badly, and neither is an AWS control.</p>



<p class="wp-block-paragraph">The first is the development environment. You do not want real client documents on a laptop or in a scratch account, so build the parsing and chunking code against synthetic documents on a small separate box. A cheap VPS from a provider like Contabo or InterServer is fine for this, and the separation is worth more than the convenience of iterating in the production account.</p>



<p class="wp-block-paragraph">The second is the reviewers. If your human review loop involves contractors working remotely, their network path and their disks are part of your boundary whether or not you drew them. A managed VPN such as NordVPN or Surfshark handles the network side, and when a matter closes and local copies have to go, a dedicated erasure tool from something like O&amp;O Software does what dragging a folder to the bin does not. Unglamorous, and it is the layer that gets audited.</p>



<h2 class="wp-block-heading">Boundary five: evidence, because someone will ask</h2>



<p class="wp-block-paragraph">At some point the question stops being &#8220;does it work&#8221; and becomes &#8220;who saw what, and when&#8221;. If you cannot answer that from stored records, the platform is not defensible regardless of how good the answers are.</p>



<ul class="wp-block-list">
<li>Turn on CloudTrail data events for the document buckets. They are off by default, they are billed separately from management events, and they are the only way to see individual object-level reads.</li>



<li>Log the retrieval, not just the generation. Store which chunks came back and which filter was applied. When somebody asks whether the wall held on a specific query, this record is the answer and there is no reconstructing it later.</li>



<li>Carry the end user&#8217;s identity through to the audit record. A Lambda execution role in the logs tells you the platform did something. It does not tell you who asked.</li>



<li>Alarm on the filter, not just on errors. A retrieval that ran without a matter filter should page someone, even though it returned HTTP 200 and a perfectly good answer.</li>
</ul>



<p class="wp-block-paragraph">That last one is the closest thing to a single takeaway in this post. The dangerous events in this architecture are successful ones.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting the things that will go wrong</h2>



<h3 class="wp-block-heading">Retrieval returns nothing after you add a filter</h3>



<p class="wp-block-paragraph">Nearly always a metadata problem rather than a query problem. Either the sidecar metadata file was missing at ingestion, so the chunks carry no attribute to match, or the attribute was added after the chunks were indexed. Check whether the attribute exists on a known chunk before you touch the query. If it does not, you need a re-sync, not a different filter expression.</p>



<h3 class="wp-block-heading">Textract returns text but the layout is scrambled</h3>



<p class="wp-block-paragraph">Common on two-column pleadings and on documents with headers and footers on every page. Raw text detection reads in a reading order that is not always yours. Use the layout and table features rather than plain detection, and hold onto the geometry so you can reassemble columns yourself if the default order is wrong for that document class.</p>



<h3 class="wp-block-heading">The model cites a document that does not exist</h3>



<p class="wp-block-paragraph">Usually the citation is being generated rather than passed through. If document names are being written by the model instead of read from the retrieval response, it will invent plausible ones. Build citations from the retrieval result metadata in your application code and never from the generated text. Bedrock Guardrails helps with contextual grounding, but the structural fix is not asking the model to produce the reference in the first place.</p>



<h3 class="wp-block-heading">Ingestion is slow and nobody knows where</h3>



<p class="wp-block-paragraph">Instrument per stage before you optimise anything. In most of these pipelines the time is in Textract for large scanned bundles, and the fix is parallelism at the document level rather than tuning the extraction itself. Step Functions with a distributed map over documents gets you there without a queue you have to babysit.</p>



<h3 class="wp-block-heading">Costs climb faster than volume</h3>



<p class="wp-block-paragraph">Look at re-processing first. A pipeline that re-extracts a document every time anything downstream changes will quietly multiply your Textract bill against a static corpus. Key the extraction cache on the object&#8217;s content hash, not its path, and make re-ingestion an explicit action. The billing mechanisms differ per service, so read the current pricing pages rather than trusting a number from a blog post.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li><strong>Treating the system prompt as an access control.</strong> It is a formatting instruction that happens to look like a rule.</li>



<li><strong>Taking the tenant identifier from the request body.</strong> If the client can name the matter, the client can name someone else&#8217;s.</li>



<li><strong>Designing the metadata schema after the first ingestion run.</strong> Retrofitting a filterable attribute means re-indexing the whole corpus.</li>



<li><strong>Assuming a deleted S3 object is gone from the platform.</strong> The chunks, the extraction output and the invocation logs all outlive it.</li>



<li><strong>Setting compliance-mode retention without testing in governance mode.</strong> There is no support ticket that fixes a seven-year mistake.</li>



<li><strong>Enabling model invocation logging without treating the log as privileged.</strong> You have just made a second copy of the documents in a less protected place.</li>



<li><strong>Building VPC endpoints and leaving the default policy on them.</strong> A private path with no policy is still a path.</li>
</ul>



<h2 class="wp-block-heading">Best practices worth the effort</h2>



<ul class="wp-block-list">
<li>One retrieval helper, server-side filter injection, and a review rule that no other code path calls the retrieve API directly.</li>



<li>Metadata schema agreed and frozen before the first production ingestion. Include the fields you might filter on later, even if unused today.</li>



<li>Page-level provenance on every chunk, surfaced in the interface. It builds trust faster than any accuracy improvement.</li>



<li>A confidence threshold chosen per document class, with a human review path for everything under it.</li>



<li>Deletion implemented as a verified workflow across every store, ending in an assertion that the content is actually unretrievable.</li>



<li>Object Lock legal holds for matters under litigation hold, tested in governance mode before anything runs in compliance mode.</li>



<li>Customer-managed KMS keys across S3, the vector store and the logs, so key access is a control you can actually revoke.</li>



<li>An alarm on unfiltered retrievals, because the failure mode is a success response.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Is a legal document intelligence platform on AWS safe for privileged material?</h3>



<p class="wp-block-paragraph">The services support it. Bedrock&#8217;s stated position is that inputs and outputs are not shared with model providers or used to train base models, PrivateLink keeps traffic off the public internet, and KMS covers encryption at rest. What determines safety is your own configuration: tenant isolation, log handling and deletion behaviour. The platform is exactly as confidential as its weakest copy of the text.</p>



<h3 class="wp-block-heading">Should I use Bedrock Knowledge Bases or build retrieval myself?</h3>



<p class="wp-block-paragraph">Start with Knowledge Bases. It handles chunking, embedding, sync and metadata filtering, and those are weeks of work with no differentiation in them. Build your own when you need retrieval behaviour it does not expose, such as unusual re-ranking or per-tenant chunking strategies. Going custom on day one usually means reimplementing the managed service badly.</p>



<h3 class="wp-block-heading">How do I stop one client&#8217;s documents surfacing in another client&#8217;s answers?</h3>



<p class="wp-block-paragraph">Tag every chunk with a tenant identifier at ingestion, and inject the matching filter server-side on every retrieval from the authenticated caller&#8217;s identity. Never accept the identifier from the client. If the contract requires per-client encryption keys, use a separate collection per client instead, accepting the extra operational load.</p>



<h3 class="wp-block-heading">Does deleting a document from S3 remove it from the search index?</h3>



<p class="wp-block-paragraph">No. The chunks stay in the vector store until an ingestion job runs and reconciles the data source. Until then the document is deleted and still fully searchable, which is the worst combination available. Always finish a deletion by querying the index and confirming nothing comes back.</p>



<h3 class="wp-block-heading">What is the difference between an S3 legal hold and a retention period?</h3>



<p class="wp-block-paragraph">A retention period runs until a fixed date and comes in governance mode, which privileged users can override, or compliance mode, which nobody can. A legal hold has no end date and stays until someone explicitly removes it. They are independent, they can both be active on the same object version, and while either is active the object cannot be deleted or overwritten.</p>



<h3 class="wp-block-heading">Do I need human review, or is the model accurate enough?</h3>



<p class="wp-block-paragraph">Accuracy is the wrong frame. The question is what happens to a wrong answer downstream. Where output feeds a decision with consequences, you want a confidence threshold and a review queue, and Amazon Augmented AI with a private workforce gives you that without building the review tooling yourself. Where output is a search aid a person will verify anyway, review is friction you do not need.</p>



<h3 class="wp-block-heading">Can I run this in a single AWS account?</h3>



<p class="wp-block-paragraph">Technically yes, and for a pilot it is reasonable. It gets uncomfortable once you have production documents and a development environment in the same place, because an IAM mistake has nowhere to stop. Separate accounts under Organizations also gives you the policy layer, including the AI services opt-out policy, which only exists at the organisation level.</p>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">A legal document intelligence platform on AWS is not hard to build. Textract reads, Bedrock reasons, OpenSearch retrieves, and the tutorial version works on the first afternoon.</p>



<p class="wp-block-paragraph">What is hard is that its failures are silent and well-formed. The cross-matter citation, the deleted document that still answers questions, the privileged passage sitting in a log bucket, the retrieval that ran without a filter and returned a beautiful paragraph. None of these throw an error. All of them are the kind of thing that ends a client relationship.</p>



<p class="wp-block-paragraph">So build the boundaries first and the features second, and make sure every one of them is something you can prove from a stored record rather than something you believe about the code. If you can only take one habit from this: alarm on the successful requests that should not have been possible.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on your document pipeline?</h2>



<p class="wp-block-paragraph">I work on AWS document processing and retrieval systems where the confidentiality requirements are real. Typical things I get called in for:</p>



<ul class="wp-block-list">
<li>Reviewing tenant isolation in an existing RAG setup and finding the paths where the filter can be bypassed.</li>



<li>Designing the metadata and chunking schema before ingestion, so filtering works without a re-index later.</li>



<li>Building Textract pipelines with confidence thresholds, human review routing and page-level provenance.</li>



<li>Implementing verified deletion across S3, derived artefacts, the vector index and logs, with a proof step at the end.</li>



<li>Setting up retention and legal hold with S3 Object Lock, including the governance-mode rehearsal before compliance mode.</li>



<li>Locking down the network and logging path: VPC endpoints with real policies, KMS key separation, and audit records that name the actual user.</li>
</ul>



<p class="wp-block-paragraph">If you have an architecture diagram, a Step Functions definition or a retrieval request you are unsure about, send it over and I will tell you what I would change and why.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/legal-document-intelligence-aws/">Legal Document Intelligence on AWS: The Five Boundaries That Have to Hold</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Lightsail vs EC2 vs ECS: What I Actually Pick for Small Client Workloads</title>
		<link>https://john-nessime.com/blog/devops/lightsail-vs-ec2-vs-ecs-small-workloads/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Fri, 21 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Hosting & Infrastructure]]></category>
		<category><![CDATA[Amazon EC2]]></category>
		<category><![CDATA[Amazon ECS]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Fargate]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Containers]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[FinOps]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Lightsail]]></category>
		<category><![CDATA[Load Balancing]]></category>
		<category><![CDATA[Managed Hosting]]></category>
		<category><![CDATA[NAT Gateway]]></category>
		<category><![CDATA[PrivateLink]]></category>
		<category><![CDATA[Vendor Lock-In]]></category>
		<category><![CDATA[VPC]]></category>
		<category><![CDATA[VPS]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=262</guid>

					<description><![CDATA[<p>Choosing between Lightsail, EC2 and ECS for a small client workload is not a performance question. It is a question about which constraints you are accepting and how expensive they are to reverse. Profiles of all three with where each wins and loses, the cost levers that actually move the bill, a six-step decision procedure, and the arguments that fall apart on contact with a real project.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/lightsail-vs-ec2-vs-ecs-small-workloads/">Lightsail vs EC2 vs ECS: What I Actually Pick for Small Client Workloads</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The ticket said &#8220;site feels slow, can you take a look.&#8221; What it turned into was a migration. The client&#8217;s Lightsail instance had outgrown its bundle, the plan was to move to EC2, and Lightsail ships an export-to-EC2 feature that makes that sound like a button press.</p>



<p class="wp-block-paragraph">The instance snapshot exported fine. The managed database did not. Neither did the load balancer, the CDN distribution, the static IP, or the DNS zone. AWS documents this plainly: the export feature covers instance snapshots and block storage disk snapshots, and nothing else. Every other piece had to be rebuilt by hand, and a maintenance window turned into an afternoon.</p>



<p class="wp-block-paragraph">That is the part nobody prices in. When you choose compute for a small client workload, you are not just choosing where the process runs. You are choosing how much of the platform you will have to recreate when the client grows, leaves, or hands the account to somebody else.</p>



<p class="wp-block-paragraph">This post compares Lightsail vs EC2 vs ECS for small client work: single sites, small APIs, internal tools, the kind of thing that runs on one or two boxes with no dedicated ops team behind it. Each option gets a profile with where it wins and where it doesn&#8217;t, then a decision procedure, then the arguments I hear most often that fall apart on contact with a real project.</p>



<h2 class="wp-block-heading">Why the exit path matters more than the launch</h2>



<p class="wp-block-paragraph">All three services will host a small workload. The launch is not the hard part. What separates them is what happens eighteen months later when the requirements change.</p>



<p class="wp-block-paragraph">Lightsail is the clearest case because its boundaries are documented. Instance and disk snapshots export to EC2 as an AMI and EBS snapshots. Managed databases, container services, distributions, load balancers, static IPs, and DNS records do not export at all. AWS also warns that a Linux instance built from an exported Lightsail snapshot can carry residual SSH keys from the Lightsail service, which you should remove.</p>



<p class="wp-block-paragraph">So the question is not &#8220;which is cheapest for a small site.&#8221; It is &#8220;how much of this stack is portable, and who operates it after handover.&#8221;</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Amazon Lightsail: where it wins and where it doesn&#8217;t</h2>



<p class="wp-block-paragraph">Lightsail bundles compute, SSD storage, and a monthly data transfer allowance into one flat plan price. There are no separate line items for the instance, the volume, and the egress. For a client who wants to know what the bill will be, that is worth a lot more than a theoretical discount.</p>



<h3 class="wp-block-heading">Where it wins</h3>



<ul class="wp-block-list">
<li>Predictable billing. One number per month, plus egress overage only if you blow through the allowance. Easy to quote, easy to hand to a client.</li>

<li>The data transfer allowance is generous for content sites. Inbound and outbound both count toward it, but only outbound overage is billed, and transfer between Lightsail resources over private addresses in the same region is free.</li>

<li>Console surface small enough to hand to a non-engineer. A client can restart an instance or take a snapshot without you, and without an IAM policy discussion.</li>

<li>IPv6-only bundles exist and cost less than the IPv4 equivalents, which is a real saving if the workload sits behind Cloudflare or another proxy anyway.</li>
</ul>



<h3 class="wp-block-heading">Where it doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Networking is deliberately shallow. Lightsail resources live in a Lightsail-managed VPC, and connecting to the rest of AWS means enabling VPC peering with the <em>default</em> VPC in the same account and the same region. Not a custom VPC. Not another account.</li>

<li>Peering is a region-level switch, not a per-instance setting, and the CIDR ranges must not overlap. The route table on the Lightsail side is managed by AWS and you cannot edit it.</li>

<li>The exit is partial, and the instance type list is short. There is no picking a memory-optimized family because the workload turned out to be memory-bound.</li>

<li>No Savings Plans, no Reserved Instances, no Spot. The flat rate is the rate.</li>
</ul>



<p class="wp-block-paragraph">Worth saying plainly: if all you want is a flat-rate box, Lightsail competes with plain VPS hosts as much as with EC2. Providers like Contabo and InterServer sell the same shape of product, often with more RAM per unit of spend. Lightsail&#8217;s edge is that it sits inside an AWS account, so IAM, billing, and eventual migration to EC2 stay in one place. If the client will never touch another AWS service, that edge is worth less than it looks.</p>



<h2 class="wp-block-heading">Amazon EC2: where it wins and where it doesn&#8217;t</h2>



<p class="wp-block-paragraph">EC2 is the general case: a real VPC, real subnets, real security groups, and every instance family AWS sells. You also get to build all of that yourself.</p>



<h3 class="wp-block-heading">Where it wins</h3>



<ul class="wp-block-list">
<li>Instance selection. If profiling shows the workload is memory-bound rather than CPU-bound, you change families instead of buying a bigger everything. For a PHP or Node app with an ARM-clean dependency tree, the Graviton families are usually the easiest price-performance win on the table.</li>

<li>Commitment discounts. Savings Plans and Reserved Instances make a steady 24/7 workload meaningfully cheaper, and Spot makes non-critical work cheaper still.</li>

<li>Full networking. Custom VPCs, private subnets, VPC endpoints, cross-account peering, Transit Gateway. If the client already has an AWS footprint, EC2 joins it without a workaround.</li>

<li>Systems Manager. Session Manager gives you shell access without opening port 22 to the internet, and Patch Manager gives you a defensible patching story at handover. If you would rather have a private network path to admin interfaces than an AWS-native one, something like NordLayer or a self-hosted WireGuard box fills the same slot.</li>
</ul>



<h3 class="wp-block-heading">Where it doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Every component is billed and configured separately: instance, EBS volume, snapshots, egress, Elastic IP, load balancer. The bill has more surface area, and so does the misconfiguration.</li>

<li>Public IPv4 addresses are billed per hour, including on running instances. On a fleet of one that is noise. Across a dozen small instances it stops being noise.</li>

<li>Data transfer out to the internet is metered per GB with no bundled allowance. A media-heavy site that would sit comfortably inside a Lightsail plan can produce a surprising egress line on EC2.</li>

<li>You own the operating system: patching, kernel updates, log rotation, backup verification. Billable work if you are the consultant, abandoned work if you are not.</li>
</ul>



<p class="wp-block-paragraph">The mitigation for the egress point is the usual one: put a CDN in front. CloudFront or Cloudflare both change the shape of that bill substantially, because most bytes stop leaving the origin at all.</p>



<h2 class="wp-block-heading">Amazon ECS: where it wins and where it doesn&#8217;t</h2>



<p class="wp-block-paragraph">ECS is two different products wearing one name, and conflating them is where most of the confusion comes from.</p>



<ul class="wp-block-list">
<li><strong>ECS on Fargate</strong> bills per vCPU-hour and GB-hour of the resources you <em>declare in the task definition</em>, not the resources you use. Over-provision a task and you pay for the over-provisioning at full rate.</li>

<li><strong>ECS on EC2</strong> means you run the cluster instances and ECS schedules containers onto them. Savings Plans and Spot apply again, and you own the host OS again.</li>
</ul>



<h3 class="wp-block-heading">Where it wins</h3>



<ul class="wp-block-list">
<li>The deployment story is genuinely better. Rolling updates with health checks and automatic rollback are configuration, not a script you wrote and hope still works.</li>

<li>Immutable artifacts. The image that passed CI is the image that runs. No drift from someone running a package update on the box in March.</li>

<li>Fargate removes host patching entirely, which for a small team is a real reduction in unowned work.</li>

<li>Scaling to and from zero is straightforward for scheduled or bursty jobs. A nightly batch task on Fargate costs nothing between runs.</li>

<li>If the app is already containerized and CI already builds images, you are most of the way there.</li>
</ul>



<h3 class="wp-block-heading">Where it doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>The supporting cast costs more than the compute on small workloads. A load balancer, a NAT gateway, CloudWatch Logs ingestion, and a registry are the floor, and none of them scale down with your one small container.</li>

<li>Tasks in private subnets pull every image layer through the NAT gateway, which bills per GB processed. Frequent deploys of a large image is a cost pattern that looks like nothing on a diagram.</li>

<li>Persistent local state is awkward, so stateful workloads generally mean EFS or a managed database.</li>

<li>Handover is harder. A client&#8217;s in-house developer can usually SSH into a box. Far fewer can debug a task that fails its health check before it produces a log line.</li>
</ul>



<p class="wp-block-paragraph">The NAT gateway problem has a standard fix, worth knowing before you price the work. Put interface endpoints in the VPC for the ECR API and the ECR Docker registry, plus a gateway endpoint for S3, because ECR stores image layers in S3 and that is where the bulk of the bytes actually move. Add interface endpoints for CloudWatch Logs and Secrets Manager if your tasks use them.</p>



<pre class="wp-block-code"><code># Interface endpoint for the ECR Docker registry (image layer pulls)
aws ec2 create-vpc-endpoint 
  --vpc-id "$VPC_ID" 
  --vpc-endpoint-type Interface 
  --service-name "com.amazonaws.$AWS_REGION.ecr.dkr" 
  --subnet-ids "$SUBNET_A" "$SUBNET_B" 
  --security-group-ids "$SG_ID" 
  --private-dns-enabled

# Gateway endpoint for S3: no hourly charge, and it carries most of the data
aws ec2 create-vpc-endpoint 
  --vpc-id "$VPC_ID" 
  --vpc-endpoint-type Gateway 
  --service-name "com.amazonaws.$AWS_REGION.s3" 
  --route-table-ids "$PRIVATE_RTB_ID"</code></pre>



<p class="wp-block-paragraph">Interface endpoints carry their own hourly charge per endpoint per availability zone, so this is a trade, not a free win. Below some volume of image pulls the NAT gateway is cheaper; above it, endpoints are. Work out which side you are on before you build either.</p>



<h2 class="wp-block-heading">Lightsail vs EC2 vs ECS on the lines that move the bill</h2>



<p class="wp-block-paragraph">Compute is rarely the biggest number on a small AWS workload. These are the levers that actually decide it, and they are the ones I check first on any account I inherit.</p>



<ul class="wp-block-list">
<li><strong>Egress.</strong> Lightsail bundles an allowance. EC2 and ECS meter it. This single difference can invert the comparison for a media-heavy site.</li>

<li><strong>NAT gateway.</strong> Charged per hour and per GB processed. On ECS with private subnets it is usually the largest fixed cost in the account. Lightsail has no equivalent line.</li>

<li><strong>Load balancing.</strong> Lightsail&#8217;s load balancer is a flat monthly price. An Application Load Balancer is an hourly charge plus a capacity-unit charge. AWS publishes traffic thresholds above which it recommends moving to an ALB; if you are near them, you have outgrown the simple option already.</li>

<li><strong>Public IPv4.</strong> Billed per address per hour on EC2 and on task ENIs. It is small per unit and easy to forget across an environment.</li>

<li><strong>Log ingestion.</strong> CloudWatch Logs bills on ingested volume. A chatty container at debug level can out-cost the container. Set retention on every log group at creation time, not later.</li>

<li><strong>Idle resources.</strong> Unattached static IPs and orphaned EBS volumes bill quietly forever. Reclaim them with an offboarding checklist, not a quarterly panic.</li>
</ul>



<h2 class="wp-block-heading">A decision procedure for Lightsail vs EC2 vs ECS</h2>



<p class="wp-block-paragraph">Run these in order. The first one that gives a clear answer is the answer.</p>



<ol class="wp-block-list">
<li><strong>Does this workload need to talk privately to other AWS resources?</strong> If it needs a custom VPC, a private RDS instance in a non-default VPC, cross-account access, or PrivateLink, Lightsail is out. Its peering only reaches the default VPC in the same account and region.</li>

<li><strong>Who operates this in a year?</strong> If the answer is &#8220;the client&#8217;s office manager&#8221; or &#8220;nobody,&#8221; bias hard toward the simplest thing that works. That is usually Lightsail, or a managed platform outside AWS entirely.</li>

<li><strong>Is the app already containerized, with CI that builds images?</strong> If yes, ECS is cheap to adopt. If no, containerizing it purely to use ECS is a project, and you should price it as one.</li>

<li><strong>Is the load steady or spiky?</strong> Steady 24/7 favors a committed EC2 instance. Spiky or scheduled favors Fargate, because you stop paying between runs.</li>

<li><strong>How much egress?</strong> Estimate monthly GB out. If it is large and cache-friendly, put a CDN in front and the question mostly dissolves. If it is large and not cacheable, Lightsail&#8217;s bundled allowance is a real argument.</li>

<li><strong>What is the audit story?</strong> If the client needs detailed IAM boundaries, VPC flow logs, or GuardDuty coverage across the workload, you want a normal VPC. That means EC2 or ECS.</li>
</ol>



<p class="wp-block-paragraph">If you land on Lightsail and want to keep the door open, take manual snapshots on a schedule and confirm the export path works <em>before</em> you need it. The commands are short:</p>



<pre class="wp-block-code"><code># Take a manual snapshot of the instance
aws lightsail create-instance-snapshot 
  --instance-name my-instance 
  --instance-snapshot-name my-instance-export-test

# Export it to EC2 as an AMI plus EBS snapshot, in the same region
aws lightsail export-snapshot 
  --source-snapshot-name my-instance-export-test

# Watch the export until the state reports completed
aws lightsail get-export-snapshot-records</code></pre>



<p class="wp-block-paragraph">Do this once at project start. Finding out that a Django or Ghost blueprint instance cannot be exported is a fact you want on day one, not cutover day.</p>



<p class="wp-block-paragraph">If you need Lightsail to reach an RDS database in the default VPC, peering is a region-level toggle. Enable it, then confirm it rather than assuming:</p>



<pre class="wp-block-code"><code>aws lightsail peer-vpc --region eu-west-1
aws lightsail is-vpc-peered --region eu-west-1</code></pre>



<p class="wp-block-paragraph">You still have to allow the traffic on both sides. The security group on the AWS resource has to accept the Lightsail CIDR range, and the Lightsail instance firewall has to allow the return path. Peering enabled is not the same as traffic flowing, and that gap is where an afternoon disappears.</p>



<p class="wp-block-paragraph">Whichever you pick, put monitoring on it on day one. CloudWatch covers the AWS-side metrics; Grafana Cloud or Datadog give you a better view across a mixed estate. The failure you are guarding against is not a dramatic outage, it is a slow degradation nobody notices until a client emails.</p>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<h3 class="wp-block-heading">&#8220;Lightsail is just EC2 with a nicer console&#8221;</h3>



<p class="wp-block-paragraph">Same infrastructure underneath, different networking model, and that is the part that bites. A Lightsail instance is not sitting in your VPC. Anything that assumes a normal VPC, from a security group reference to a VPC endpoint, needs a rethink.</p>



<h3 class="wp-block-heading">&#8220;Start on Lightsail, migrate to EC2 when you outgrow it&#8221;</h3>



<p class="wp-block-paragraph">Half true. The instance migrates cleanly. The database, load balancer, distribution, static IP, and DNS do not. It is a real migration project, so plan for it as one rather than treating it as an upgrade button.</p>



<h3 class="wp-block-heading">&#8220;Fargate is serverless, so it scales to zero and costs nothing when idle&#8221;</h3>



<p class="wp-block-paragraph">The tasks scale to zero. The load balancer, the NAT gateway, and the VPC endpoints do not. On a small workload the always-on infrastructure usually dominates, which is exactly backwards from the intuition the word &#8220;serverless&#8221; gives you.</p>



<h3 class="wp-block-heading">&#8220;Containers are better practice, so ECS is the professional choice&#8221;</h3>



<p class="wp-block-paragraph">Containers are better practice for reproducibility, and that benefit is mostly delivered by the image and the CI pipeline, not by the orchestrator. A container running under systemd on one EC2 instance captures most of the win at a fraction of the operational surface. Reach for ECS when you need scheduling, rolling deploys, and health-check-driven replacement.</p>



<h3 class="wp-block-heading">&#8220;Reserved Instances make EC2 cheaper, so always pick EC2&#8221;</h3>



<p class="wp-block-paragraph">Commitment discounts are worth real money on steady workloads. They are also a one or three year commitment on a client relationship that may not last that long. Committing on a client&#8217;s behalf is a conversation, not a default.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Is Lightsail cheaper than EC2 for a small website?</h3>



<p class="wp-block-paragraph">Usually, for a steady small workload, because the bundle includes storage and a data transfer allowance you would pay for separately on EC2. The comparison flips when you can apply a Savings Plan or Reserved Instance to a long-running EC2 instance, or when you need an instance family Lightsail does not offer. Price both against your actual egress volume rather than comparing headline compute rates.</p>



<h3 class="wp-block-heading">Can a Lightsail instance connect to an RDS database?</h3>



<p class="wp-block-paragraph">Yes, through VPC peering, with conditions. Peering is between your Lightsail VPC and the <em>default</em> VPC in the same account and region, a default VPC has to exist, and the CIDR blocks must not overlap. Enable it in every region where you have Lightsail resources you want connected.</p>



<h3 class="wp-block-heading">Does everything migrate when I export Lightsail to EC2?</h3>



<p class="wp-block-paragraph">No. Instance snapshots and block storage disk snapshots export. Managed databases, container services, CDN distributions, load balancers, static IPs, and DNS records do not. Certain blueprint instances, including Django, Ghost, and cPanel &amp; WHM, cannot be exported at all. Check your specific blueprint before you commit to a migration date.</p>



<h3 class="wp-block-heading">Is ECS overkill for a single small container?</h3>



<p class="wp-block-paragraph">Often, yes. For one container with no scaling requirement, running it under systemd or Docker Compose on a single instance gets the same immutable-artifact benefit without a load balancer, a NAT gateway, or a cluster to reason about. ECS earns its place when you need rolling deploys with automatic rollback, or more than a couple of services.</p>



<h3 class="wp-block-heading">Why is my Fargate bill higher than the vCPU-hour rate suggests?</h3>



<p class="wp-block-paragraph">Two reasons, usually. Fargate bills the CPU and memory you declared in the task definition, not what the container consumed, so over-provisioning is billed at full rate. And the surrounding infrastructure (load balancer, NAT gateway data processing, public IPv4 addresses, CloudWatch Logs ingestion) is charged separately and often exceeds the compute on small services.</p>



<h3 class="wp-block-heading">Should I use Lightsail container service instead of ECS?</h3>



<p class="wp-block-paragraph">Reasonable middle option if you want containers with flat-rate billing and no VPC to design. The trade is the same as everywhere else in Lightsail: limited configuration, and container services are among the resources that do not export to EC2. Treat it as a destination, not a stepping stone.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">Lightsail vs EC2 vs ECS is not a question about performance. All three will serve a small client workload perfectly well. It is a question about which constraints you are accepting and how expensive they are to reverse.</p>



<p class="wp-block-paragraph">One site with no other AWS resources and a client-visible bill: Lightsail. Anything needing a private database, a custom VPC, or an existing AWS footprint: EC2. Already containerized, deploying weekly, with someone technical staying involved: ECS. And if nobody will operate it after handover, push back on all three and price managed hosting instead, because an unpatched instance with no owner is worse than a more expensive managed platform.</p>



<p class="wp-block-paragraph">Lightsail trades networking depth and portability for a bill you can put in a proposal. EC2 trades operational work for control. ECS trades a higher fixed floor for deployment mechanics that stop being your problem. Pick based on who operates the thing in a year and how much has to be rebuilt when requirements change, not on which service sounds most professional in a kickoff call.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second opinion before you commit?</h2>



<p class="wp-block-paragraph">Most of the cost and portability problems above are cheap to avoid at design time and expensive to unwind later. What I help with on exactly this decision:</p>



<ul class="wp-block-list">
<li>Sizing a Lightsail, EC2, or ECS deployment against real traffic and egress numbers instead of guesses</li>

<li>Working out whether VPC endpoints or a NAT gateway is cheaper for your specific image size and deploy frequency</li>

<li>Planning and executing Lightsail to EC2 migrations, including the parts that do not export</li>

<li>Setting up VPC peering between Lightsail and RDS, and debugging why traffic still is not flowing after you enabled it</li>

<li>Auditing an inherited AWS account for idle addresses, orphaned volumes, and log groups with no retention policy</li>

<li>Writing the handover runbook so the next person can operate it without calling you</li>
</ul>



<p class="wp-block-paragraph">If you want a straight answer on which of the three fits, send me a month of Cost Explorer output or a task definition and I will tell you what I would change.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/lightsail-vs-ec2-vs-ecs-small-workloads/">Lightsail vs EC2 vs ECS: What I Actually Pick for Small Client Workloads</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Salesforce AWS Integration Architecture: The Four Paths and What Breaks Silently</title>
		<link>https://john-nessime.com/blog/devops/salesforce-aws-integration-architecture/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Wed, 19 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Redshift]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[AppFlow]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Lambda]]></category>
		<category><![CDATA[Change Data Capture]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[ETL]]></category>
		<category><![CDATA[Event Relay]]></category>
		<category><![CDATA[EventBridge]]></category>
		<category><![CDATA[Named Credentials]]></category>
		<category><![CDATA[OAuth]]></category>
		<category><![CDATA[Platform Events]]></category>
		<category><![CDATA[PrivateLink]]></category>
		<category><![CDATA[Pub/Sub API]]></category>
		<category><![CDATA[Salesforce]]></category>
		<category><![CDATA[Serverless]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=209</guid>

					<description><![CDATA[<p>Most Salesforce AWS integrations don't fail loudly, they drift. This guide covers the four integration paths between Salesforce and AWS, when to use each, the silent failure mode hiding in every one of them, and the reconciliation job you have to build yourself.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/salesforce-aws-integration-architecture/">Salesforce AWS Integration Architecture: The Four Paths and What Breaks Silently</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Someone in the ops channel asks why the opportunity count on the QuickSight dashboard doesn&#8217;t match the count in the Salesforce report. It&#8217;s off by a few hundred. Nobody can answer that, and worse, nobody can say when the two numbers stopped agreeing. Every pipeline is green. The scheduled flow reports successful runs. The EventBridge rule shows invocations. Everything is healthy except the data.</p>



<p class="wp-block-paragraph">That&#8217;s the shape of most Salesforce AWS integration problems. They&#8217;re rarely outages. They&#8217;re divergence, accumulating quietly while every status page stays green, and you find out about it when someone in finance runs a number past a customer.</p>



<p class="wp-block-paragraph">This guide covers the four integration paths between Salesforce and AWS, which one to reach for and when, the specific failure each one hides, and the reconciliation layer you&#8217;ll have to build yourself because neither vendor ships it.</p>



<h2 class="wp-block-heading">Why a Salesforce AWS integration fails silently</h2>



<p class="wp-block-paragraph">Start here, because it changes how you design everything downstream. There are three mechanisms that lose data without raising an error anywhere you&#8217;re looking.</p>



<p class="wp-block-paragraph"><strong>Events sent to an unassociated partner event bus are dropped on the floor.</strong> When you create an Event Relay, Salesforce creates a partner event source in your AWS account. Until you associate that source with an event bus, EventBridge discards anything the partner publishes to it. AWS is explicit that those events are not persisted at rest. There&#8217;s no queue holding them for you, no backlog to drain once you finish the setup. If the relay is running and the bus isn&#8217;t associated, you are streaming into a void.</p>



<p class="wp-block-paragraph"><strong>The Salesforce event bus retains high-volume events for 72 hours.</strong> Platform events and change data capture events sit on the event bus for three days, and each one carries a replay ID so a durable subscriber can resume from where it left off. Past that window, they&#8217;re purged. Salesforce doesn&#8217;t guarantee availability beyond it. So a relay that goes into an error state on a Friday afternoon is a data loss event by Monday morning, and the only symptom during those three days is an absence of messages, which looks identical to a quiet weekend.</p>



<p class="wp-block-paragraph"><strong>Deletes don&#8217;t show up in query-based syncs.</strong> A scheduled pull selects records that exist. Records that stopped existing between runs are simply not in the result set, so your S3 prefix or Redshift table keeps a row that Salesforce no longer has. Nothing errors. The row count on the AWS side just creeps upward relative to the CRM, forever, until someone notices.</p>



<p class="wp-block-paragraph">Every architecture decision below is really a decision about which of these three you&#8217;re accepting and how you&#8217;re going to detect it.</p>



<h2 class="wp-block-heading">A Salesforce AWS integration starts with a direction, not a service</h2>



<p class="wp-block-paragraph">The most common mistake I see is picking a service first, usually whichever one appeared in a conference demo, and then bending the requirement to fit it. Two questions decide the path, and they take about a minute to answer.</p>



<ol class="wp-block-list">
<li><strong>Which way does the data move?</strong> Salesforce to AWS, or AWS to Salesforce. If the answer is &#8220;both&#8221;, you have two integrations, not one, and they will have different failure modes and different owners.</li>

<li><strong>What shape is the workload?</strong> Batch, where you want a consistent set of records on a schedule, or event-driven, where you want a reaction within seconds of a change.</li>
</ol>



<p class="wp-block-paragraph">That gives you four cells, and each cell has a default answer.</p>



<h3 class="wp-block-heading">Salesforce to AWS, batch: Amazon AppFlow</h3>



<p class="wp-block-paragraph">Amazon AppFlow is the managed answer for pulling Salesforce objects into S3, Redshift, or the Glue Data Catalog on a schedule or on demand. It handles the OAuth connection, field mapping, and incremental pulls without you writing or hosting anything. For analytics loads it&#8217;s usually the right default, and it&#8217;s the path of least resistance if the destination is a data lake you&#8217;re already cataloguing with Glue and querying with Athena.</p>



<p class="wp-block-paragraph">The trade-off is that it&#8217;s a polling client, and it spends your Salesforce API allocation to do it. AWS documents this plainly with an example: an hourly flow pulling five pages of data makes 120 API calls a day, and all of them count against the org&#8217;s 24-hour API request limit. Multiply that by every object you&#8217;re syncing and every environment you&#8217;re syncing from, and AppFlow can become the largest single consumer of an allocation that Apex callouts, integration users, and third-party tools are also drawing on.</p>



<p class="wp-block-paragraph">The failure mode that catches people: if a field referenced in the flow&#8217;s mapping is deleted in Salesforce, the flow run fails. Someone cleans up an unused custom field on a Tuesday and your nightly load breaks on Wednesday, in a service that nobody on the Salesforce side knows exists. AppFlow will automatically pick up newly created Salesforce fields when the destination is S3, but the reverse case, a removed field, needs a human to edit the mapping. Put the flow names somewhere the Salesforce admins will see them before they run a field cleanup.</p>



<h3 class="wp-block-heading">Salesforce to AWS, event-driven: Event Relay and Amazon EventBridge</h3>



<p class="wp-block-paragraph">Event Relay subscribes to a channel on the Salesforce event bus and streams platform events and change data capture events into Amazon EventBridge as a partner event source. No middleware, no subscriber you have to keep alive, no polling and therefore no API allocation burn on the read path. From EventBridge you fan out to Lambda, Step Functions, SQS, Firehose, whatever the downstream is.</p>



<p class="wp-block-paragraph">Setup runs across both consoles, and the order matters. On the Salesforce side you create a named credential that points at your AWS account and region. It uses no authentication protocol, because trust is established by the partner event source relationship rather than by a credential in the request. The region goes in uppercase, which is the single most common reason a first attempt doesn&#8217;t work.</p>



<pre class="wp-block-code"><code>POST /services/data/vXX.X/tooling/sobjects/NamedCredential/

{
  "FullName": "AwsRelayCredential",
  "Metadata": {
    "label": "AwsRelayCredential",
    "endpoint": "arn:aws:US-EAST-1:111122223333",
    "principalType": "NamedUser",
    "protocol": "NoAuthentication",
    "generateAuthorizationHeader": true
  }
}</code></pre>



<p class="wp-block-paragraph">The endpoint is not a URL. It&#8217;s an ARN-shaped string carrying the AWS region and the twelve-digit account ID, and that&#8217;s the whole addressing scheme. Get the account wrong and the partner event source appears in a stranger&#8217;s console, not yours.</p>



<p class="wp-block-paragraph">You then create an event channel, add the platform events or change events you want as channel members, and create the relay configuration against that channel. A relay is created in a stopped state deliberately, so you get a chance to verify the AWS account and region before anything flows. That pause is the useful part of the design, not an inconvenience to click through.</p>



<p class="wp-block-paragraph">Now switch to AWS and associate the source. This is the step that, skipped, produces the silent drop described earlier.</p>



<pre class="wp-block-code"><code># Find the partner event source Salesforce created in your account
aws events list-event-sources --region us-east-1

# Create the matching bus. Both values must be the source name, character for character.
aws events create-event-bus 
  --name "aws.partner/salesforce.com/&lt;org-and-channel-suffix&gt;" 
  --event-source-name "aws.partner/salesforce.com/&lt;org-and-channel-suffix&gt;" 
  --region us-east-1

# Confirm the source moved from PENDING to ACTIVE
aws events describe-event-source 
  --name "aws.partner/salesforce.com/&lt;org-and-channel-suffix&gt;" 
  --region us-east-1</code></pre>



<p class="wp-block-paragraph"><code>describe-event-source</code> returning <code>ACTIVE</code> is your proof that a matching bus exists and is enabled. <code>PENDING</code> means either you haven&#8217;t created the bus or it&#8217;s deactivated, and every event published in that state is gone. This is the single check worth wiring into a synthetic monitor, because it&#8217;s cheap to run and it catches the most expensive mistake.</p>



<p class="wp-block-paragraph">Only once the source is active do you set the relay state to run. From the Salesforce side, the feedback object is where relay errors surface rather than in any log you&#8217;d normally tail:</p>



<pre class="wp-block-code"><code>SELECT Id, RemoteResource, Status, ErrorMessage, ErrorTime, ErrorIdentifier
FROM EventRelayFeedback
WHERE EventRelayConfigId = '7k2XXXXXXXXXXXXXXX'</code></pre>



<p class="wp-block-paragraph">If you&#8217;d rather not use Event Relay at all, the Pub/Sub API is the direct route: a gRPC interface to the same event bus, delivering Avro-encoded payloads, which you subscribe to from your own consumer. That&#8217;s more code and a process to keep running, on ECS or a VPS from a provider like InterServer or Hetzner if you&#8217;re keeping it outside AWS, but you get explicit control over replay ID checkpointing. Teams already running Kafka often go this way and land the stream through a managed connector from Confluent instead. Event Relay is the lower-operational-cost option; the Pub/Sub API is the one to reach for when you need the checkpoint under your own control.</p>



<h3 class="wp-block-heading">AWS to Salesforce, event-driven: EventBridge API Destinations</h3>



<p class="wp-block-paragraph">Going the other way, an API destination lets EventBridge call the Salesforce REST API directly, with no Lambda in the middle. You create a connection holding OAuth client credentials from a Salesforce connected app, point the destination at your org&#8217;s My Domain endpoint, and route matched events to it. Publishing back into an inbound platform event is the common pattern, so the write lands on the Salesforce event bus and Apex or Flow picks it up from there.</p>



<p class="wp-block-paragraph">Two constraints shape the design, and both are documented:</p>



<ul class="wp-block-list">
<li><strong>Five second client execution timeout.</strong> If Salesforce takes longer than that to respond, EventBridge times out the request and treats it as a retryable failure. Anything you make synchronous behind that endpoint, a trigger doing a callout, a validation chain, a busy org at peak, will eat into it.</li>

<li><strong>The default retry policy is generous, which is the problem.</strong> For event buses, EventBridge will retry a failed delivery for up to 24 hours and up to 185 times by default. If the endpoint is genuinely broken, that&#8217;s a day of retries against your org, consuming API calls, before the event is discarded with nowhere to go.</li>
</ul>



<p class="wp-block-paragraph">Tighten the policy and always attach a dead letter queue. Failed events land somewhere you can inspect and replay rather than evaporating at the end of the retry window:</p>



<pre class="wp-block-code"><code>{
  "Id": "salesforce-writeback",
  "Arn": "arn:aws:events:us-east-1:111122223333:api-destination/sf-platform-event/&lt;id&gt;",
  "RoleArn": "arn:aws:iam::111122223333:role/EventBridgeApiDestinationRole",
  "RetryPolicy": {
    "MaximumRetryAttempts": 20,
    "MaximumEventAgeInSeconds": 3600
  },
  "DeadLetterConfig": {
    "Arn": "arn:aws:sqs:us-east-1:111122223333:salesforce-writeback-dlq"
  }
}</code></pre>



<p class="wp-block-paragraph">Set the invocation rate limit on the destination to something your org can absorb. It exists specifically so you can align delivery with a third-party API&#8217;s limits, and Salesforce is very much a third-party API with limits.</p>



<p class="wp-block-paragraph">One thing worth knowing before you design around it: API destinations don&#8217;t support mutual TLS. If your security review requires mTLS on outbound calls, you&#8217;re putting a Lambda or a proxy in the path regardless.</p>



<h3 class="wp-block-heading">AWS to Salesforce, batch: Lambda and the Bulk API</h3>



<p class="wp-block-paragraph">For loading volume back into the org, nothing managed does this well and you write it yourself. A Lambda or a container job authenticates with the JWT bearer flow against a connected app, submits a Bulk API job, and polls for results. The REST API is fine for small writes; past a few thousand records the per-call overhead and the API allocation make the Bulk API the only sensible choice.</p>



<p class="wp-block-paragraph">This is the cell where you own everything: retries, idempotency, partial-failure handling, and backoff when the org throttles you. Use external ID fields and upsert semantics rather than insert, so a retried batch converges instead of duplicating. Store the job ID before you submit, not after, so a Lambda timeout mid-flight leaves you something to reconcile against. And keep the Salesforce credential in Secrets Manager with rotation wired up, because a hardcoded connected app secret in an environment variable is the finding that shows up in every audit.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The network layer: public internet, PrivateLink, or Private Connect</h2>



<p class="wp-block-paragraph">All four paths above work over the public internet with TLS, and for most workloads that&#8217;s genuinely fine. The question is whether your compliance posture or your risk appetite says otherwise.</p>



<p class="wp-block-paragraph">Salesforce Private Connect is the managed private path. Salesforce operates a transit VPC in each supported AWS region, and you create a PrivateLink connection between that transit VPC and yours. Traffic between the two never traverses the public internet. The terminology is worth getting right because it trips people up in design reviews:</p>



<ul class="wp-block-list">
<li><strong>Inbound</strong> means traffic flowing <em>into</em> Salesforce. AWS to Salesforce, or on-premises to Salesforce.</li>

<li><strong>Outbound</strong> means traffic flowing <em>out of</em> Salesforce. Salesforce calling a service in your VPC.</li>
</ul>



<p class="wp-block-paragraph">The direction is named from Salesforce&#8217;s point of view, not yours, and getting it backwards means provisioning the wrong connection.</p>



<p class="wp-block-paragraph">AppFlow can ride on this too. Choosing the PrivateLink option when you create the Salesforce connection has AppFlow route through Private Connect and manage the endpoint lifecycle for you, which is a meaningful simplification over building it by hand.</p>



<p class="wp-block-paragraph">Where I&#8217;d push back: Private Connect is a licensed add-on, and each VPC needs its own connection, so a multi-account AWS estate with separate VPCs per environment multiplies the cost. If the driver is a checkbox on a questionnaire rather than a real threat model, TLS with strict IP allowlisting on the Salesforce side and a well-scoped connected app gets you most of the way for nothing. If the driver is a regulator who wants to see that customer data never touches a public network path, Private Connect is the clean answer and the audit conversation ends quickly. Decide which one you&#8217;re in before you start pricing it.</p>



<h2 class="wp-block-heading">The reconciliation job nobody builds</h2>



<p class="wp-block-paragraph">This is the part that separates an integration that survives from one that quietly rots, and it&#8217;s about thirty lines of code plus a schedule.</p>



<p class="wp-block-paragraph">Run a count comparison on a window that both sides can express. Nightly is plenty:</p>



<pre class="wp-block-code"><code>-- Salesforce side, via the REST query endpoint
SELECT COUNT(Id) FROM Opportunity
WHERE LastModifiedDate = LAST_N_DAYS:1

-- AWS side, same window, in Athena
SELECT COUNT(DISTINCT id) FROM crm.opportunity
WHERE last_modified_date &gt;= date_add('day', -1, current_date)</code></pre>



<p class="wp-block-paragraph">If the two numbers differ by more than a small tolerance for clock skew at the window edges, alert. That single check catches a stopped relay, a failed flow, a mapping that silently dropped a field, and a partner event source sitting in <code>PENDING</code>. It catches all of them without you having to instrument each path separately, which is exactly why it&#8217;s worth building first.</p>



<p class="wp-block-paragraph">For the delete problem, Salesforce exposes a dedicated endpoint. Ask it for records removed in a window and apply those deletions on the AWS side:</p>



<pre class="wp-block-code"><code>GET /services/data/vXX.X/sobjects/Opportunity/deleted/
    ?start=&lt;ISO-8601 timestamp, URL encoded&gt;
    &amp;end=&lt;ISO-8601 timestamp, URL encoded&gt;</code></pre>



<p class="wp-block-paragraph">The window you can ask about is bounded by how long deleted records stay recoverable, so run it more often than that bound, not less. Change data capture delivers delete events too, which is the better answer when it&#8217;s available, but the endpoint is what you use to backfill after a gap, and after a 72-hour gap it&#8217;s the <em>only</em> thing that will tell you what you missed.</p>



<p class="wp-block-paragraph">Put both checks on one dashboard alongside your AppFlow run status and EventBridge failed-invocation metrics. CloudWatch will do it, and if you&#8217;re already running Grafana Cloud or Datadog for the rest of the estate, pull the Salesforce-side numbers in as a custom metric so one panel answers &#8220;are the two systems agreeing right now&#8221;.</p>



<h2 class="wp-block-heading">Troubleshooting a Salesforce AWS integration</h2>



<ul class="wp-block-list">
<li><strong>Relay is running, nothing arrives in EventBridge.</strong> Run <code>describe-event-source</code>. If the state is <code>PENDING</code>, the bus was never created or has been deactivated, and everything published so far is gone. Create the bus, then treat the gap as a backfill job.</li>

<li><strong>Relay won&#8217;t start, or flips to an error state.</strong> Query <code>EventRelayFeedback</code> for the config. The usual causes are a wrong AWS account ID in the named credential, a region in lowercase, or the channel having no members.</li>

<li><strong>Events arrive but no rule matches.</strong> Check the rule is attached to the partner event bus, not the default bus. This one costs people an afternoon. Match on the source prefix and the detail type, and test with an event pattern before wiring a real target.</li>

<li><strong>An AppFlow flow that ran for months suddenly fails.</strong> Look for a field in the mapping that no longer exists in the source object. Also check for a bad gateway error, which points at a connection configured with the wrong instance URL and is fixed by deleting the connection and recreating it rather than editing it.</li>

<li><strong>API destination returns 401 intermittently.</strong> The connection refreshes its OAuth token when it sees a 401 or 407. Persistent 401s usually mean the connected app has no usable scopes for the client credentials flow, or the run-as user was changed. Check the connected app&#8217;s scopes before you touch anything in AWS.</li>

<li><strong>Everything times out at almost exactly five seconds.</strong> That&#8217;s the API destination timeout, not Salesforce being slow. Move the work behind an inbound platform event so the HTTP call returns immediately and the processing happens asynchronously in the org.</li>

<li><strong>Unexplained API limit exhaustion.</strong> Sum the scheduled AppFlow runs against page counts before blaming a rogue integration user. A frequent flow across several objects adds up faster than most people expect.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Starting the relay before associating the partner event bus, and assuming the events queued somewhere.</li>

<li>Treating the 72-hour retention as a comfortable buffer. It&#8217;s three days including a weekend, and an unmonitored relay burns through it without a single alert.</li>

<li>Using change data capture as a replacement for an initial load. It streams changes from the moment you enable it; it doesn&#8217;t backfill history.</li>

<li>Building the whole integration bidirectionally through one mechanism, then discovering you&#8217;ve created an echo loop where a write from AWS fires a change event that triggers another write.</li>

<li>Sizing the AppFlow schedule for freshness without checking what it does to the org&#8217;s API allocation.</li>

<li>Leaving the default 24-hour EventBridge retry policy on an API destination with no dead letter queue.</li>

<li>Provisioning Private Connect for every VPC because it&#8217;s the secure-sounding option, without modelling the per-connection cost across environments.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Build the reconciliation check before you build the second integration path. It&#8217;s the only thing that tells you the whole system is working.</li>

<li>Monitor the partner event source state, not just rule invocations. Invocations go to zero on a quiet day and on a broken day, and the state field distinguishes them.</li>

<li>Give every integration its own connected app and its own integration user. Shared credentials make it impossible to attribute API consumption or revoke one path without breaking three.</li>

<li>Make writes idempotent with external IDs and upsert. Every path here retries, and some retry aggressively.</li>

<li>Version your event payloads. Platform events are immutable once published, and adding a required field later means a new event definition, not an edit.</li>

<li>Keep the AWS-side resource names discoverable from the Salesforce side. Put the flow name or event bus name in the named credential label so a Salesforce admin can see what&#8217;s attached before they change something.</li>

<li>Define the whole thing in Terraform or CloudFormation on the AWS side and in metadata on the Salesforce side, so a sandbox refresh doesn&#8217;t leave you clicking through two consoles from memory.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Do I need MuleSoft to integrate Salesforce with AWS?</h3>



<p class="wp-block-paragraph">No. For the four patterns above, the native services cover it without an integration platform. MuleSoft, Boomi, and similar tools earn their place when you&#8217;re orchestrating across many systems, need transformation logic that lives outside both platforms, or have a governance requirement for a single integration layer. For a two-system integration, they&#8217;re usually more machinery than the problem needs.</p>



<h3 class="wp-block-heading">Does Event Relay use my Salesforce API allocation?</h3>



<p class="wp-block-paragraph">Event delivery counts against your platform event delivery allocation, which is a separate budget from the 24-hour API request limit that REST calls and AppFlow polling draw on. That separation is one of the strongest arguments for the event-driven path when API consumption is already tight in the org.</p>



<h3 class="wp-block-heading">Can I replay Salesforce events that AWS missed?</h3>



<p class="wp-block-paragraph">Within the 72-hour retention window, yes, using replay IDs from a durable subscriber. Beyond it, no. The events are purged and there&#8217;s no recovery path through the event bus. Backfilling from that point means querying the objects directly and using the deleted-records endpoint to catch removals, which is exactly why the reconciliation job matters.</p>



<h3 class="wp-block-heading">Should I use AppFlow or write my own Lambda extractor?</h3>



<p class="wp-block-paragraph">Start with AppFlow. It removes the auth handling, the pagination, and the scheduling, and for standard object pulls into S3 or Redshift it&#8217;s hard to justify writing that yourself. Move to a custom extractor when you need transformation logic AppFlow can&#8217;t express, when you&#8217;re hitting connector limitations on a specific object, or when you want tighter control over API consumption than a fixed schedule allows.</p>



<h3 class="wp-block-heading">How do I send data from AWS back into Salesforce without a Lambda?</h3>



<p class="wp-block-paragraph">EventBridge API destinations call the Salesforce REST API directly using OAuth client credentials from a connected app. Point them at an inbound platform event endpoint so the write returns fast and processing happens asynchronously in the org. You still need a Lambda if you require mTLS or payload transformation beyond what the input transformer handles.</p>



<h3 class="wp-block-heading">Is Private Connect required for a secure Salesforce AWS integration?</h3>



<p class="wp-block-paragraph">Not required. Everything runs over TLS by default. Private Connect removes the public internet from the path entirely, which matters when a regulator or a security review asks you to demonstrate it. It&#8217;s a licensed add-on with a per-connection cost, so model it against your actual VPC count before committing.</p>



<h3 class="wp-block-heading">Why does my EventBridge rule never fire even though events are arriving?</h3>



<p class="wp-block-paragraph">Almost always because the rule is on the default event bus rather than the partner event bus. Partner events land on their own bus and rules are bus-scoped, so a correct pattern on the wrong bus matches nothing and reports nothing.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">A Salesforce AWS integration doesn&#8217;t usually break loudly. It drifts. A relay stops, a mapping goes stale, a delete never propagates, and the two systems keep answering questions confidently with different numbers. The services themselves are well built and the setup is genuinely not hard once you know the region goes in uppercase and the bus has to be associated first.</p>



<p class="wp-block-paragraph">So pick the path by direction and shape, wire the failure modes you&#8217;ve accepted into alerts, and build the nightly count comparison before you build anything clever. That one job is worth more than any amount of pipeline polish, because it&#8217;s the only thing in the architecture that can tell you the whole thing is still true.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on your Salesforce and AWS setup?</h2>



<p class="wp-block-paragraph">I work with teams on the infrastructure side of CRM integrations, usually where the data is disagreeing and nobody can prove why. Things I can help with:</p>



<ul class="wp-block-list">
<li>Reviewing an existing Salesforce to AWS pipeline and finding where records are being lost between the two.</li>

<li>Standing up Event Relay into EventBridge end to end, including the partner bus association and the monitoring that proves it&#8217;s live.</li>

<li>Designing the AppFlow schedule and object set so it doesn&#8217;t eat the org&#8217;s API allocation.</li>

<li>Building the reconciliation and delete-propagation jobs, with alerting in CloudWatch, Grafana, or whatever you already run.</li>

<li>Writing the write-back path properly: Bulk API jobs, idempotent upserts, dead letter queues, and credential rotation through Secrets Manager.</li>

<li>Working out whether Private Connect is worth the licence in your specific environment, or whether TLS and tighter scoping gets you there.</li>
</ul>



<p class="wp-block-paragraph">If you&#8217;ve got a flow that&#8217;s failing, a relay stuck in an error state, or two record counts that won&#8217;t agree, send me the config or the error and I&#8217;ll tell you what I&#8217;d check first.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/salesforce-aws-integration-architecture/">Salesforce AWS Integration Architecture: The Four Paths and What Breaks Silently</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>How to Connect BI Tools to AWS Data Without a Runaway Bill</title>
		<link>https://john-nessime.com/blog/technical-guides/connect-bi-tools-to-aws-data/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Fri, 14 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Athena]]></category>
		<category><![CDATA[Amazon Redshift]]></category>
		<category><![CDATA[Apache Iceberg]]></category>
		<category><![CDATA[AWS Glue]]></category>
		<category><![CDATA[Business Intelligence]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[Data Lake]]></category>
		<category><![CDATA[FinOps]]></category>
		<category><![CDATA[IAM]]></category>
		<category><![CDATA[Identity Propagation]]></category>
		<category><![CDATA[JDBC]]></category>
		<category><![CDATA[Lake Formation]]></category>
		<category><![CDATA[Power BI]]></category>
		<category><![CDATA[PrivateLink]]></category>
		<category><![CDATA[Query Result Reuse]]></category>
		<category><![CDATA[Row-Level Security]]></category>
		<category><![CDATA[Tableau]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=231</guid>

					<description><![CDATA[<p>Getting Tableau or Power BI to return rows from your data lake takes an hour. The problems arrive three weeks later. Here are the four access patterns worth considering, why fine-grained security and cheap dashboards pull against each other, and the cost levers that actually move the invoice.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/connect-bi-tools-to-aws-data/">How to Connect BI Tools to AWS Data Without a Runaway Bill</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The request usually arrives with a deadline already attached. Finance bought Tableau licences, the growth team wants Power BI pointed at the clickstream data, and someone in a meeting said the data is &#8220;already in S3, so it should be quick.&#8221;</p>



<p class="wp-block-paragraph">It is quick. You can have a working connection inside an hour. What isn&#8217;t quick is everything that surfaces three weeks later, when forty people are refreshing dashboards on a schedule nobody on your team controls and the Athena line on the invoice has quietly tripled.</p>



<p class="wp-block-paragraph">Getting a BI tool to authenticate against AWS and return rows is a solved problem with decent vendor documentation. What is not well covered is what happens afterwards. This post covers how to connect BI tools to AWS data in a way that survives contact with real users: the four access patterns worth considering, why fine-grained security and cheap dashboards pull in opposite directions, how identity actually flows through a shared service account, and the cost levers in the order they pay off.</p>



<h2 class="wp-block-heading">The failure that shows up on the invoice, not in the logs</h2>



<p class="wp-block-paragraph">Start with the shape of the problem, because it explains most of the design decisions that follow.</p>



<p class="wp-block-paragraph">A dashboard is not one query. Every tile is a query. A twelve-tile dashboard opened by forty analysts, with an hourly scheduled refresh on top, is a query generator running against your data lake. Athena bills on bytes scanned, so the cost of that pattern is set almost entirely by how much data each tile touches, not by how &#8220;small&#8221; the answer looks in the browser.</p>



<p class="wp-block-paragraph">Athena has a feature built for exactly this. Query result reuse lets a repeated query return the previous stored result instead of scanning again, within a maximum age you choose. If you don&#8217;t specify an age, Athena treats results older than 60 minutes as expired, and the longest age you can specify is the equivalent of seven days.</p>



<p class="wp-block-paragraph">Here is the part that catches people, and it is the whole reason I wrote this post.</p>



<p class="wp-block-paragraph">Result reuse is not available for tables that have Lake Formation row or column filters. It is also unavailable when the table&#8217;s S3 location is registered as a Lake Formation data location, for queries that reference more than one data catalog, for queries referencing more than twenty tables, and for federated catalogs or an external Hive metastore.</p>



<p class="wp-block-paragraph">So the sequence goes like this. Dashboards are connected and cheap. A security review asks for row-level security on the customer dimension. You add Lake Formation data filters, which is exactly the right call. Everyone signs off. And the reuse rate on those tables silently drops to zero. No error. No alert. No code change. Just a larger bill the following month and a very awkward conversation about what changed.</p>



<p class="wp-block-paragraph">You can still have both governance and cheap dashboards. You just can&#8217;t have both on the same table. The usual escape is to materialise pre-filtered, per-audience tables that carry no filters of their own, and grant access to those instead. That costs you a pipeline and some storage. It buys back the caching.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Pick the door before you pick the driver</h2>



<p class="wp-block-paragraph">Most guides start with driver installation. That&#8217;s backwards. The driver is a twenty-minute job once you&#8217;ve decided which door the BI tool walks through. There are four realistic doors, and they fail in different ways.</p>



<h3 class="wp-block-heading">Athena over the AWS Glue Data Catalog</h3>



<p class="wp-block-paragraph">The default, and usually the right first answer. Files stay where they are, the Glue Data Catalog is the single source of truth for schema, and anything that speaks JDBC or ODBC can connect. Tableau, Power BI, Looker, Metabase, Apache Superset and Grafana all handle this without exotic configuration.</p>



<p class="wp-block-paragraph">Where it wins: no cluster to size, no idle spend, and you can expose a new dataset by cataloguing it rather than by loading it anywhere.</p>



<p class="wp-block-paragraph">Where it doesn&#8217;t: per-query billing punishes chatty dashboards, and interactive latency is measured in seconds rather than milliseconds. If your users expect a filter dropdown to respond instantly, they will be disappointed and they will tell you so.</p>



<p class="wp-block-paragraph">One practical note that saves an afternoon: for Tableau Desktop, the Athena JDBC driver has to be dropped in a specific directory before the connector appears. On macOS that&#8217;s <code>~/Library/Tableau/Drivers</code>, and on Windows it&#8217;s <code>C:Program FilesTableauDrivers</code>. If the analyst says &#8220;I don&#8217;t see Amazon Athena in the list,&#8221; that&#8217;s almost always why.</p>



<h3 class="wp-block-heading">A Redshift datashare into a consumer warehouse</h3>



<p class="wp-block-paragraph">If the data already lives in Redshift, or if concurrency is the binding constraint, a datashare is the cleaner answer. The producer keeps ownership; the consumer gets live, transactionally consistent access and pays with its own compute, so a badly written dashboard can&#8217;t degrade your ETL warehouse.</p>



<pre class="wp-block-code"><code>-- On the producer
CREATE DATASHARE bi_share;
ALTER DATASHARE bi_share ADD SCHEMA reporting;
ALTER DATASHARE bi_share ADD TABLE reporting.fact_orders;
GRANT USAGE ON DATASHARE bi_share TO ACCOUNT '111122223333';</code></pre>



<p class="wp-block-paragraph">The schema has to be added before the tables inside it, which is a common first stumble. Cross-account sharing is a two-way handshake: the producer authorises the datashare, and an administrator in the consumer account associates it before anything is queryable. Both sides must be encrypted for cross-account sharing. Objects are read-only unless the producer explicitly grants write privileges and the consumer associates for writes, which is the behaviour you want for a BI audience.</p>



<p class="wp-block-paragraph">Two things to price in. Cross-region sharing moves data and is billed accordingly. And the consumer warehouse doesn&#8217;t hold statistics on the producer&#8217;s data, so plan expectations around query planning rather than assuming parity with local tables.</p>



<h3 class="wp-block-heading">The Iceberg REST catalog</h3>



<p class="wp-block-paragraph">If the consumer is another query engine rather than a dashboard tool, this is the interesting door. Amazon S3 Tables exposes an Iceberg REST endpoint at <code>https://s3tables.&lt;region&gt;.amazonaws.com/iceberg</code>, with your table bucket ARN as the warehouse location and SigV4 for authentication. Engines like Snowflake, Trino, StarRocks, Spark and PyIceberg can register that as an external catalog and read the tables in place.</p>



<p class="wp-block-paragraph">Worth knowing: AWS positions the S3 Tables endpoint for basic read and write access against a single table bucket, and recommends the AWS Glue Iceberg REST endpoint when you need unified table management, centralised governance and fine-grained access control. If Lake Formation is already your control plane, go through Glue.</p>



<p class="wp-block-paragraph">Where it doesn&#8217;t help: most BI tools don&#8217;t speak Iceberg REST. They speak SQL over JDBC. This is an engine-to-engine door, not a dashboard door, and treating it as the latter wastes a week.</p>



<h3 class="wp-block-heading">A materialised extract</h3>



<p class="wp-block-paragraph">The unglamorous option that is frequently correct. Build a narrow, pre-aggregated table on a schedule and land it where the BI tool already keeps data: QuickSight SPICE, a Power BI import model, a Tableau extract, or a small Postgres instance. Tools like dbt handle the modelling side; Fivetran or Airbyte handle it if the source is a SaaS system rather than your own lake.</p>



<p class="wp-block-paragraph">Where it wins: it is by a wide margin the cheapest and fastest option for stable reporting, because the expensive scan happens once per day instead of once per tile per user. If you self-host Metabase or Superset, a modest VPS from a provider like InterServer or Hetzner will comfortably serve a department off a nightly rollup.</p>



<p class="wp-block-paragraph">Where it doesn&#8217;t: freshness. And you now own a pipeline that can fail silently, which is its own category of on-call pain.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Identity: the shared service role problem</h2>



<p class="wp-block-paragraph">Nearly every BI deployment starts with one IAM role for the whole tool. It works immediately, which is why it survives longer than it should.</p>



<p class="wp-block-paragraph">The consequences accumulate quietly. CloudTrail records every query as the same principal, so audit answers stop being useful. Lake Formation row filters key off the principal, so if all traffic arrives as one role, your row-level security collapses to whatever that role can see. And cost attribution disappears, so when the bill jumps you cannot say which team caused it.</p>



<p class="wp-block-paragraph">Three ways out, in increasing order of effort:</p>



<ol class="wp-block-list">
<li><strong>One role and one Athena workgroup per consumer group.</strong> Coarse, but honest. Finance gets a role, marketing gets a role, each with its own workgroup. Scan volume becomes attributable and you can set different limits per audience.</li>

<li><strong>Real identity propagation.</strong> Federate the BI tool&#8217;s users through IAM Identity Center or SAML so Lake Formation grants apply to the actual person. This is the correct boundary and the most work, and driver and tool support varies. Test it with the exact tool version your users have before you promise it in a design review.</li>

<li><strong>Push the filter into the data.</strong> Per-tenant tables, views or partitions with separate grants. Unfashionable and more to maintain, but it works with every tool, survives driver upgrades, and keeps result reuse eligible because the tables themselves carry no filters.</li>
</ol>



<p class="wp-block-paragraph">Option three is the one I reach for first when the audience is a handful of stable groups, and option two when the audience is the whole company.</p>



<h2 class="wp-block-heading">The network path is a design decision, not a checkbox</h2>



<p class="wp-block-paragraph">For a SaaS BI tool, the public service endpoint with SigV4-signed requests over TLS is perfectly reasonable, and it is what most vendors document. Don&#8217;t let anyone talk you into complexity you don&#8217;t need.</p>



<p class="wp-block-paragraph">You need more when the tool runs inside a VPC, or on-premises via Direct Connect or VPN, and traffic must not traverse the internet. Athena supports an interface VPC endpoint, and the Athena documentation is explicit that you should pair it with an AWS Glue endpoint, because the catalog lookup is a separate API call that will otherwise still go out to the public endpoint.</p>



<pre class="wp-block-code"><code>aws ec2 create-vpc-endpoint 
  --vpc-id &lt;your-vpc-id&gt; 
  --service-name com.amazonaws.&lt;region&gt;.athena 
  --vpc-endpoint-type Interface 
  --subnet-ids &lt;subnet-id&gt; 
  --security-group-ids &lt;security-group-id&gt; 
  --private-dns-enabled</code></pre>



<p class="wp-block-paragraph">With private DNS enabled, the standard <code>athena.&lt;region&gt;.amazonaws.com</code> hostname resolves to the endpoint, so no client configuration changes. Without it, VPC gives you a hostname in the form <code>&lt;endpoint-id&gt;.athena.&lt;region&gt;.vpce.amazonaws.com</code>, which you have to configure in the tool by hand. Attach an endpoint policy while you&#8217;re there; that&#8217;s where you restrict which principals and resources can be reached.</p>



<p class="wp-block-paragraph">For Redshift, the equivalent is a Redshift-managed VPC endpoint, which requires either RA3 node types or Redshift Serverless. On provisioned clusters, cluster relocation has to be enabled first.</p>



<p class="wp-block-paragraph">One cost item people forget: if the BI tool lives outside AWS, every row it pulls back is data transfer out. A live dashboard returning aggregates is cheap. A nightly extract refresh dragging millions of rows across the boundary is not. Add an S3 gateway endpoint for the query results bucket so the driver&#8217;s result download stays on the AWS network where it can.</p>



<h2 class="wp-block-heading">Cost levers, in the order they pay off</h2>



<p class="wp-block-paragraph">Ordering matters here. Most teams reach for the guardrails first, which bounds the worst case without improving the base case.</p>



<ol class="wp-block-list">
<li><strong>Fix the storage layout.</strong> Columnar formats and sensible partitioning reduce bytes scanned on every query forever. Nothing else you do competes with this.</li>

<li><strong>Pre-aggregate for the dashboard.</strong> A tile that reads a purpose-built rollup is a different animal from one that scans a fact table and aggregates on the fly.</li>

<li><strong>Enable result reuse where governance allows it.</strong> Free wins on repeated queries, provided the tables aren&#8217;t disqualified by the limitations above.</li>

<li><strong>Give every consumer its own workgroup with a per-query cutoff.</strong> This is the guardrail, and it belongs after the first three, not before them.</li>

<li><strong>Move the highest-traffic dashboards to extract mode.</strong> Once a dashboard&#8217;s usage is predictable, importing beats querying live.</li>
</ol>



<p class="wp-block-paragraph">The workgroup guardrail in Terraform. The per-query cutoff cancels any single query that scans more than the limit, which is what stops an accidental <code>SELECT *</code> against a multi-terabyte table:</p>



<pre class="wp-block-code"><code>resource "aws_athena_workgroup" "bi_finance" {
  name  = "bi-finance"
  state = "ENABLED"

  configuration {
    enforce_workgroup_configuration    = true
    publish_cloudwatch_metrics_enabled = true
    bytes_scanned_cutoff_per_query     = 10737418240  # 10 GiB

    result_configuration {
      output_location = "s3://my-athena-results/bi-finance/"
    }
  }
}</code></pre>



<p class="wp-block-paragraph">Two flags there do real work. <code>enforce_workgroup_configuration</code> stops clients overriding the results location and encryption settings, which is what keeps a BI tool from writing results somewhere unmanaged. <code>publish_cloudwatch_metrics_enabled</code> is what makes the whole thing observable, and without it you&#8217;re back to reading the invoice to find out what happened.</p>



<p class="wp-block-paragraph">Workgroup-wide limits are configured separately from the workgroup resource. Those let you set hourly or daily thresholds on aggregate data scanned and fire an SNS notification when a threshold is crossed, which can page someone or disable the workgroup outright.</p>



<p class="wp-block-paragraph">Result reuse can be set per query through the API, which is the mechanism the drivers expose as a connection option:</p>



<pre class="wp-block-code"><code>aws athena start-query-execution 
  --work-group "bi-finance" 
  --query-string "SELECT region, SUM(net) FROM reporting.orders GROUP BY 1" 
  --result-reuse-configuration 
    "ResultReuseByAgeConfiguration={Enabled=true,MaxAgeInMinutes=60}"</code></pre>



<p class="wp-block-paragraph">Set the max age to match how often the underlying data actually changes. If the table is rebuilt once a night, an hour is far too conservative.</p>



<p class="wp-block-paragraph">Finally, alarm on the scan metrics. Whether that lands in CloudWatch dashboards, Grafana or Datadog matters less than the alarm existing at all, owned by the team whose workgroup it is.</p>



<h2 class="wp-block-heading">Troubleshooting the connections that half work</h2>



<ul class="wp-block-list">
<li><strong>The tool connects but lists no tables.</strong> Almost always Lake Formation rather than IAM. Check that the role has been granted on the database and tables, and note that the <code>DESCRIBE</code> permission on the default database is required for Athena.</li>

<li><strong>&#8220;Bytes scanned limit exceeded.&#8221;</strong> That&#8217;s your per-query cutoff doing its job. Look at what the tile is actually querying before raising the limit. Usually the partition filter is missing.</li>

<li><strong>Query works in the console, fails from the tool.</strong> Three usual suspects: a different workgroup, a results location the role can&#8217;t write to, or a role that has catalog access but not S3 access to the underlying data.</li>

<li><strong>A view works for you but not for the analyst.</strong> Lake Formation requires permissions on the tables, columns and S3 locations the view is built on. Column-level permissions are not available on views, so a view is not a substitute for column filtering.</li>

<li><strong>Costs jumped with no deployment.</strong> Check whether anyone added Lake Formation filters, and check whether partition pruning quietly stopped working after a schema or layout change.</li>

<li><strong>The first query of the morning is slow, the rest are fine.</strong> Partition metadata listing. Partition projection removes most of that cost on large tables.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Granting the BI role broad S3 read access and calling it least privilege. It sees every bucket prefix it can list, catalog or not.</li>

<li>Running every consumer through the default primary workgroup, which makes attribution and per-team limits impossible.</li>

<li>Exposing raw fact tables to business users and expecting them to write efficient SQL. They will drag columns into a canvas, which is what the tool is for.</li>

<li>Setting dashboard refresh schedules without asking how fresh the underlying data actually is. Refreshing hourly against a nightly-loaded table burns money for identical results.</li>

<li>Promising row-level security through a shared service role. It will not behave the way the security review thinks it does.</li>

<li>Skipping the Glue VPC endpoint after building the Athena one, then spending a day wondering why traffic still leaves the VPC.</li>
</ul>



<h2 class="wp-block-heading">Best practices worth the effort</h2>



<ul class="wp-block-list">
<li>Create the workgroup, the role and the tag before you hand over the connection string, not after the first invoice.</li>

<li>Publish a curated reporting layer. The BI tool should see purpose-built tables with business column names, not your raw landing zone.</li>

<li>Decide consciously whether a dataset is governed by Lake Formation filters or optimised for caching, and document which, because the two do not coexist on one table.</li>

<li>Apply cost allocation tags at creation time. Retrofitting them across a live estate is thankless work.</li>

<li>Version the connection setup as code. Terraform for the workgroup, roles and endpoints; a short runbook for the driver install on the client side.</li>

<li>Re-test the whole path after any driver upgrade. Authentication behaviour is the thing that changes between driver major versions.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">What is the fastest way to connect Power BI or Tableau to AWS data?</h3>



<p class="wp-block-paragraph">Athena over the Glue Data Catalog, using the Athena JDBC or ODBC driver. Catalogue the data, create a workgroup with a results bucket, create a role with catalog and S3 read access, install the driver on the client, connect. The connection itself is the easy part; the workgroup and role are what you&#8217;ll wish you had done deliberately.</p>



<h3 class="wp-block-heading">Should I use Athena or Redshift for BI dashboards?</h3>



<p class="wp-block-paragraph">Athena if query volume is moderate, the data already lives in S3, and users can tolerate a few seconds per query. Redshift if you need high concurrency, consistently low latency, or you already run a warehouse and can share into a separate consumer endpoint. The honest test is whether your peak is a spike or a plateau: Athena handles spikes without idle cost, Redshift handles plateaus without per-query cost.</p>



<h3 class="wp-block-heading">Does Lake Formation row-level security work with any BI tool?</h3>



<p class="wp-block-paragraph">The filtering itself is enforced by the query engine, so it works regardless of the tool. What varies is whether the tool can present each end user as a distinct principal. Through a single shared service role, every user gets the same filtered view, which is not what row-level security is for. Verify the identity path with your actual tool before designing around it.</p>



<h3 class="wp-block-heading">Why did my Athena bill increase after enabling fine-grained access control?</h3>



<p class="wp-block-paragraph">Because query result reuse is not supported for tables with Lake Formation row or column filters, or where the S3 location is registered as a Lake Formation data location. Queries that were being served from cached results started scanning again. Nothing errors, which is why it takes a billing cycle to notice.</p>



<h3 class="wp-block-heading">Do I need PrivateLink to connect a BI tool to AWS?</h3>



<p class="wp-block-paragraph">No, unless your requirements say traffic must not traverse the public internet, or the tool runs inside a VPC or on-premises network. Public endpoints with signed requests over TLS are the normal path for SaaS BI tools. Add interface endpoints when there&#8217;s a stated requirement, and remember to add the Glue endpoint alongside the Athena one.</p>



<h3 class="wp-block-heading">Can BI tools read Amazon S3 Tables directly?</h3>



<p class="wp-block-paragraph">Indirectly. S3 Tables integrates with the Glue Data Catalog, so Athena, EMR and Redshift can query them, and any BI tool that connects through Athena inherits that. The Iceberg REST endpoint is aimed at query engines and Iceberg clients rather than dashboard tools, which generally expect a SQL connection instead.</p>



<h3 class="wp-block-heading">Is QuickSight a better fit than an external BI tool?</h3>



<p class="wp-block-paragraph">It removes the network, driver and identity plumbing described above, and SPICE gives you an in-memory cache without building an extract pipeline. Against that, your organisation may already be committed to Tableau, Power BI or Looker for reasons that have nothing to do with infrastructure, and retraining an analytics team is a genuine cost. Evaluate it on that basis rather than on the connection mechanics alone.</p>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">When you connect BI tools to AWS data, the connection is not the project. The project is deciding, deliberately, which datasets are governed by fine-grained filters and which are optimised for repeated cheap reads, then giving every consumer its own role, workgroup and cost tag before anyone gets a connection string.</p>



<p class="wp-block-paragraph">Do that and a surprise bill becomes a question with an answer. Skip it and you&#8217;ll be reading CloudTrail on a Friday afternoon trying to work out which dashboard scanned four terabytes, with no way to prove it.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help wiring this up properly?</h2>



<p class="wp-block-paragraph">This is a large part of what I do. If you&#8217;re opening up AWS data to an analytics team and want it done so it still makes sense in six months, I can help with:</p>



<ul class="wp-block-list">
<li>Choosing between Athena, a Redshift datashare, an Iceberg REST catalog or a scheduled extract for your specific query pattern and user count</li>

<li>Setting up per-consumer IAM roles, Athena workgroups, scan limits and cost allocation tags as Terraform rather than console clicks</li>

<li>Designing a Lake Formation permission model that gives you row and column security without silently destroying your caching</li>

<li>Building the curated reporting layer and partition strategy that makes dashboards fast and cheap at the source</li>

<li>Private network paths with interface VPC endpoints, Redshift-managed endpoints and endpoint policies for tools that can&#8217;t use public endpoints</li>

<li>Diagnosing an Athena or Redshift bill that grew without an obvious cause, and putting alarms in place so it doesn&#8217;t happen twice</li>
</ul>



<p class="wp-block-paragraph">Send me a workgroup config, a slow dashboard query, or a Cost Explorer screenshot you can&#8217;t explain, and I&#8217;ll tell you what I&#8217;d look at first.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/connect-bi-tools-to-aws-data/">How to Connect BI Tools to AWS Data Without a Runaway Bill</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Agentforce and AWS: Where the Trust Layer Stops and Your Logs Begin</title>
		<link>https://john-nessime.com/blog/devops/agentforce-aws-architecture-security/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 06 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Agentforce]]></category>
		<category><![CDATA[AI Agents]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[API Gateway]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Lambda]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[CloudWatch]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[Einstein Trust Layer]]></category>
		<category><![CDATA[EventBridge]]></category>
		<category><![CDATA[Generative AI]]></category>
		<category><![CDATA[IAM]]></category>
		<category><![CDATA[MCP]]></category>
		<category><![CDATA[Observability]]></category>
		<category><![CDATA[PrivateLink]]></category>
		<category><![CDATA[Salesforce]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=155</guid>

					<description><![CDATA[<p>Agentforce and AWS wire together in four standard patterns, and every one of them has a point where Salesforce's guarantees stop and yours start. This traces a single request across each boundary it crosses, covers the Trust Layer default most write-ups get wrong (LLM data masking is disabled for agents), and sets out what changes the moment a callout lands in your own account: retention, audit trail, and user identity that does not travel.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/agentforce-aws-architecture-security/">Agentforce and AWS: Where the Trust Layer Stops and Your Logs Begin</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The architecture review goes fine right up until someone from security asks the only question that matters: when the agent answers a customer&#8217;s question, where exactly does that customer&#8217;s data go, and who keeps a copy?</p>



<p class="wp-block-paragraph">&#8220;Salesforce handles it&#8221; is not an answer. Neither is &#8220;it&#8217;s in our VPC,&#8221; because it usually isn&#8217;t, and the parts that are in your VPC are the parts nobody has threat-modelled.</p>



<p class="wp-block-paragraph">This post traces a single Agentforce request end to end and stops at each boundary it crosses. Agentforce and AWS get wired together in three or four standard patterns, and every one of them has a specific point where Salesforce&#8217;s guarantees stop and yours start. I will cover where that line sits, what changes about the security model when you cross it, and one default that surprises almost everyone.</p>



<h2 class="wp-block-heading">The boundaries a single request crosses</h2>



<p class="wp-block-paragraph">One user turn touches more systems than the diagrams suggest. Roughly:</p>



<ol class="wp-block-list">
<li><strong>Retrieval.</strong> The agent grounds itself in CRM records, Data Cloud objects and knowledge content.</li>

<li><strong>Reasoning.</strong> The request gets classified to a topic, then an action gets chosen. Both steps involve a model call.</li>

<li><strong>Generation.</strong> A prompt is assembled and sent through the LLM gateway to a model provider.</li>

<li><strong>Action.</strong> If the agent decides to do something, it calls out. That callout is where AWS usually enters the picture.</li>
</ol>



<p class="wp-block-paragraph">Steps one to three happen inside Salesforce&#8217;s trust boundary, and the Einstein Trust Layer applies. Step four does not, and it doesn&#8217;t. That sentence is the whole post, but the detail matters.</p>



<h2 class="wp-block-heading">Retrieval: the part that&#8217;s better than you&#8217;d expect</h2>



<p class="wp-block-paragraph">Grounding is where Agentforce gets its context, and the architecturally interesting decision is that vector search lives inside Data Cloud rather than in a separate vector database. Structured records and unstructured content are retrieved from the same layer, which removes a whole class of sync problem you would otherwise be building.</p>



<p class="wp-block-paragraph">The security-relevant property is that retrieval is permission-aware. The agent retrieves as the running user, so sharing rules and field-level security apply. If a rep cannot see an opportunity, the agent acting on their behalf should not be able to summarise it either.</p>



<p class="wp-block-paragraph">Two caveats worth putting in your notes. First, this only holds while retrieval goes through Salesforce; the moment you ground from an external source, you are enforcing access control yourself. Second, if your CRM sharing model is loose, the agent inherits that looseness and makes it much easier to exploit, because a model will cheerfully summarise a thousand records a human would never have opened one by one. Agentforce does not create the over-permissioning problem, it just removes the friction that was hiding it.</p>



<p class="wp-block-paragraph">Where AWS shows up here: if your analytical data lives in Redshift, an Iceberg lake or Databricks, zero-copy federation lets Data Cloud register those tables and query them in place rather than replicating them. Good for cost and duplication, and it moves an access-control decision into the federation configuration, which is a place people forget to audit.</p>



<h2 class="wp-block-heading">The Trust Layer, and the default nobody expects</h2>



<p class="wp-block-paragraph">The Einstein Trust Layer sits between every Salesforce-originated prompt and the model. Its components are well documented: secure grounding, data masking, system policies against prompt injection, the LLM gateway, zero data retention agreements with model providers, toxicity checks on the way back, and an audit trail of the whole journey.</p>



<p class="wp-block-paragraph">Here is the part that gets misreported constantly, including in a lot of otherwise decent write-ups: <strong>LLM data masking is disabled for agents.</strong> Salesforce says so plainly in its own documentation on the Trust Layer architecture and repeats it in the Trailhead module on masking. Masking remains available and configurable for embedded generative features like service replies and work summaries. For Agentforce agents, it is off.</p>



<p class="wp-block-paragraph">The reason is not hard to work out once you think about what an agent does. Masking swaps real values for placeholders. An agent that has to actually perform an action needs the real record ID, the real amount, the real email address, because those become tool arguments. You cannot pass <code>&lt;Person_0&gt;</code> to a Flow and expect it to update a contact. Masking and acting are in genuine tension, and Salesforce resolved it in favour of acting.</p>



<p class="wp-block-paragraph">What this changes in practice:</p>



<ul class="wp-block-list">
<li>Real PII reaches the model provider. The zero data retention commitment is what protects it, not de-identification. Those are different controls with different failure modes: masking is technical and verifiable, a retention agreement is contractual.</li>

<li>If your compliance position was &#8220;no personal data leaves our boundary in identifiable form,&#8221; Agentforce does not satisfy it the way you may have assumed, and this is worth raising before your security review rather than during it.</li>

<li>Controlling exposure moves upstream. What the model sees is now decided by what you ground on and what your sharing model permits, not by a masking filter downstream.</li>
</ul>



<p class="wp-block-paragraph">None of this makes the Trust Layer weak. Zero retention, the gateway, system policies and the audit trail all still apply to agent traffic, and the Trust Layer typically runs several times in a single user turn, once for topic classification and again for generation. It is a real control plane. Just know which control is doing the work.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Connecting Agentforce and AWS: four patterns</h2>



<p class="wp-block-paragraph">There are four ways these two systems talk, and they have different latency, failure and security characteristics.</p>



<h3 class="wp-block-heading">Synchronous outbound: agent action calls AWS</h3>



<p class="wp-block-paragraph">A custom agent action, implemented via External Services with an OpenAPI schema, an Apex invocable method, or a Flow, calls an endpoint you own. Behind it sits API Gateway and Lambda, and often a Bedrock agent or knowledge base.</p>



<p class="wp-block-paragraph">Authentication uses named credentials, which keep the secret out of your Apex and centralise the auth config. Use them; hardcoding a key into an Apex class is the version of this that ends up in a git history.</p>



<p class="wp-block-paragraph">The trade-off is that you have put a network call and a second model invocation inside a conversational turn. Users notice. Budget for it, and set the timeout deliberately rather than inheriting a default.</p>



<h3 class="wp-block-heading">Asynchronous outbound: Event Relay to EventBridge</h3>



<p class="wp-block-paragraph">Salesforce Event Relay delivers platform events from the Salesforce event bus to Amazon EventBridge without code, and EventBridge routes onward to Lambda, S3, Step Functions or a Bedrock agent.</p>



<p class="wp-block-paragraph">This is the right pattern for anything that does not need to complete inside the conversation: enrichment, downstream processing, long-running work. It also fails better, because a retry is a queue concern rather than a user watching a spinner.</p>



<h3 class="wp-block-heading">Inbound: AWS calls Agentforce</h3>



<p class="wp-block-paragraph">The Agent API lets something on the AWS side invoke an Agentforce agent, typically through EventBridge API destinations. That gives you bidirectional multi-agent flows: a Bedrock agent that detects a problem can open a case through Agentforce rather than writing to the Salesforce API directly and bypassing the agent&#8217;s own logic.</p>



<p class="wp-block-paragraph">Worth being deliberate here. An inbound path means an AWS workload can trigger agent reasoning, which means anything that can inject content into that workload can influence a prompt. Treat the payload as untrusted input, because it is.</p>



<h3 class="wp-block-heading">MCP: tools rather than integrations</h3>



<p class="wp-block-paragraph">Agentforce has a native MCP client, so an agent can connect to any MCP-compliant server without a bespoke integration, alongside a server registry intended to let admins govern which agents connect to which tools under central policy.</p>



<p class="wp-block-paragraph">This is the pattern that will cause the most trouble over the next couple of years, and not because the protocol is bad. It is because MCP makes adding a tool trivial, and every tool is a new path by which text from somewhere else reaches your agent&#8217;s context and by which your agent&#8217;s decisions reach someone else&#8217;s system. The registry exists precisely so that &#8220;which tools can this agent reach&#8221; is an administered answer rather than an emergent one. Use it as a control, not a catalogue.</p>



<h3 class="wp-block-heading">Keeping the traffic private</h3>



<p class="wp-block-paragraph">By default these callouts traverse the public internet with TLS. Salesforce Private Connect uses AWS PrivateLink to give you a private path to your own AWS endpoints instead. Whether that is required is a policy question rather than a technical one, but it is much easier to set up at design time than to retrofit after an auditor asks.</p>



<h2 class="wp-block-heading">What changes the moment you cross into AWS</h2>



<p class="wp-block-paragraph">This is the seam, and it is where most real incidents will come from. Four things stop being true simultaneously.</p>



<ul class="wp-block-list">
<li><strong>Zero retention becomes your policy, not Salesforce&#8217;s.</strong> If your Lambda logs its input at INFO, you now have customer PII in CloudWatch Logs with whatever retention that log group happens to have, which by default is forever. Your own debug logging is the single most likely leak path in this entire architecture, and it will not show up in any Salesforce audit report.</li>

<li><strong>The audit trail splits.</strong> Salesforce logs the prompt journey. AWS logs the invocation. Nothing correlates them unless you pass a request identifier through the callout and log it on both sides. Do that from day one; reconstructing an incident across two disconnected audit systems is genuinely unpleasant.</li>

<li><strong>User identity does not travel.</strong> This is the important one. Inside Salesforce the agent acts as the running user and sharing rules apply. Your Lambda runs under an IAM execution role with fixed permissions and no idea who asked. If that role can read a whole DynamoDB table, then every user of the agent can effectively read the whole table through it, regardless of their CRM permissions. That is a classic confused deputy, and it is easy to build without noticing. If the action needs to be scoped per user, pass the identity explicitly and enforce it in your own code.</li>

<li><strong>Model governance forks.</strong> A Bedrock model you invoke from your own Lambda is outside the Trust Layer. Its guardrails, logging and retention are configured in your account and are yours to get right. If you want prompt filtering or PII detection on that leg, you are building or buying it.</li>
</ul>



<p class="wp-block-paragraph">Note the difference between two things people conflate. Salesforce running on Hyperforce, which is built on public cloud infrastructure including AWS, means Salesforce&#8217;s own workloads run there under Salesforce&#8217;s controls. It does not mean your Salesforce data sits in your AWS account. When an agent action calls your Lambda, that is the first moment data lands in infrastructure you control and are responsible for.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Assuming masking protects agent traffic to the model. It does not; check the current documentation yourself rather than taking a blog&#8217;s word for it, including this one.</li>

<li>Logging full request payloads in Lambda during development and never turning it off.</li>

<li>Giving the Lambda execution role broad data access because &#8220;the agent is trusted,&#8221; when the agent is a proxy for every user who can talk to it.</li>

<li>Putting a slow AWS call inside a synchronous agent action when the work did not need to be synchronous.</li>

<li>Not passing a correlation identifier across the boundary, so Salesforce and CloudWatch tell two unlinked stories.</li>

<li>Treating content returned from an external tool or MCP server as trusted context rather than as untrusted input that will be read by a model.</li>

<li>Deploying agents against a permissive sharing model and discovering the blast radius later.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Does Agentforce data get used to train models?</h3>



<p class="wp-block-paragraph">Not on the Salesforce leg. Salesforce&#8217;s zero data retention arrangements with model providers mean prompts and responses are not stored or used for training. That commitment covers traffic going through the LLM gateway. It says nothing about a model you invoke yourself from your own AWS account, where the retention and logging configuration is entirely yours.</p>



<h3 class="wp-block-heading">Can I use my own Bedrock model inside Agentforce?</h3>



<p class="wp-block-paragraph">There are two distinct things here. Bedrock-hosted models can be brought into Salesforce through the Einstein and Bedrock integration, in which case they sit behind the Trust Layer. Separately, you can call a Bedrock model or agent from a custom agent action, in which case you are outside the Trust Layer and own the controls. Be clear which one you are building, because they have completely different compliance stories.</p>



<h3 class="wp-block-heading">Sync or async for AWS integrations?</h3>



<p class="wp-block-paragraph">Default to async through Event Relay and EventBridge unless the agent genuinely needs the result to continue the conversation. Synchronous callouts put a network round trip and often a second inference call inside a user&#8217;s turn, and they fail in front of the user rather than into a retry queue.</p>



<h3 class="wp-block-heading">How do I stop the agent seeing data a user shouldn&#8217;t?</h3>



<p class="wp-block-paragraph">Inside Salesforce, fix the sharing model, because retrieval respects it and inherits its flaws. Outside Salesforce, sharing rules do not apply at all: pass the user identity through the callout and enforce authorisation in your own code, or scope the integration to data that is safe for every user of that agent.</p>



<h3 class="wp-block-heading">Do I need PrivateLink for this?</h3>



<p class="wp-block-paragraph">Technically no; callouts work over TLS on the public internet. Practically it depends on your regulatory position and whether your AWS endpoints should be publicly reachable at all. If they should not be, Private Connect over PrivateLink is the supported path and is far cheaper to design in than to add later.</p>



<h3 class="wp-block-heading">What should I log, and where?</h3>



<p class="wp-block-paragraph">On the AWS side, log the correlation identifier, the action name, the outcome and the timing. Do not log the payload. If you need payload-level debugging, gate it behind a flag that is off in production and set an explicit short retention on the log group. Ship the operational metrics somewhere you already watch, whether that is CloudWatch alarms, Grafana or Datadog, because an agent action that silently starts failing looks like nothing at all from the Salesforce side.</p>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">Wiring up Agentforce and AWS is not hard. The patterns are documented, the connectors exist, and a working proof of concept is a day&#8217;s work.</p>



<p class="wp-block-paragraph">What is hard is knowing precisely where Salesforce&#8217;s guarantees end. They end at the callout. Everything before it is covered by the Trust Layer, with the significant exception that masking is off for agents, so what reaches the model is real data protected by a retention agreement rather than by de-identification. Everything after it is yours: your retention, your audit trail, your authorisation, your logs.</p>



<p class="wp-block-paragraph">Draw that line on your architecture diagram before the security review, not during it.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help designing or reviewing this?</h2>



<p class="wp-block-paragraph">I work on the AWS side of Salesforce integrations, which is usually the side nobody owns. Things I can help with:</p>



<ul class="wp-block-list">
<li>Threat-modelling the seam between Agentforce and your AWS account, and writing down what each side actually guarantees.</li>

<li>Building agent actions on API Gateway and Lambda with named credentials, sane timeouts and no payload logging.</li>

<li>Event-driven integration via Event Relay and EventBridge, including retry, dead-letter and replay design.</li>

<li>Scoping IAM execution roles so an agent action cannot become a confused deputy for every user who can reach it.</li>

<li>Correlated observability across Salesforce and CloudWatch so an incident can be reconstructed in one timeline.</li>

<li>Private Connect and PrivateLink setup where callouts should not touch the public internet.</li>
</ul>



<p class="wp-block-paragraph">If you have an integration already built, send me the agent action definition and the Lambda&#8217;s IAM policy. Those two together usually tell the whole story.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/agentforce-aws-architecture-security/">Agentforce and AWS: Where the Trust Layer Stops and Your Logs Begin</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
