<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>IAM Access Analyzer | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/iam-access-analyzer/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/iam-access-analyzer/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Sat, 10 Oct 2026 06:00:52 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>IAM Access Analyzer | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/iam-access-analyzer/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>IAM Access Analyzer Unused Access: Removing Permissions Without Breaking Production</title>
		<link>https://john-nessime.com/blog/technical-guides/iam-access-analyzer-unused-access/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sat, 10 Oct 2026 06:00:52 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Access Advisor]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[Blast Radius]]></category>
		<category><![CDATA[CloudTrail]]></category>
		<category><![CDATA[Data Events]]></category>
		<category><![CDATA[Decommissioning]]></category>
		<category><![CDATA[Explicit Deny]]></category>
		<category><![CDATA[Finding Triage]]></category>
		<category><![CDATA[IAM]]></category>
		<category><![CDATA[IAM Access Analyzer]]></category>
		<category><![CDATA[Least Privilege]]></category>
		<category><![CDATA[Policy Generation]]></category>
		<category><![CDATA[Rollback Plan]]></category>
		<category><![CDATA[Trust Policies]]></category>
		<category><![CDATA[Unused Access Findings]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=782</guid>

					<description><![CDATA[<p>Unused access findings tell you what AWS did not record activity for, which is not the same as what your workload does not need. Here are the four blind spots in the underlying data, and a deny-before-delete procedure that lets you trim IAM permissions with a one-call rollback.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/iam-access-analyzer-unused-access/">IAM Access Analyzer Unused Access: Removing Permissions Without Breaking Production</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The finding said the role hadn&#8217;t touched Amazon S3 in ninety days. The policy granted it full S3 access. Removing that looked like free security work, right up until the nightly export job started failing and nobody could explain why, because the role <em>was</em> using S3, constantly, in a way the analyzer could not see.</p>



<p class="wp-block-paragraph">That gap is the whole problem with permission cleanup. IAM Access Analyzer unused access findings are genuinely useful, and they are also built on evidence with known holes in it. AWS says so in its own documentation. Treat a finding as proof rather than as a lead and sooner or later you take something down.</p>



<p class="wp-block-paragraph">This post covers what an unused access finding actually asserts, the four blind spots that turn a clean-looking finding into an outage, and a removal procedure that fails safe.</p>



<h2 class="wp-block-heading">What an unused access finding actually claims</h2>



<p class="wp-block-paragraph">An unused access analyzer is a separate analyzer type from the external access one. You create it per account or across an organization, and it produces four kinds of finding: unused roles, unused IAM user access keys, unused IAM user passwords, and unused permissions at the service and action level for principals that <em>are</em> active.</p>



<pre class="wp-block-code"><code>aws accessanalyzer create-analyzer 
  --analyzer-name unused-access-prod 
  --type ACCOUNT_UNUSED_ACCESS 
  --configuration '{"unusedAccess": {"unusedAccessAge": 90}}'</code></pre>



<p class="wp-block-paragraph"><code>unusedAccessAge</code> is the threshold in days beyond which access counts as unused, and it accepts 1 to 365. Use <code>ORGANIZATION_UNUSED_ACCESS</code> from a delegated administrator account. Unused access findings don&#8217;t vary by Region, so you don&#8217;t need one analyzer per Region.</p>



<p class="wp-block-paragraph">Here is the part that matters. The analyzer does not replay CloudTrail and reconstruct what your workload did. It reads IAM last accessed information, the same data behind the Access Advisor tab in the console. Everything the analyzer knows, and everything it doesn&#8217;t, comes from there. The blind spots below are last accessed blind spots, inherited wholesale.</p>



<h2 class="wp-block-heading">The four blind spots that break production</h2>



<h3 class="wp-block-heading">1. Action-level tracking doesn&#8217;t cover everything</h3>



<p class="wp-block-paragraph">Action last accessed information only exists for the services and actions AWS has explicitly added to the tracked list, and it covers management actions. The IAM console spells out one case in plain text: action last accessed information is not available for Amazon S3 data events.</p>



<p class="wp-block-paragraph">Think about what that means for a data pipeline. A role whose entire purpose is <code>s3:GetObject</code> and <code>s3:PutObject</code> against one bucket is doing object-level work all day. Bucket-level calls like <code>ListBucket</code> are management events and get tracked. The object-level calls that constitute the actual job are data events, and they sit outside action last accessed.</p>



<p class="wp-block-paragraph">AWS puts the warning in the documentation itself: do not make permissions decisions based solely on the absence of tracking information. Almost nobody reads that sentence.</p>



<h3 class="wp-block-heading">2. Denied attempts count as access</h3>



<p class="wp-block-paragraph">Service last accessed data includes <em>all</em> attempts to call an AWS API, not just successful ones. A role that has been hammering a service it has no permission for, failing every time, shows up as having used that service.</p>



<p class="wp-block-paragraph">This runs opposite to the first blind spot, and it is why a findings list can be simultaneously too aggressive and too timid. Dead permissions look alive because a misconfigured client keeps retrying against them, so you clean up around them and leave the real cruft in place.</p>



<h3 class="wp-block-heading">3. The calendar is longer than your window</h3>



<p class="wp-block-paragraph">A ninety-day tracking period is the common default and it is wrong for a predictable class of role:</p>



<ul class="wp-block-list">
<li>Disaster recovery and failover roles, exercised once or twice a year</li>

<li>Quarterly close, audit, and regulatory reporting roles</li>

<li>Annual key rotation or certificate renewal automation</li>

<li>Break-glass roles, which should be unused by design</li>

<li>Seasonal workloads that only wake up for a peak period</li>
</ul>



<p class="wp-block-paragraph">There is also a ceiling worth knowing. <code>unusedAccessAge</code> tops out at 365 days, while IAM reports last accessed activity for at least the trailing 400 days. The analyzer cannot be configured to span the full window the underlying data actually holds, so for anything on an annual cycle it will not settle the question on its own.</p>



<h3 class="wp-block-heading">4. Service-level use says nothing about resource-level need</h3>



<p class="wp-block-paragraph">A finding operates on services and actions. Your policy probably operates on services, actions, resources, and conditions. Knowing a role used <code>secretsmanager:GetSecretValue</code> last week does not tell you which secret, and narrowing the policy to a specific resource ARN based on a finding is a guess dressed up as evidence.</p>



<p class="wp-block-paragraph">This is where policy generation earns its place. It reads CloudTrail activity for a principal over a window and drafts a policy from what it observed, including resource-level detail where the log has it. Same caveat: policy generation does not identify action-level activity for data events either. Better first draft, not an oracle.</p>



<h2 class="wp-block-heading">A removal procedure that fails safe</h2>



<p class="wp-block-paragraph">The core idea: <strong>deny before you delete</strong>. An explicit deny is one API call to add and one to remove, effective immediately in both directions. Deleting a policy statement or a role is not symmetrical; restoring it means someone reconstructing intent under pressure. Use deny as a reversible dry run for deletion.</p>



<ol class="wp-block-list">
<li><strong>Sort findings by blast radius, not by count.</strong> An unused permission on a role assumed by one Lambda is a different risk from one on a role fifty engineers assume daily.</li>

<li><strong>Confirm the role&#8217;s activity independently</strong> before touching permissions.</li>

<li><strong>Attach a scoped deny as an inline policy on the role,</strong> leaving the existing allow untouched. An explicit deny wins over any allow, so this simulates removal exactly with none of the loss.</li>

<li><strong>Watch CloudTrail for denials attributed to that role.</strong> This is the observation window and it is the step people skip.</li>

<li><strong>Wait through at least one full business cycle,</strong> longer if the workload has monthly or quarterly rhythm.</li>

<li><strong>Only then edit the allow</strong> and remove the deny.</li>
</ol>



<p class="wp-block-paragraph">For step two, the cheapest independent check is the role&#8217;s own last-used timestamp:</p>



<pre class="wp-block-code"><code>aws iam get-role --role-name legacy-reporting-role 
  --query 'Role.RoleLastUsed'</code></pre>



<p class="wp-block-paragraph">That returns <code>LastUsedDate</code> and the <code>Region</code> where the role was last used. <code>LastUsedDate</code> is null if the role has not been used within the tracking period, which is not the same as never used. The Region field is quietly valuable: a supposedly dead role showing activity in a Region nobody works in is worth a conversation before a deletion.</p>



<p class="wp-block-paragraph">For step three, the deny is deliberately narrow. Deny the flagged actions on the flagged service and nothing else:</p>



<pre class="wp-block-code"><code>{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "CanaryDenyUnusedRedshift",
      "Effect": "Deny",
      "Action": "redshift:*",
      "Resource": "*"
    }
  ]
}</code></pre>



<p class="wp-block-paragraph">Give it a Sid you can grep for, because in six months somebody will find this policy and have no idea whether it is a canary or a control.</p>



<p class="wp-block-paragraph">For step four, you want CloudTrail records where the request failed on authorization. The field is <code>errorCode</code>, and the values that matter are <code>AccessDenied</code> plus the EC2-specific <code>Client.UnauthorizedOperation</code>. Filter on the assumed-role session issuer so you catch every session. CloudTrail Lake or Athena over your trail bucket both work.</p>



<p class="wp-block-paragraph">To make this an ongoing signal rather than a one-off query, wire a CloudWatch Logs metric filter on <code>AccessDenied</code> to a dashboard. Amazon Managed Grafana, self-hosted Grafana, or Grafana Cloud all render it fine. What matters is that it is broken down by principal, so a spike points at a role rather than at the account.</p>



<h2 class="wp-block-heading">Deleting the role itself has a separate trap</h2>



<p class="wp-block-paragraph">When you name a principal by ARN in a policy, IAM stores the underlying unique principal ID rather than the string. Delete that role, recreate it with the identical name, and you get a new unique ID. Every policy that referenced the old one now points at something that no longer exists. Nothing errors at deletion time. The break surfaces later, in another account, when a cross-account assume-role call starts failing and the trust policy on screen looks exactly correct.</p>



<p class="wp-block-paragraph">Before deleting a role, check the things a findings list will never tell you:</p>



<ul class="wp-block-list">
<li>Is it named as a principal in any resource policy, S3 bucket policy, KMS key policy, or another account&#8217;s trust policy?</li>

<li>Is it attached to an instance profile?</li>

<li>Is it managed by CloudFormation, Terraform, or OpenTofu? Deleting out of band leaves drift that surfaces on the next apply.</li>

<li>Is it a service-linked role? Those are owned by a service and have their own deletion procedure.</li>

<li>Is it the execution role for a Lambda function, ECS task, or Step Functions state machine that runs rarely?</li>
</ul>



<p class="wp-block-paragraph">Safe order: detach the permissions, leave the role and its trust policy in place, wait another cycle, then delete. An empty role is harmless. A missing one is not.</p>



<h2 class="wp-block-heading">Keeping the findings list small enough to read</h2>



<p class="wp-block-paragraph">A findings list nobody triages is worse than none, because it produces the appearance of coverage. Two levers keep it honest. The first is exclusions, which can skip whole accounts or skip principals by tag, and can be updated at any time:</p>



<pre class="wp-block-code"><code>aws accessanalyzer update-analyzer 
  --analyzer-name unused-access-prod 
  --configuration '{
    "unusedAccess": {
      "analysisRule": {
        "exclusions": [
          {"accountIds": ["111122223333"]},
          {"resourceTags": [{"lifecycle": "break-glass"}]}
        ]
      }
    }
  }'</code></pre>



<p class="wp-block-paragraph">Tag-based exclusion is the one to reach for. Sandbox accounts churn roles and generate noise with no security value; break-glass roles are supposed to look unused. Excluding by tag survives new accounts joining the organization. An account ID list does not.</p>



<p class="wp-block-paragraph">The second lever is archiving findings you have consciously accepted:</p>



<pre class="wp-block-code"><code>aws accessanalyzer update-findings 
  --analyzer-arn &lt;your-analyzer-arn&gt; 
  --ids &lt;finding-id&gt; 
  --status ARCHIVED</code></pre>



<p class="wp-block-paragraph">Archiving is a decision record, not a dismissal. Write down why somewhere durable, because the same finding will come back to a different person.</p>



<p class="wp-block-paragraph">Scope has a cost dimension too. Unused access analysis is billed on the number of IAM roles and users analyzed per month, per analyzer, so a second analyzer with a different tracking period doubles the principals counted. If you watch spend through Cost Explorer or a layer like Vantage or CloudZero, confirm where this line lands before widening scope organization-wide.</p>



<h2 class="wp-block-heading">Troubleshooting</h2>



<p class="wp-block-paragraph"><strong>No findings yet, or a finding that won&#8217;t clear after you fixed it.</strong> After creating or updating an analyzer it takes time for findings to appear, recent activity takes around four hours to reach last accessed data, and scans are periodic rather than instant. Give it a full cycle. If you use the CLI or API for last accessed reports, generate a fresh one; the old report will not update itself.</p>



<p class="wp-block-paragraph"><strong>The finding says unused but the team insists otherwise.</strong> Assume the team is right and the tracking has a hole. Check whether the work is data-plane activity, then whether the actions involved are on the tracked list at all. Absence of tracking data and absence of use look identical from the findings page.</p>



<p class="wp-block-paragraph"><strong>Your canary deny broke something unexpected.</strong> That is the canary working. Detach the inline policy, confirm the failures stop, then narrow the deny to specific actions rather than the whole service and run it again.</p>



<p class="wp-block-paragraph"><strong>Something broke but no AccessDenied appears in CloudTrail.</strong> Check whether the failing call is a data event. A trail logs management events by default; data event logging is opt-in per resource type and carries its own charges. No configured data events, no record of the denial.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating a finding as evidence of non-use rather than absence of recorded use. Different claims.</li>

<li>Deleting the role in the same change as removing its permissions, collapsing two reversible steps into one that isn&#8217;t.</li>

<li>Setting the tracking period once and never revisiting it against the workload&#8217;s actual rhythm.</li>

<li>Cleaning up in the console while Terraform or CloudFormation remains the source of truth, so the next apply puts it all back.</li>

<li>Batching removals across many roles at once. When something breaks you have a bisect problem instead of a rollback.</li>
</ul>



<h2 class="wp-block-heading">Best practices for IAM Access Analyzer unused access cleanup</h2>



<ul class="wp-block-list">
<li>Tag principals with their expected usage cadence at creation time. It costs nothing then and answers the hardest question later.</li>

<li>Start with unused access keys and passwords on IAM users. Highest value, lowest risk removals in the whole list.</li>

<li>Route findings to EventBridge and into your ticket queue so cleanup is tracked work rather than a periodic heroic sweep.</li>

<li>Make the permissions change in code, apply it through the normal pipeline, and let the diff be the record.</li>

<li>Keep an AccessDenied dashboard permanently, not just during a cleanup. Cheapest early warning you can build for this class of change.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Does IAM Access Analyzer read my CloudTrail logs?</h3>



<p class="wp-block-paragraph">Not for unused access findings. Those come from IAM last accessed information. Policy generation is the feature that reads CloudTrail, and it is a separate operation run against a specific principal and time range.</p>



<h3 class="wp-block-heading">What tracking period should I set?</h3>



<p class="wp-block-paragraph">Ninety days is reasonable for interactive human roles and continuously running workloads, and too short for anything on a quarterly or annual cycle. The value is configurable from 1 to 365 days. The right answer is a property of your workload, not a security best practice number.</p>



<h3 class="wp-block-heading">Can a finding be wrong?</h3>



<p class="wp-block-paragraph">It can mislead in both directions. Permissions used only through data events can appear unused, and permissions that only ever produced denied attempts can appear used, because last accessed data counts all attempts rather than only successful ones.</p>



<h3 class="wp-block-heading">Do I need an analyzer in every Region?</h3>



<p class="wp-block-paragraph">No. Unused access findings do not change based on Region, so one per account, or one at the organization level, is enough. External access analyzers are the ones with a Region dimension.</p>



<h3 class="wp-block-heading">How is unused access analysis billed?</h3>



<p class="wp-block-paragraph">On the number of IAM roles and users analyzed per month, per analyzer. Scope is the cost lever, which is why exclusions matter for the bill and not just the noise. Check current published rates before enabling it organization-wide.</p>



<h3 class="wp-block-heading">Should I delete unused roles or just strip their permissions?</h3>



<p class="wp-block-paragraph">Strip first, delete later, in two separate changes. A role with no permissions has almost no attack surface, and keeping it alive preserves any principal references pointing at it while you find out if anything still cares.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Conclusion</h2>



<p class="wp-block-paragraph">IAM Access Analyzer unused access findings answer a narrower question than the one you care about. They tell you what AWS did not record activity for. They do not tell you what your workload does not need, and they cannot tell you whether a permission that <em>is</em> used should be. The distance between those statements is where the outages live.</p>



<p class="wp-block-paragraph">The one thing worth remembering: deny before you delete. An explicit deny gives you the exact behaviour of a removal with a one-call undo, and that single habit turns permission cleanup from a change you have to be brave about into one you can be systematic about.</p>



<h2 class="wp-block-heading">Need help trimming IAM permissions without the outage</h2>



<p class="wp-block-paragraph">The risk in permission cleanup sits almost entirely in what the tooling cannot see. Things I can help with:</p>



<ul class="wp-block-list">
<li>Setting up account or organization-level unused access analyzers with tracking periods and tag-based exclusions that fit how your workloads run</li>

<li>Building the AccessDenied observability layer, from CloudTrail queries through metric filters to a dashboard the team will actually look at</li>

<li>Running staged deny-then-delete cleanups with the rollback path defined before anything changes</li>

<li>Auditing roles for cross-account principal references, instance profiles, and IaC ownership before deletion</li>

<li>Moving console-era IAM into Terraform or OpenTofu so the next cleanup is a reviewable diff</li>
</ul>



<p class="wp-block-paragraph">If you have a findings list you are not sure how to act on, send me the role&#8217;s policy JSON and the finding detail and I will tell you what I would check before touching it.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/iam-access-analyzer-unused-access/">IAM Access Analyzer Unused Access: Removing Permissions Without Breaking Production</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>AI-Generated IAM Policies: Where the Hard Boundary Belongs</title>
		<link>https://john-nessime.com/blog/devops/ai-generated-iam-policies/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sun, 13 Sep 2026 06:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[AI-Assisted Coding]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[Cloudsplaining]]></category>
		<category><![CDATA[CloudTrail]]></category>
		<category><![CDATA[Generative AI]]></category>
		<category><![CDATA[IAM]]></category>
		<category><![CDATA[IAM Access Analyzer]]></category>
		<category><![CDATA[Least Privilege]]></category>
		<category><![CDATA[Parliament]]></category>
		<category><![CDATA[Permissions Boundaries]]></category>
		<category><![CDATA[Policy Generation]]></category>
		<category><![CDATA[Policy Validation]]></category>
		<category><![CDATA[Privilege Escalation]]></category>
		<category><![CDATA[Service Control Policies]]></category>
		<category><![CDATA[Trust Policies]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=858</guid>

					<description><![CDATA[<p>A generated policy with no wildcards can still hand over your account. Here are the three failure shapes that survive human review, why the review step is the wrong place to put your defence, and how to build a ceiling that holds regardless of what the model writes.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/ai-generated-iam-policies/">AI-Generated IAM Policies: Where the Hard Boundary Belongs</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The policy looked fine. No action wildcard, no resource wildcard, a Sid on every statement, formatting tidier than anything I write by hand. It came out of a chat window, went into a pull request, and got two approvals in about ninety seconds.</p>



<p class="wp-block-paragraph">It granted <code>iam:PassRole</code> on one role ARN and <code>lambda:CreateFunction</code> in the same document. That is not a wildcard and it does not read as administrative access. It is administrative access anyway, to whatever that role can do, for anyone who can write a Lambda function.</p>



<p class="wp-block-paragraph">This is the shape of the problem with AI-generated IAM policies. The failure is almost never the obvious one. Models have absorbed enough style guidance to avoid <code>"Action": "*"</code>, so what comes back passes the eyeball test and the linter and still hands over more than you meant to give. The dangerous output is the one that looks careful.</p>



<p class="wp-block-paragraph">This post covers what policy generation genuinely does well, the failure shapes that survive human review, and where the hard boundary belongs: not in the review step, which is fallible, but in a mechanism that cannot be talked out of its answer.</p>



<h2 class="wp-block-heading">What AI-generated IAM policies get right</h2>



<p class="wp-block-paragraph">Start with the honest case, because it is a strong one. IAM has thousands of actions across hundreds of services and the naming is inconsistent in ways nobody holds in their head. Some services use <code>Describe</code>, some use <code>Get</code>, some use both for different things. ARN formats vary per service and per resource type inside a service. Condition keys apply to some actions and not others.</p>



<p class="wp-block-paragraph">A model is good at that recall problem, and at shape work: splitting a monolithic statement into per-resource statements, adding conditions, converting an inline policy into Terraform. Paste an <code>AccessDenied</code> message and you get a plausible read of which layer refused. So the argument is not that the tooling is useless. It is that recall and reasoning about consequences are different capabilities, and IAM is a system where the second one protects you.</p>



<h2 class="wp-block-heading">The failure modes that survive review</h2>



<h3 class="wp-block-heading">Permissions that compose into something bigger</h3>



<p class="wp-block-paragraph">Every action is individually defensible. The escalation lives in the combination.</p>



<p class="wp-block-paragraph">The canonical case is <code>iam:PassRole</code> beside any service that runs code: Lambda, EC2, ECS, Glue. Passing a role to a compute service means your code executes with that role&#8217;s permissions, so your ceiling is not your own policy, it is whatever the most powerful passable role can do. Rhino Security Labs catalogued a long list of these chains and every one of them is built from permissions that look unremarkable alone.</p>



<p class="wp-block-paragraph">Another that catches people: <code>iam:CreatePolicyVersion</code> on a policy you are attached to. A new version normally needs to be set as default to take effect, but the create call accepts a flag that makes it default immediately, and that flag does not require <code>iam:SetDefaultPolicyVersion</code>. A permission that reads as &#8220;can update policy documents&#8221; reads to an attacker as &#8220;can write myself an admin policy.&#8221;</p>



<p class="wp-block-paragraph">Ask a model whether a policy grants admin and it will check the statements and say no, correctly, one at a time. Composition is the part it does not reliably do, and it is the part that matters.</p>



<h3 class="wp-block-heading">Actions that do not exist</h3>



<p class="wp-block-paragraph">Plausible action names that no service publishes. <code>s3:ListBucketContents</code> instead of <code>s3:ListBucket</code>. IAM accepts a policy containing an unrecognized action string because the document is syntactically valid, so nothing fails at attach time.</p>



<p class="wp-block-paragraph">What happens instead is worse. The application throws <code>AccessDenied</code> in production, somebody debugging under pressure decides the granular list must be wrong, and the fix is a service wildcard. The invented action does not grant too much. It causes a human to grant too much a week later, and by then nobody connects the two events.</p>



<h3 class="wp-block-heading">Resource ARNs that look constrained and are not</h3>



<p class="wp-block-paragraph">An ARN with real characters in it feels safer than a star. Often it is not. The common versions are a trailing wildcard placed one segment too high, and an action whose resource element the service ignores. <code>s3:ListAllMyBuckets</code> is account-scoped no matter what you put in <code>Resource</code>. Several <code>ec2:Describe</code> actions behave the same way. A policy can read as resource-scoped throughout and be account-wide for the actions that count.</p>



<p class="wp-block-paragraph">The related trap is the missing condition: cross-account trust policies without an external ID, bucket policies without <code>aws:PrincipalOrgID</code>, integration roles without <code>aws:SourceIp</code> where the caller has a stable egress address. Generated policies include conditions when you ask and omit them when you do not, because an absent constraint never produces an error.</p>



<h2 class="wp-block-heading">Four things the model cannot know</h2>



<p class="wp-block-paragraph">Some of this is not a capability gap you can prompt past. It is missing input:</p>



<ul class="wp-block-list"><li><strong>Which resources exist.</strong> A bucket name is a guess unless you supplied it. Guessed names either fail closed or, if the wildcard is loose enough, match something you never intended.</li><li><strong>Which resources are sensitive.</strong> Your backup bucket and your public assets bucket are indistinguishable strings. Sensitivity lives nowhere in the document.</li><li><strong>What else is attached.</strong> Effective permissions come from identity policies, resource policies, boundaries, SCPs and session policies together. Reviewing one document tells you very little.</li><li><strong>What the blast radius is.</strong> &#8220;Can delete objects&#8221; means one thing for a scratch bucket and another for the only copy of a client&#8217;s data.</li></ul>



<p class="wp-block-paragraph">The training material is also skewed. Public IAM examples, including plenty of vendor documentation and countless forum answers, are permissive because permissive examples work on the first try. A model that has read the internet has read a lot of over-broad policies presented as correct.</p>



<h2 class="wp-block-heading">Where the hard boundary belongs</h2>



<p class="wp-block-paragraph">Review is not a boundary. Review is a filter with a pass rate that drops the more policies you look at in a week, and generated policies are cheap to produce, so volume rises exactly when attention falls.</p>



<p class="wp-block-paragraph">The boundary has to be something that says no without reading the document. AWS gives you two mechanisms and they are not interchangeable.</p>



<p class="wp-block-paragraph">A <strong>permissions boundary</strong> is a managed policy attached to a user or role that caps what identity-based policies can grant it. It grants nothing itself. Effective permissions become the intersection of the identity policy and the boundary, so an identity policy allowing <code>iam:*</code> against a boundary that omits IAM produces no IAM access at all. If no boundary is attached, none is evaluated, so this is opt-in per identity.</p>



<p class="wp-block-paragraph">A <strong>service control policy</strong> sets the maximum available permissions for every principal in an account or organizational unit, and also grants nothing. It is the right home for things nobody in that account should ever do, and it applies whether or not somebody remembered to attach a boundary.</p>



<p class="wp-block-paragraph">The division I use: SCPs carry account-wide invariants, such as denying escalation-relevant IAM write actions to everyone outside the break-glass role and denying regions you do not operate in. Boundaries carry the per-workload ceiling, and they are what makes it safe to let a pipeline role create other roles. You can permit that role to create roles while requiring, through a condition on the boundary key, that every role it creates carries your boundary. The pipeline can then generate whatever policy it likes, because the ceiling is enforced at evaluation time rather than at review time.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph">If the only thing between a generated policy and production is a person reading JSON, you do not have a boundary. You have a habit.</p>
</blockquote>



<h2 class="wp-block-heading">The gate I would actually run</h2>



<p class="wp-block-paragraph">Mechanical checks in a pipeline, failing the build, not a checklist somebody is supposed to remember.</p>



<ol class="wp-block-list"><li><strong>Lint for nonsense.</strong> Parliament catches unknown action names, resource formats that cannot match the action, and type mismatches. Cheapest way to catch an invented action before it becomes a wildcard.</li><li><strong>Run AWS policy validation.</strong> Access Analyzer knows service-specific rules a generic linter does not.</li><li><strong>Assert the escalation actions are absent.</strong> Name the actions you never want granted and fail if any appear. This is the check that maps to the composition failure.</li><li><strong>Compare against the current policy.</strong> For edits, check whether the proposed document grants access the existing one does not. This one is billed per call, so run it on changed policies rather than the whole repository.</li><li><strong>Scan for known risky patterns.</strong> Cloudsplaining scores policies against categories including privilege escalation, data exfiltration and resource exposure, and produces something you can hand to a client.</li></ol>



<pre class="wp-block-code"><code># 1. Lint. Exit status is non-zero when there are findings.
parliament --file policy.json

# 2. AWS-side validation
aws accessanalyzer validate-policy 
  --policy-document file://policy.json 
  --policy-type IDENTITY_POLICY

# 3. Fail if any escalation-relevant action is granted
aws accessanalyzer check-access-not-granted 
  --policy-document file://policy.json 
  --access actions="iam:PassRole","iam:CreatePolicyVersion","iam:AttachRolePolicy" 
  --policy-type IDENTITY_POLICY

# 4. Does the proposed policy grant more than the current one?
aws accessanalyzer check-no-new-access 
  --existing-policy-document file://current.json 
  --new-policy-document file://proposed.json 
  --policy-type IDENTITY_POLICY

# 5. Risk-categorised report
cloudsplaining scan-policy-file --input-file policy.json</code></pre>



<p class="wp-block-paragraph">Step three is the one to add first if you only add one. It turns &#8220;we reviewed it&#8221; into &#8220;the build fails.&#8221;</p>



<p class="wp-block-paragraph">Attaching the boundary is a single call, and it is the part people skip because the role already works without it:</p>



<pre class="wp-block-code"><code>aws iam put-role-permissions-boundary 
  --role-name app-deploy-role 
  --permissions-boundary arn:aws:iam::123456789012:policy/WorkloadCeiling</code></pre>



<h2 class="wp-block-heading">Generate from observed behavior instead</h2>



<p class="wp-block-paragraph">There is a better source of truth than a model&#8217;s guess about what an application needs, and it is the application. Access Analyzer policy generation reads CloudTrail events for a role over a chosen window and writes a policy from what was actually called. Know its limits before relying on it:</p>



<ul class="wp-block-list"><li>It analyzes up to ninety days of history and needs a trail already logging for the account, so a role that only exercises certain paths quarterly produces an incomplete policy.</li><li>Coverage is per-service. For some services it identifies individual actions; for others it can only tell you the service was used and prompts you to fill in the actions yourself.</li><li>It does not produce action-level detail for data events such as S3 object-level operations.</li><li><code>iam:PassRole</code> is not included in generated policies. If your workload needs it, a human adds it back by hand, so the most escalation-relevant line in the document is the one the generator did not write.</li></ul>



<p class="wp-block-paragraph">Behavior-derived policies also fail in a useful direction. A missing permission produces a denial you can see in CloudTrail, which beats a permission you did not know you granted.</p>



<h2 class="wp-block-heading">Arguments that do not survive contact</h2>



<ul class="wp-block-list"><li><strong>&#8220;We review every policy before merge.&#8221;</strong> Review quality is a function of volume and attention, and generation raises volume. Composition failures are also the hardest to spot by reading, because spotting them means knowing what other roles exist and what they can do.</li><li><strong>&#8220;Access Analyzer came back clean.&#8221;</strong> Validation checks the document. It does not know that the role ARN you may pass happens to have <code>AdministratorAccess</code> attached. Clean findings on a policy that grants a dangerous chain is the normal case, not an anomaly.</li><li><strong>&#8220;A permissions boundary is overkill for a small project.&#8221;</strong> Small projects are where this bites hardest. One account, one deploy role, no organization, nothing between a bad policy and everything you own. The boundary is one managed policy and one CLI call.</li><li><strong>&#8220;We will tighten it after launch.&#8221;</strong> Nobody does. If you genuinely intend to, drive it from Access Analyzer&#8217;s unused access findings so you have a work queue instead of an intention.</li><li><strong>&#8220;Just prompt the model to follow least privilege.&#8221;</strong> That changes the output surface, not the reasoning. Fewer wildcards, better statements, no evaluation of what the combination permits.</li></ul>



<h2 class="wp-block-heading">How I would decide</h2>



<p class="wp-block-paragraph">My rule is about which actions are in the document, not about who or what wrote it.</p>



<p class="wp-block-paragraph"><strong>Ship generated policies freely</strong> when the document touches only data-plane actions on named resources, the identity has a boundary, and the pipeline checks pass. Read a bucket prefix, publish to a topic, write to a table. That is most policies, and hand-writing them wastes your afternoon.</p>



<p class="wp-block-paragraph"><strong>Treat generated output as a first draft</strong> when the policy includes any IAM write action, any <code>PassRole</code>, any trust policy edit, any resource policy readable from outside the account, or any KMS key policy. Use it to save typing, then verify every statement against the service authorization reference yourself.</p>



<p class="wp-block-paragraph"><strong>Do not generate at all</strong> for the boundary policies and SCPs. Those are the mechanism. Something that constrains everything else should be short, hand-written, understood by whoever owns the account, and changed rarely.</p>



<p class="wp-block-paragraph">Two notes for small engagements. If you run application infrastructure on a VPS from a provider like Contabo or InterServer and pull AWS in for storage or mail, that server holds long-lived credentials somewhere your AWS controls do not reach, so scope its policy to a bucket prefix and attach a boundary before worrying about anything else. And if the credentials go to a SaaS connector, a marketing platform such as GoHighLevel or a payments integration, you are handing keys to a system whose behavior you cannot audit. Those are the policies to write by hand.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Are AI-generated IAM policies safe to use in production?</h3>



<p class="wp-block-paragraph">For data-plane permissions on named resources, generally yes, provided the identity carries a permissions boundary and the policy passes automated validation. For anything involving IAM write actions, <code>PassRole</code>, trust policies or key policies, treat the output as a draft and verify each statement yourself. What matters is the blast radius of the actions, not the authorship.</p>



<h3 class="wp-block-heading">Why does a generated policy pass validation and still grant too much?</h3>



<p class="wp-block-paragraph">Validation examines one document in isolation. Privilege escalation usually comes from a combination of individually reasonable permissions, or from the interaction between the policy and something outside it, such as which roles are passable and what those roles can do. A document can be internally correct and still open a path.</p>



<h3 class="wp-block-heading">What is the difference between a permissions boundary and an SCP?</h3>



<p class="wp-block-paragraph">Both cap permissions and neither grants any. A boundary attaches to a single IAM user or role and limits what that identity&#8217;s policies can grant it. An SCP applies to every principal in an account or organizational unit and needs AWS Organizations. With one account and no organization, boundaries are what you have. With an organization, use both.</p>



<h3 class="wp-block-heading">Can IAM Access Analyzer replace writing policies by hand?</h3>



<p class="wp-block-paragraph">Partly. Generation from CloudTrail activity beats a guess because it reflects what the workload actually called. It is limited by the analysis window, by per-service coverage differences, by the absence of action-level detail for data events, and by <code>iam:PassRole</code> being excluded. Treat it as a strong draft that needs a human pass over anything the generator could not see.</p>



<h3 class="wp-block-heading">Which IAM actions should I block outright in a small account?</h3>



<p class="wp-block-paragraph">The ones that let a principal rewrite its own permissions or borrow another identity&#8217;s: creating and setting default policy versions, attaching and putting inline policies on users, roles and groups, creating access keys for other users, and passing roles to compute services. Deny them for everything except a break-glass identity, then grant them back deliberately where a workload genuinely needs one.</p>



<h3 class="wp-block-heading">How do I catch an invented action name before production?</h3>



<p class="wp-block-paragraph">Run a linter that carries the IAM action catalogue. Parliament flags unknown actions and exits non-zero when it finds anything, so it drops straight into a pipeline step. Catching it early matters less because the fake action is dangerous and more because the eventual fix for the denial tends to be a service-level wildcard.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">AI-generated IAM policies are not more dangerous because a model wrote them. They are more dangerous because they arrive faster than you can think about them, and because they arrive looking finished.</p>



<p class="wp-block-paragraph">So stop trying to make the review step better. Put the boundary somewhere that does not depend on anyone paying attention: a permissions boundary on every identity a pipeline can create or modify, an SCP denying the escalation actions to everyone who does not need them, and a build step that fails when a forbidden action shows up. Then let generation happen at whatever speed it wants.</p>



<p class="wp-block-paragraph">The policy approved in ninety seconds is not the failure. The absence of anything underneath it was.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help drawing that boundary in your account?</h2>



<p class="wp-block-paragraph">Most of the IAM work I take on is exactly this: an account that grew organically, permissions nobody wants to touch in case something breaks, and no ceiling underneath any of it. Things I can help with:</p>



<ul class="wp-block-list"><li>Auditing existing roles and policies for privilege escalation chains, including the passable-role paths that never show up in a single-document review</li><li>Designing and attaching permissions boundaries for pipeline, workload and human identities, including the delegation pattern that lets a deploy role create roles safely</li><li>Writing the SCP set for a small organization: escalation-relevant IAM actions, region restrictions, and protection for logging and billing resources</li><li>Adding Access Analyzer validation and custom policy checks to a CI pipeline so forbidden actions fail the build instead of relying on review</li><li>Replacing over-broad roles with policies generated from CloudTrail activity, then closing the gaps the generator cannot see</li><li>Scoping down credentials handed to third-party SaaS integrations and to servers outside AWS, where your account controls do not apply</li></ul>



<p class="wp-block-paragraph">If you want a second opinion on something specific, send me the policy JSON, the output of a failing <code>check-access-not-granted</code> run, or the role list from your account, and I will tell you what I would change and why.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/ai-generated-iam-policies/">AI-Generated IAM Policies: Where the Hard Boundary Belongs</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Twenty Client AWS Accounts and Not One Shared Admin Credential</title>
		<link>https://john-nessime.com/blog/devops/client-aws-account-access/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 27 Aug 2026 09:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[AWS Organizations]]></category>
		<category><![CDATA[AWS STS]]></category>
		<category><![CDATA[Client Onboarding]]></category>
		<category><![CDATA[CloudTrail]]></category>
		<category><![CDATA[Confused Deputy]]></category>
		<category><![CDATA[Cross-Account Access]]></category>
		<category><![CDATA[External ID]]></category>
		<category><![CDATA[IAM]]></category>
		<category><![CDATA[IAM Access Analyzer]]></category>
		<category><![CDATA[IAM Identity Center]]></category>
		<category><![CDATA[Least Privilege]]></category>
		<category><![CDATA[Managed Service Providers]]></category>
		<category><![CDATA[MFA]]></category>
		<category><![CDATA[Multi-Account Strategy]]></category>
		<category><![CDATA[Offboarding]]></category>
		<category><![CDATA[Permissions Boundaries]]></category>
		<category><![CDATA[Role Chaining]]></category>
		<category><![CDATA[Session Policies]]></category>
		<category><![CDATA[Source Identity]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=546</guid>

					<description><![CDATA[<p>Shared admin credentials work right up until someone resigns or an auditor asks who did what. Here's how to design client AWS account access around cross-account roles, unique external IDs and per-engineer attribution, so twenty accounts need zero shared keys and revocation is one action.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/client-aws-account-access/">Twenty Client AWS Accounts and Not One Shared Admin Credential</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">A client&#8217;s auditor asks one question: list every person at your company who can sign in to our AWS account, and show us how you know. If the honest answer starts with &#8220;whoever has the shared vault,&#8221; that isn&#8217;t an answer. It&#8217;s a hope.</p>



<p class="wp-block-paragraph">That question is what actually bites when you run infrastructure for twenty clients. Not a breach. An audit, or a resignation. Someone gives notice on a Friday and you spend Monday working out which accounts they still hold keys to, because the credential everybody shared was never attached to a person. You can&#8217;t revoke a person from a password. You can only rotate it and hope you got all twenty.</p>



<p class="wp-block-paragraph">This post covers how to design client AWS account access so that humans authenticate once against something you control, every client account holds a role that trusts exactly one thing, and every API call in a client&#8217;s CloudTrail can be traced back to a named engineer. It&#8217;s organised by failure family, because the mechanics are well documented and the failure modes are not.</p>



<h2 class="wp-block-heading">The shape: one identity plane, twenty separate trust decisions</h2>



<p class="wp-block-paragraph">The instinct that causes all the trouble is putting credentials in the middle. Client gives you a login, you store it somewhere shared, engineers take it out when they need it. Everything downstream of that decision is damage control.</p>



<p class="wp-block-paragraph">The shape that works is boring. One AWS account whose only job is identity. Your engineers exist there, once, in IAM Identity Center or federated from whatever directory you already pay for. Each client account holds an IAM role whose trust policy names one principal in that account and requires an external ID. Nobody stores a client credential, because there is no client credential. There&#8217;s a role your engineers can ask STS for a short-lived session against, and a client who can delete that role whenever they like.</p>



<p class="wp-block-paragraph">There is a second design worth knowing about: inviting client accounts into your own AWS Organization. It&#8217;s genuinely better in some ways. You get service control policies, resource control policies, and centralised root access management across the estate, all of which are organisation-scoped features you simply cannot use on an account you don&#8217;t own. It&#8217;s also a much bigger commitment. You take on consolidated billing, the client&#8217;s spend flows through you, and pulling an account back out of an organisation at the end of a contract is a real project rather than a policy change.</p>



<p class="wp-block-paragraph">For most consultancies and managed service providers, the client keeps their own organisation and you get a role. That&#8217;s the model the rest of this assumes.</p>



<h2 class="wp-block-heading">Failure family 1: credentials you can&#8217;t revoke</h2>



<p class="wp-block-paragraph">The shared IAM user with an access key is the default because it works immediately. It keeps working right up until one of three moments: an engineer leaves, a client asks who did something, or a laptop gets stolen.</p>



<p class="wp-block-paragraph">Access keys have no expiry. They&#8217;re bearer tokens with an indefinite lifetime, and the only way to invalidate one is to go into each account and delete it. At twenty accounts that&#8217;s twenty deliberate actions performed under time pressure, and the one you forget is the one that matters. A role session, by contrast, expires on its own. That&#8217;s the entire argument.</p>



<p class="wp-block-paragraph">A shared password manager still earns its place, just not for admin keys. Bitwarden or 1Password is the right home for per-client external IDs, account numbers, role ARNs and the client-side contacts you need when something breaks at two in the morning. Metadata, not credentials.</p>



<h2 class="wp-block-heading">Failure family 2: a trust policy that trusts more than you meant</h2>



<p class="wp-block-paragraph">This is the invisible one. The role gets created, the access works, everyone moves on, and the trust policy quietly grants far more than anyone intended.</p>



<p class="wp-block-paragraph">Two mistakes travel together. The first is naming your account root in the principal. Writing <code>arn:aws:iam::111122223333:root</code> does not mean &#8220;the root user of my account.&#8221; It means any principal in that account that has permission to call AssumeRole. Every engineer, every Lambda function, every CI runner, every future role you haven&#8217;t created yet. It delegates the decision to your own account&#8217;s IAM, which is fine when your IAM is tidy and quietly catastrophic when it isn&#8217;t.</p>



<p class="wp-block-paragraph">The second is skipping the external ID. AWS calls the problem it solves the confused deputy: you are a third party that holds the ability to assume roles in many customer accounts, and without a shared secret in the condition, one customer who learns another customer&#8217;s role ARN could persuade your systems to act against it. The external ID is the value that makes each trust relationship specific rather than generic.</p>



<p class="wp-block-paragraph">Generate it yourself, one per client, as a UUID or similar unguessable value. The client must not be the one choosing it, because if they can choose it they can choose someone else&#8217;s, and the control does nothing.</p>



<pre class="wp-block-code"><code>{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111122223333:role/MspOperationsAccess"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02"
        }
      }
    }
  ]
}</code></pre>



<p class="wp-block-paragraph">The principal names one role, not the account. If that role is compromised you have one thing to fix. If the account is named, you have an inventory problem you didn&#8217;t know you had.</p>



<h2 class="wp-block-heading">Failure family 3: CloudTrail says the role, not the human</h2>



<p class="wp-block-paragraph">You&#8217;ve done everything above and a client emails asking who deleted the staging load balancer. You open their CloudTrail and every event attributes to the same assumed role. The role is shared by your whole team by design, so the log tells you your company did it. That&#8217;s not what they asked.</p>



<p class="wp-block-paragraph">There are two knobs and they are not equivalent. The role session name is chosen freely by whoever calls AssumeRole, appears in the assumed-role ARN, and is trivially set to anything. It&#8217;s useful, but it&#8217;s a label, not evidence. Source identity is the one that carries weight: once set on a session it cannot be changed, it persists across chained role sessions, and it lands in CloudTrail in the session context of every subsequent call.</p>



<p class="wp-block-paragraph">The catch is that the role&#8217;s trust policy has to permit <code>sts:SetSourceIdentity</code> alongside <code>sts:AssumeRole</code>, and you can require it with a condition so a session without one is refused outright.</p>



<pre class="wp-block-code"><code>"Action": [
  "sts:AssumeRole",
  "sts:SetSourceIdentity"
],
"Condition": {
  "StringEquals": {
    "sts:ExternalId": "6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02"
  },
  "StringLike": {
    "sts:SourceIdentity": "*@yourcompany.com"
  }
}</code></pre>



<p class="wp-block-paragraph">If your engineers arrive through SAML federation, the identity provider can populate source identity from a directory attribute such as email, which means nobody has to remember to pass it. For direct CLI use it&#8217;s an explicit flag:</p>



<pre class="wp-block-code"><code>aws sts assume-role 
  --role-arn arn:aws:iam::444455556666:role/MspEngineerAccess 
  --role-session-name acme-investigation 
  --external-id 6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02 
  --source-identity jane.doe@yourcompany.com</code></pre>



<p class="wp-block-paragraph">Getting this right is what turns &#8220;we take security seriously&#8221; into a sentence you can back with a log query. It is also, in my experience of what clients actually ask for, the single most persuasive thing on the list.</p>



<h2 class="wp-block-heading">Failure family 4: one identity, twenty blast radii</h2>



<p class="wp-block-paragraph">A hub account is a concentration of risk. That&#8217;s the trade you&#8217;re making, and it&#8217;s worth making, but only if the hub is held to a higher standard than the accounts it reaches into.</p>



<p class="wp-block-paragraph">Practically: not every engineer gets permission to assume every client&#8217;s role. Group permissions in the hub so the people on the Acme engagement reach Acme and nothing else. Use per-client role names rather than one shared name across all twenty, so a policy in the hub can&#8217;t wildcard its way into a client you stopped working with a year ago.</p>



<p class="wp-block-paragraph">Tier the roles by what they&#8217;re for, not by who&#8217;s asking. A read-only role for investigation covers most of the work most days. A change role carries the write permissions the engagement actually needs. Break-glass sits separate, with a short maximum session duration and an alarm on every use.</p>



<p class="wp-block-paragraph">Two extras are worth knowing about even if you don&#8217;t reach for them immediately. Session policies let the caller narrow a session further at assume time, which is useful for automation that only needs one bucket. Permissions boundaries cap what any role or user created inside the client account can do, which matters when part of your job is creating IAM there.</p>



<p class="wp-block-paragraph">If your team already routes through a business VPN or zero trust gateway with stable egress addresses, a source IP condition on the trust policy is a cheap extra layer. NordLayer, Tailscale and Cloudflare&#8217;s zero trust products all give you predictable egress. Be honest about the cost, though: the moment something runs from CI, a client&#8217;s network or a laptop on hotel wifi, an IP condition becomes a support ticket. I&#8217;d add it once everything else works, not before.</p>



<h2 class="wp-block-heading">Failure family 5: standing administrator access nobody remembers granting</h2>



<p class="wp-block-paragraph">Somewhere in the twenty accounts there is a role with AdministratorAccess attached, created during a migration, described in the ticket as temporary. It&#8217;s been eighteen months.</p>



<p class="wp-block-paragraph">Here&#8217;s the honest part: AWS does not give you a native just-in-time approval workflow for cross-account role assumption. There&#8217;s no built-in &#8220;request four hours of write access, get approved in Slack, access expires automatically&#8221; for this pattern. You either build that on top of STS and your ticketing system, or you accept standing access and compensate with tight session durations, aggressive monitoring and a genuine periodic review. Both are defensible. Pretending you have JIT when you have a wiki page is not.</p>



<p class="wp-block-paragraph">The cheap version that actually helps: set the maximum session duration on client roles to something short, alert on assumption of the break-glass role, and run a quarterly pass over service last accessed data to strip permissions nobody has used. IAM can generate that report per role.</p>



<pre class="wp-block-code"><code># Kick off the report, then fetch it with the returned job ID
aws iam generate-service-last-accessed-details 
  --arn arn:aws:iam::444455556666:role/MspEngineerAccess

aws iam get-service-last-accessed-details --job-id JOB_ID_FROM_ABOVE</code></pre>



<p class="wp-block-paragraph">Run that in each client account and the conversation about narrowing permissions stops being theoretical. You&#8217;re not guessing what the engagement needs; you&#8217;re reading it.</p>



<h2 class="wp-block-heading">Failure family 6: offboarding that only half happens</h2>



<p class="wp-block-paragraph">The whole point of the hub is that offboarding is one action. Disable the person in your directory and every path into every client account closes at once, because there was never a credential in a client account with their name on it.</p>



<p class="wp-block-paragraph">That&#8217;s true only if nothing crept in sideways. Things that creep in sideways:</p>



<ul class="wp-block-list">
<li>IAM users created directly in a client account during an incident, because it was faster</li>

<li>Access keys minted under a role by automation and never inventoried</li>

<li>A client&#8217;s own console user shared with your team &#8220;just for the migration&#8221;</li>

<li>Third party tools connected with their own cross-account roles that outlived the trial</li>

<li>A departing engineer&#8217;s laptop with cached SSO tokens and stale profiles still on disk</li>
</ul>



<p class="wp-block-paragraph">IAM Access Analyzer is the tool that finds most of the first four. Its external access findings show every principal outside the account that can reach into it, which is exactly the list a client wants to see, and it&#8217;s also the list that reveals the monitoring vendor nobody remembers signing up for. For the laptop, whatever your hardware disposal process is, make sure it involves an actual wipe rather than a factory reset; tools like O&amp;O SafeErase exist for that reason and are cheap next to the alternative.</p>



<h2 class="wp-block-heading">Rolling client AWS account access out across twenty accounts</h2>



<p class="wp-block-paragraph">Doing this by hand twenty times guarantees twenty subtly different configurations. Make it a template and a checklist.</p>



<ol class="wp-block-list">
<li>Write one CloudFormation template that creates the role, its trust policy and its permissions, parameterised by external ID and your operations account ID.</li>

<li>Generate a unique external ID per client and store it with the client&#8217;s account metadata, not in the template.</li>

<li>Send the client a launch stack link rather than asking for credentials. They deploy it in their own account, in their own console, and can read exactly what they&#8217;re granting.</li>

<li>Record the account ID, role ARN, external ID location, engagement scope and review date in a single inventory file in version control.</li>

<li>Generate everyone&#8217;s local AWS profiles from that inventory so nobody hand-edits a config file.</li>

<li>Verify each one with a call that proves identity and nothing else before you touch anything real.</li>
</ol>



<p class="wp-block-paragraph">The generated profile chains from your federated session into the client role. The AWS CLI reads these from the shared config file and handles the assume-role call for you:</p>



<pre class="wp-block-code"><code># ~/.aws/config
[profile msp-hub]
sso_session = company
sso_account_id = 111122223333
sso_role_name = MspOperationsAccess
region = eu-west-1

[profile acme-readonly]
role_arn = arn:aws:iam::444455556666:role/MspEngineerReadOnly
source_profile = msp-hub
external_id = 6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02
role_session_name = jane-doe
duration_seconds = 3600
region = eu-west-1</code></pre>



<p class="wp-block-paragraph">Then the two-second sanity check that has saved me from more mistakes than any other command:</p>



<pre class="wp-block-code"><code>aws sts get-caller-identity --profile acme-readonly</code></pre>



<p class="wp-block-paragraph">It returns the account ID and the assumed-role ARN. Read the account number before you run anything destructive. Twenty profiles in one config file is exactly the situation where muscle memory picks the wrong one.</p>



<p class="wp-block-paragraph">For scheduled work that runs without a human, the same trust relationship serves a Lambda function or an EC2 instance role in your hub. A low-cost VPS from Contabo or InterServer will also do the job, but note the trade: an instance profile inside AWS gets rotated credentials automatically, while a box outside AWS needs a long-lived credential to bootstrap from, which reintroduces exactly what this design removes. Inside AWS is the cleaner answer.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting the failures you will actually hit</h2>



<h3 class="wp-block-heading">AccessDenied on AssumeRole</h3>



<p class="wp-block-paragraph">Three separate causes produce nearly the same message, and they need different fixes. Either the client&#8217;s trust policy doesn&#8217;t name your principal, or the external ID you sent doesn&#8217;t match what the condition expects, or your own identity in the hub lacks permission to call AssumeRole on that role ARN. Check them in that order, because the first two require the client and the third doesn&#8217;t.</p>



<h3 class="wp-block-heading">Switch Role in the console silently fails</h3>



<p class="wp-block-paragraph">This one wastes an afternoon if you don&#8217;t know it. The console&#8217;s switch role feature does not support roles that require an external ID. It&#8217;s a documented limitation, not a bug in your policy. Engineers who need console access to those accounts should reach them through federation rather than the switch role box.</p>



<h3 class="wp-block-heading">Sessions expiring after an hour when you asked for longer</h3>



<p class="wp-block-paragraph">When one role session assumes another role, that&#8217;s role chaining, and a chained session is capped at one hour regardless of the role&#8217;s maximum session duration setting. Since your engineers assume the client role from an already-assumed session in the hub, you&#8217;re chaining by definition. Plan around the hour rather than fighting it.</p>



<h3 class="wp-block-heading">An authorization message you can&#8217;t read</h3>



<p class="wp-block-paragraph">Some denials come back with a long encoded blob attached. STS will decode it if your identity has permission to, and the decoded version usually names the exact statement that denied you.</p>



<pre class="wp-block-code"><code>aws sts decode-authorization-message --encoded-message &lt;encoded-blob&gt;</code></pre>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Naming the account root as the trust principal instead of the specific operations role</li>

<li>Letting the client pick the external ID, or reusing one value across several clients</li>

<li>Treating the role session name as an audit trail when it&#8217;s a caller-supplied string</li>

<li>Asking a client for console credentials because a template felt like too much effort</li>

<li>Granting AdministratorAccess for a migration and never revisiting it</li>

<li>Building the hub properly and then leaving one legacy IAM user in one account as a fallback</li>

<li>Assuming that removing someone from your directory also removed the keys they created by hand</li>
</ul>



<h2 class="wp-block-heading">Best practices worth the effort</h2>



<ul class="wp-block-list">
<li>One identity plane, federated, with MFA enforced at the point the human authenticates</li>

<li>A unique unguessable external ID per client, generated by you and stored with client metadata</li>

<li>Source identity required in the trust policy so CloudTrail names a person, not a role</li>

<li>Roles tiered by purpose: read-only for investigation, change for the engagement, break-glass alarmed</li>

<li>Client-owned policies, deployed by the client from your template, revocable by the client in one action</li>

<li>A version-controlled inventory of every account, role, scope and review date</li>

<li>Access Analyzer external access findings reviewed on a schedule, in every client account</li>
</ul>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Can&#8217;t I just ask each client to create an IAM user for me?</h3>



<p class="wp-block-paragraph">You can, and it&#8217;ll work on day one. The problem is day four hundred. That user&#8217;s access key never expires, it isn&#8217;t attached to any individual on your team, and revoking it means someone in the client&#8217;s organisation remembering to act. A role costs the client the same five minutes to create and hands them a control they can exercise unilaterally.</p>



<h3 class="wp-block-heading">Do I need a different external ID for every client?</h3>



<p class="wp-block-paragraph">Yes. A shared external ID across your client base defeats the purpose, because the value&#8217;s entire job is to be specific to one trust relationship. Generate a UUID per client, treat it as a secret, and never let the client choose it.</p>



<h3 class="wp-block-heading">Should I bring client accounts into my own AWS Organization?</h3>



<p class="wp-block-paragraph">Only if you&#8217;re also taking on their billing and the commercial relationship supports it. Inside your organisation you get service control policies, resource control policies and centralised root access management, which are real advantages. Outside it, those stay the client&#8217;s tools and your job is to recommend them well. Most consulting engagements sit outside.</p>



<h3 class="wp-block-heading">How do I prove a departed engineer no longer has access?</h3>



<p class="wp-block-paragraph">Show the directory record disabling them, then show that no IAM user or access key in the client account carries their name, then show Access Analyzer external access findings listing only your operations role as an external principal. Three artefacts, all exportable. That&#8217;s the evidence pack.</p>



<h3 class="wp-block-heading">Does MFA carry across the cross-account role assumption?</h3>



<p class="wp-block-paragraph">Don&#8217;t assume it does. The reliable place to enforce MFA is where the human authenticates in your identity plane, and the reliable control at the client boundary is the combination of a narrowly scoped principal and the external ID condition. If you want an MFA condition in the client&#8217;s trust policy as well, test it end to end against your actual federation path before you rely on it, because the request context differs between direct assumption and chained sessions.</p>



<h3 class="wp-block-heading">What&#8217;s the minimum viable version if I only have three clients?</h3>



<p class="wp-block-paragraph">A hub account with federated engineers, one role per client scoped to a named principal with a unique external ID, and source identity required. Skip the tiering, skip the IP conditions, skip the JIT ambitions. Those three things give you revocation, attribution and a clean answer for the auditor, which is most of the value.</p>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">Client AWS account access is not really a permissions problem. It&#8217;s a revocation and attribution problem wearing a permissions costume. Any design where a credential exists that isn&#8217;t tied to a specific human will fail you at exactly the wrong moment, and no amount of policy tuning fixes that, because the gap isn&#8217;t in the policy.</p>



<p class="wp-block-paragraph">Put the humans in one place you control. Put a narrowly scoped, externally identified role in each client account. Require source identity so the log names a person. Everything else on this page is refinement; those three are the structure. And the pleasant side effect is that &#8220;here&#8217;s how you revoke us, unilaterally, in one action&#8221; is a genuinely good thing to be able to say in a sales conversation.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on your client access model?</h2>



<p class="wp-block-paragraph">This is the kind of work I do for consultancies and managed service providers who&#8217;ve grown past the point where a shared vault is survivable. Specifically:</p>



<ul class="wp-block-list">
<li>Reviewing existing cross-account trust policies for over-broad principals and missing external IDs</li>

<li>Designing a hub account and federation model that scales past a handful of clients without per-client special cases</li>

<li>Building the CloudFormation or Terraform template clients deploy themselves, so onboarding stops involving credentials</li>

<li>Turning on source identity end to end, including the identity provider side, so CloudTrail attributes to named engineers</li>

<li>Producing the offboarding evidence pack clients and auditors ask for, and automating it so it isn&#8217;t a scramble</li>

<li>Narrowing long-standing administrator roles down using service last accessed data, without breaking the engagement</li>
</ul>



<p class="wp-block-paragraph">If you want a concrete starting point, send me one trust policy and the AccessDenied message it produces. That&#8217;s usually enough to tell where the model is leaking.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/client-aws-account-access/">Twenty Client AWS Accounts and Not One Shared Admin Credential</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>The AWS Account Health Check I Run Before Quoting a Project</title>
		<link>https://john-nessime.com/blog/cloud-computing/aws-account-health-check/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Mon, 24 Aug 2026 09:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[FinOps]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Regions]]></category>
		<category><![CDATA[CloudTrail]]></category>
		<category><![CDATA[Confused Deputy]]></category>
		<category><![CDATA[Cost Explorer]]></category>
		<category><![CDATA[Credential Report]]></category>
		<category><![CDATA[Cross-Account Access]]></category>
		<category><![CDATA[Drift Detection]]></category>
		<category><![CDATA[External ID]]></category>
		<category><![CDATA[Freelancing]]></category>
		<category><![CDATA[GuardDuty]]></category>
		<category><![CDATA[IAM]]></category>
		<category><![CDATA[IAM Access Analyzer]]></category>
		<category><![CDATA[Least Privilege]]></category>
		<category><![CDATA[Project Scoping]]></category>
		<category><![CDATA[Root Access Management]]></category>
		<category><![CDATA[Terraform]]></category>
		<category><![CDATA[Trusted Advisor]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=552</guid>

					<description><![CDATA[<p>Before I quote AWS work, I run a read-only account health check: six passes covering identity, logging, region sprawl, infrastructure code, cost shape and support escalation. Here are the commands, the pass tests, and the findings that change the number.</p>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/aws-account-health-check/">The AWS Account Health Check I Run Before Quoting a Project</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">&#8220;It&#8217;s a small AWS account, nothing complicated, we just need someone to tidy it up.&#8221; That sentence has preceded more scope disasters than any other I know of in freelance cloud work, and it&#8217;s almost never a lie. The person saying it believes it, because the parts of an AWS account that make work expensive are the parts nobody looks at.</p>



<p class="wp-block-paragraph">The expensive stuff is not the running instances. It&#8217;s the access keys nobody can attribute, the CloudTrail trail that quietly stopped delivering months ago, the Terraform state that no longer matches reality, and the region somebody spun a proof of concept up in two years ago. None of that comes up on a kickoff call. All of it comes up in week three, after you&#8217;ve quoted.</p>



<p class="wp-block-paragraph">So I run an AWS account health check before I give anyone a number. It&#8217;s read-only, it takes a couple of hours, and it&#8217;s the cheapest insurance in consulting. This post is the actual check: the six passes, the commands, what each finding does to the estimate, and what to do when the client won&#8217;t give you the access to run it.</p>



<h2 class="wp-block-heading">Why the AWS account health check comes before the quote</h2>



<p class="wp-block-paragraph">A quote is a bet on how much unknown there is. You&#8217;re pricing the work in the brief plus everything you&#8217;ll find while doing it, and the second half is invisible until you look. Two accounts can look identical and differ by a factor of three in effort. The difference is nearly always archaeology: how much has to be reverse engineered before you can safely change anything.</p>



<p class="wp-block-paragraph">There&#8217;s a second reason that matters more than pricing. If you touch an account that already had a compromised key sitting in it, the incident becomes yours by proximity. Establishing what the account looked like before you arrived isn&#8217;t paranoia. It&#8217;s the only version of events you&#8217;ll be able to evidence later.</p>



<h2 class="wp-block-heading">The access I ask for, and what a refusal tells me</h2>



<p class="wp-block-paragraph">I ask for a cross-account IAM role that the client creates and my account assumes. Not an IAM user. Not keys in an email. A role, with an external ID the client generates.</p>



<p class="wp-block-paragraph">The external ID solves one specific problem. Without it, a trust policy that names my account can be used by anyone who talks me into assuming a role on their behalf, and I end up acting inside a third party&#8217;s account without knowing it. AWS calls this the confused deputy problem. It&#8217;s the piece of cross-account setup people skip most often.</p>



<p class="wp-block-paragraph">On that role I ask for two AWS managed policies: <code>SecurityAudit</code> and <code>ViewOnlyAccess</code>. The pairing is the point. <code>ViewOnlyAccess</code> lets you enumerate resources across services, so you can see what exists. <code>SecurityAudit</code> lets you read configuration and policy documents, so you can see how it&#8217;s wired together. Neither alone gives you the picture, and neither lets you change anything. Billing is separate again: Cost Explorer read access has to be added explicitly and the feature has to be enabled in the account.</p>



<p class="wp-block-paragraph">The reply to that request tells you as much as the check does.</p>



<ul class="wp-block-list">
<li>&#8220;Here&#8217;s the role ARN and external ID&#8221; means somebody there understands AWS identity. Good sign.</li>

<li>&#8220;I&#8217;ll just make you an admin user, easier&#8221; means nobody is managing access. Expect that looseness everywhere else.</li>

<li>&#8220;Our previous developer set it up and we can&#8217;t reach him&#8221; means you&#8217;re inheriting credentials nobody can account for. Price the rotation work as its own line.</li>

<li>&#8220;We&#8217;d rather not grant access before signing&#8221; is fair. Offer a paid discovery engagement. If they won&#8217;t pay for two hours of assessment, they won&#8217;t pay for the fix.</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The six passes</h2>



<p class="wp-block-paragraph">Each pass answers one question. Everything here is read-only, and the order matters, because each pass changes how you read the next.</p>



<h3 class="wp-block-heading">Pass 1: Who can do what</h3>



<p class="wp-block-paragraph">Confirm where you actually are first. That sounds trivial until you&#8217;re in an organization with a dozen similarly named accounts.</p>



<pre class="wp-block-code"><code>aws sts get-caller-identity
aws iam list-account-aliases
aws organizations describe-organization</code></pre>



<p class="wp-block-paragraph">If the third command fails because the account isn&#8217;t in an organization, that&#8217;s a finding by itself. Standalone means no service control policies, no organization trail, and no central place to revoke anything.</p>



<p class="wp-block-paragraph">Then the credential surface. The summary gives counts; the credential report gives per-principal detail, including when each password and key was last used.</p>



<pre class="wp-block-code"><code>aws iam get-account-summary

aws iam generate-credential-report
aws iam get-credential-report --query Content --output text | base64 --decode &gt; credential-report.csv</code></pre>



<p class="wp-block-paragraph">The report generates asynchronously, so the second call may need a retry the first time. What I read from it: how many IAM users exist at all, how many have keys that have never been used, and how many have keys last used a long time ago that are still active.</p>



<p class="wp-block-paragraph">An unused key is worse than a busy one. A key in daily use is attributable to something. A key last used eighteen months ago and still enabled belongs to a person or script nobody remembers, which means nobody will notice when someone else starts using it.</p>



<p class="wp-block-paragraph">Then check whether anyone has ever asked this question before you:</p>



<pre class="wp-block-code"><code>aws accessanalyzer list-analyzers</code></pre>



<p class="wp-block-paragraph">IAM Access Analyzer offers external access analyzers, which flag resources shared outside a zone of trust you define, and unused access analyzers, which surface unused roles, keys and passwords. Unused access analysis is billed per IAM role and user analyzed per month, so it isn&#8217;t free, but it answers the least privilege question far faster than reading policy documents by hand. An empty list means nobody has ever asked this account who can reach what.</p>



<p class="wp-block-paragraph">Root closes the pass. In an AWS Organizations setup, centralized root access management lets the management account or a delegated IAM administrator remove root credentials from member accounts entirely, so there&#8217;s no root password, key or MFA device to manage per account. Live root credentials on every member account is a real finding, and one of the few security fixes that&#8217;s genuinely quick.</p>



<p class="wp-block-paragraph"><em>Pass test: you can name every principal that can reach production, and say when each last did.</em></p>



<h3 class="wp-block-heading">Pass 2: Would anyone notice</h3>



<p class="wp-block-paragraph">This is the pass that most often changes my number, because it&#8217;s the failure that stays invisible until it isn&#8217;t. Logging that exists but has silently stopped working is more dangerous than no logging, since everybody believes they&#8217;re covered.</p>



<pre class="wp-block-code"><code>aws cloudtrail describe-trails 
  --query 'trailList[].{Name:Name,MultiRegion:IsMultiRegionTrail,Org:IsOrganizationTrail}'

aws cloudtrail get-trail-status --name TRAIL_NAME_HERE

aws configservice describe-configuration-recorder-status</code></pre>



<p class="wp-block-paragraph">The first command tells you a trail is configured. The second tells you whether it&#8217;s delivering, which is the part that matters. A trail whose last delivery failed because someone edited the destination bucket policy still shows up in the console as a configured trail. The status call is where the truth is.</p>



<p class="wp-block-paragraph">Multi-region matters for the same reason region sprawl does: a single-region trail records nothing about the region where somebody is quietly running compute. Check GuardDuty too, and how long logs are actually retained. Ninety days of retention against a one-year obligation is a project, not a task, and it belongs in the quote rather than in week four.</p>



<p class="wp-block-paragraph">One naming trap worth knowing: AWS renamed the original Security Hub to Security Hub CSPM and launched a separate, newer service under the Security Hub name. When a client says &#8220;we have Security Hub,&#8221; ask which one, because the findings format and the configuration are not the same thing.</p>



<p class="wp-block-paragraph"><em>Pass test: a trail is delivering right now, covers every region, and its logs are kept for at least as long as the business needs them.</em></p>



<h3 class="wp-block-heading">Pass 3: How wide the blast radius is</h3>



<p class="wp-block-paragraph">People forget regions. Not occasionally, constantly. A demo in Singapore, a test bucket in Ireland, a Lambda deployed from a tutorial that defaulted to us-east-1.</p>



<pre class="wp-block-code"><code>for r in $(aws ec2 describe-regions --query 'Regions[].RegionName' --output text); do
  count=$(aws ec2 describe-instances --region "$r" 
    --query 'length(Reservations[].Instances[])' --output text)
  echo "$r $count"
done</code></pre>



<p class="wp-block-paragraph">Left without <code>--all-regions</code>, <code>describe-regions</code> returns the regions enabled for the account, which is exactly what you want. Adding the flag pulls in disabled regions too, and the per-region calls against those just error out.</p>



<p class="wp-block-paragraph">Run the same loop shape for anything carrying state or cost: RDS instances, load balancers, EBS volumes, NAT gateways. NAT gateways are the classic. They bill hourly whether or not anything routes through them, and a forgotten one in a region nobody visits is pure waste. Then the two exposure checks I never skip:</p>



<pre class="wp-block-code"><code>aws ec2 describe-security-groups 
  --filters Name=ip-permission.cidr,Values=0.0.0.0/0 
  --query 'SecurityGroups[].{Id:GroupId,Name:GroupName,VpcId:VpcId}'

aws s3api list-buckets --query 'Buckets[].Name' --output text</code></pre>



<p class="wp-block-paragraph">For each bucket that matters, check its public access block configuration and its policy. An open security group on 443 in front of a load balancer is fine. An open security group on a database port is a different conversation, and you want that conversation before you own the account.</p>



<p class="wp-block-paragraph"><em>Pass test: you have a list of every region with resources in it, and nothing on that list surprises the client.</em></p>



<h3 class="wp-block-heading">Pass 4: Whether any of this is described in code</h3>



<p class="wp-block-paragraph">The question isn&#8217;t &#8220;do they use Terraform.&#8221; It&#8217;s whether the code still describes the account. Those diverge quietly. Somebody makes an urgent console change during an incident, never brings it back into code, and from then on the repo is fiction that plans cleanly right up until the day it destroys something.</p>



<pre class="wp-block-code"><code>terraform plan -refresh-only

terraform plan -detailed-exitcode</code></pre>



<p class="wp-block-paragraph">The refresh-only plan shows where real infrastructure has drifted from state without proposing to change anything. The detailed exit code version is the one for scripts: it exits 0 for no changes, 1 for an error and 2 when there&#8217;s a diff, so you can gate on drift instead of reading output by eye.</p>



<p class="wp-block-paragraph">Three things I want to know here:</p>



<ul class="wp-block-list">
<li>What fraction of live resources are managed by code? Half a codebase is often worse than none, because it creates false confidence about which half.</li>

<li>Where does state live and who can write to it? Remote state in a versioned, locked S3 bucket is a different risk profile from a state file on a laptop.</li>

<li>Does the pipeline that applies changes still run? Credentials that expired six months ago mean everything since went in by hand.</li>
</ul>



<p class="wp-block-paragraph">Tagging belongs here too. With nothing tagged you can&#8217;t attribute cost, can&#8217;t scope a change safely, and can&#8217;t tell a production resource from an abandoned experiment. Untagged environments add hours to everything that follows.</p>



<p class="wp-block-paragraph"><em>Pass test: a refresh-only plan comes back clean, or the drift is a short list you can read out loud.</em></p>



<h3 class="wp-block-heading">Pass 5: What shape the spend is in</h3>



<p class="wp-block-paragraph">I don&#8217;t care much about the total. I care about the shape: which services dominate, whether the curve is flat or climbing, and whether anything jumped without an explanation.</p>



<pre class="wp-block-code"><code>aws ce get-cost-and-usage 
  --time-period Start=$(date -d '6 months ago' +%Y-%m-01),End=$(date +%Y-%m-01) 
  --granularity MONTHLY 
  --metrics UnblendedCost 
  --group-by Type=DIMENSION,Key=SERVICE</code></pre>



<p class="wp-block-paragraph">The <code>date -d</code> syntax is GNU coreutils; on macOS use the BSD equivalent or a fixed start date. Worth knowing before you loop this: the Cost Explorer API bills per paginated request, unlike the console view, so a wide query across several dimensions is a small but real charge on the client&#8217;s bill. Keep it narrow and mention it in your notes.</p>



<ul class="wp-block-list">
<li>Data transfer in the top three usually means an architecture problem rather than a rate problem. Cross-AZ chatter and NAT gateway egress are the usual suspects.</li>

<li>A step change with no matching launch deserves a direct question. Sometimes it&#8217;s a feature. Sometimes it&#8217;s a runaway job. Occasionally it&#8217;s mining on a stolen key.</li>

<li>No budgets and no anomaly monitors means nobody finds out about a spike until the invoice arrives.</li>

<li>Heavy commitment purchases against workloads you&#8217;re about to re-architect are a constraint on your design, and need to be on the table before you propose anything.</li>
</ul>



<p class="wp-block-paragraph">If the client wants ongoing cost visibility past what Cost Explorer offers, platforms like Vantage or CloudZero sit on the Cost and Usage Report and give per-team allocation and unit economics that native tooling makes awkward. Raise it as a separate line, not part of the engineering work.</p>



<p class="wp-block-paragraph"><em>Pass test: you can explain the top three cost lines in a sentence each, and account for every month-on-month jump.</em></p>



<h3 class="wp-block-heading">Pass 6: What happens at two in the morning</h3>



<p class="wp-block-paragraph">Short pass, about escalation. What support plan is the account on, and would anything wake a human up?</p>



<p class="wp-block-paragraph">Support plan matters more than people expect. On Basic you get the service limits checks and a small subset of Trusted Advisor security and fault tolerance checks, manual refresh only, and no technical support cases at all. The full check library and the Trusted Advisor API need a paid plan. If your engagement depends on opening a case during a cutover, that&#8217;s a prerequisite, not a nice to have.</p>



<p class="wp-block-paragraph">There&#8217;s one date worth carrying in your head, because it&#8217;s a real deprecation rather than a version number. AWS has announced that Developer Support, Business Support and Enterprise On-Ramp are discontinued on 1 January 2027, with Business Support+ as the replacement tier. If you&#8217;re scoping anything that runs past that boundary, check which plan the client is on and whether their support cost is about to move.</p>



<p class="wp-block-paragraph">Then monitoring. CloudWatch alarms wired to an SNS topic with no confirmed subscribers are the same failure as the dead trail: configured, visible, useless. If they run anything beyond alarms, whether that&#8217;s Grafana Cloud, Datadog or a self-hosted Prometheus stack, find out who holds the credentials and whether the routing has been tested this year.</p>



<p class="wp-block-paragraph"><em>Pass test: you can name the human who gets woken up, and the path that reaches them.</em></p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The findings that actually move the number</h2>



<ul class="wp-block-list">
<li><strong>Unattributable credentials.</strong> Each one has to be rotated or removed, and each needs a conversation with whoever might break. That&#8217;s coordination time, which is the hardest kind to compress.</li>

<li><strong>No working audit trail.</strong> No baseline means you can&#8217;t prove what you did or didn&#8217;t change. Add the cost of standing logging up before the real work starts.</li>

<li><strong>Significant drift between code and reality.</strong> Reconciliation is slow, fiddly and carries destruction risk. The single largest multiplier I see.</li>

<li><strong>No tags.</strong> A tax on every subsequent task rather than a task of its own, since every scoping decision becomes an investigation.</li>

<li><strong>Resources in unmentioned regions.</strong> Each is a small parallel environment with its own security groups, its own logging gap and its own bill.</li>

<li><strong>One shared admin identity.</strong> Nothing in CloudTrail is attributable to a person, so your first task is identity work the client didn&#8217;t ask for.</li>
</ul>



<p class="wp-block-paragraph">I don&#8217;t present these as criticism. I present them as line items with an hour estimate against each, and let the client decide what&#8217;s in scope. That conversation is much easier before a contract exists than after.</p>



<h2 class="wp-block-heading">If you only run five checks</h2>



<p class="wp-block-paragraph">Sometimes you get twenty minutes and a screen share.</p>



<ol class="wp-block-list">
<li>Pull the credential report and count active keys that haven&#8217;t been used recently.</li>

<li>Confirm a CloudTrail trail exists, is multi-region, and is currently delivering.</li>

<li>Loop the enabled regions for running instances and NAT gateways.</li>

<li>Run a refresh-only plan against whatever infrastructure code exists.</li>

<li>Pull six months of cost grouped by service and look at the shape.</li>
</ol>



<p class="wp-block-paragraph">Those cover identity, evidence, blast radius, delivery speed and money. The rest of the check refines a picture these five already gave you.</p>



<h2 class="wp-block-heading">Where this check goes wrong</h2>



<ul class="wp-block-list">
<li><strong>Treating it as an audit.</strong> You&#8217;re scoping work, not issuing a compliance report. A twenty page findings document makes a prospect defensive and costs you a day.</li>

<li><strong>Checking one region.</strong> The default region in your profile is not where the problems are.</li>

<li><strong>Trusting the console over the API.</strong> Configured and working are different states, and the console is better at showing the first.</li>

<li><strong>Taking long-lived keys because it&#8217;s faster.</strong> Accept an admin access key by email to save fifteen minutes and you&#8217;ve become part of the problem you were hired to assess.</li>

<li><strong>Running the loops unannounced.</strong> A burst of API calls from an unfamiliar principal is exactly what GuardDuty exists to flag. Tell them you&#8217;re doing it and roughly when.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">How long should an AWS account health check take?</h3>



<p class="wp-block-paragraph">Two to three hours for a single account once you have access, including the write-up. Multi-account organizations take longer but not linearly, because the interesting variation concentrates in one or two accounts and the rest are copies of a landing zone pattern.</p>



<h3 class="wp-block-heading">Should I charge for the assessment?</h3>



<p class="wp-block-paragraph">For a short engagement, folding it into the proposal is fine. For anything substantial, or where the client is shopping several consultants, quote it as a small standalone piece with a written deliverable. It qualifies the client, protects your time, and the findings have value to them even if they hire someone else.</p>



<h3 class="wp-block-heading">What if the client won&#8217;t give me read-only access before signing?</h3>



<p class="wp-block-paragraph">Quote time and materials, or quote fixed price with an explicit assumptions list naming every unknown. Do not quote a fixed price on an account you haven&#8217;t seen. The pattern that works is a small paid discovery phase, then a fixed price set once you know what you&#8217;re dealing with.</p>



<h3 class="wp-block-heading">Is ReadOnlyAccess enough on its own?</h3>



<p class="wp-block-paragraph">It&#8217;s broader than you probably want. <code>ReadOnlyAccess</code> extends to reading data in some services, not just configuration, which is more than you need for scoping and more than many clients will be comfortable granting. The <code>SecurityAudit</code> and <code>ViewOnlyAccess</code> pairing is narrower and easier to justify in writing.</p>



<h3 class="wp-block-heading">Can I automate the whole thing?</h3>



<p class="wp-block-paragraph">The enumeration, yes, and it&#8217;s worth scripting since you&#8217;ll run it repeatedly. The interpretation, no. Whether five unused IAM users are a problem depends entirely on what the client is trying to do, and that judgement is most of what you&#8217;re selling.</p>



<h3 class="wp-block-heading">What if I find something actively compromised?</h3>



<p class="wp-block-paragraph">Stop and tell them immediately, in writing, with timestamps. Don&#8217;t remediate unilaterally: you&#8217;re read-only, and containment steps can destroy evidence. Your job is a clear description of what you saw and when, so they can involve whoever they need to.</p>



<h3 class="wp-block-heading">Does this apply to small accounts too?</h3>



<p class="wp-block-paragraph">Especially to small accounts. Larger organizations usually have some governance, even imperfect. A three year old account at a small business that four different contractors have touched is where the surprises live, and it&#8217;s exactly the account that gets described as simple.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">An AWS account health check isn&#8217;t a security exercise dressed up as sales. It&#8217;s the only reliable way to price uncertainty, and uncertainty is what you&#8217;re actually quoting on.</p>



<p class="wp-block-paragraph">If you take one habit from this, take the second pass. Everything else on the list is something you&#8217;d eventually trip over while doing the work. A logging gap is different, because it stays invisible while it costs you, and by the time it matters you&#8217;re the person who was in the account when it happened. Check the trail is delivering. Then check the rest.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on an AWS account?</h2>



<p class="wp-block-paragraph">I do this for consultancies scoping engagements and for businesses who inherited an account they don&#8217;t fully understand. Usually that looks like:</p>



<ul class="wp-block-list">
<li>A read-only AWS account health check with a written findings list, hour estimates per item, and a clear split between urgent and later.</li>

<li>Cross-account audit role setup, with a trust policy and external ID you can hand to any consultant without granting more than you meant to.</li>

<li>Credential archaeology: working out what every active access key belongs to, then rotating or removing the ones that belong to nothing.</li>

<li>Getting CloudTrail, AWS Config and log retention into a state where they&#8217;re delivering rather than just configured.</li>

<li>Terraform drift reconciliation, including importing unmanaged resources back under code without a destructive plan.</li>

<li>Cost shape reviews: the forgotten NAT gateways, orphaned volumes and idle regions, plus budgets and anomaly alerts that reach a real inbox.</li>
</ul>



<p class="wp-block-paragraph">If you&#8217;ve got a credential report, a <code>terraform plan</code> output or a Cost Explorer screenshot you can&#8217;t make sense of, send it over and I&#8217;ll tell you what I&#8217;d look at next.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/aws-account-health-check/">The AWS Account Health Check I Run Before Quoting a Project</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>AWS Cross-Account Data Sharing Done Safely: Share Access, Not Copies</title>
		<link>https://john-nessime.com/blog/cloud-computing/aws-cross-account-data-sharing/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sat, 22 Aug 2026 09:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS KMS]]></category>
		<category><![CDATA[AWS Organizations]]></category>
		<category><![CDATA[AWS RAM]]></category>
		<category><![CDATA[Bucket Policy]]></category>
		<category><![CDATA[Cross-Account Access]]></category>
		<category><![CDATA[Data Governance]]></category>
		<category><![CDATA[Data Sharing]]></category>
		<category><![CDATA[External ID]]></category>
		<category><![CDATA[IAM]]></category>
		<category><![CDATA[IAM Access Analyzer]]></category>
		<category><![CDATA[Lake Formation]]></category>
		<category><![CDATA[Least Privilege]]></category>
		<category><![CDATA[Multi-Account Strategy]]></category>
		<category><![CDATA[Resource Control Policies]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=445</guid>

					<description><![CDATA[<p>Adding another account to a bucket policy takes one line and delegates far more than most people realize. A practical look at AWS cross-account data sharing: the delegation model, the KMS gaps, object ownership, references versus copies, and the guardrails that catch the mistake you will eventually make.</p>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/aws-cross-account-data-sharing/">AWS Cross-Account Data Sharing Done Safely: Share Access, Not Copies</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The request usually arrives as one sentence in Slack. &#8220;Analytics moved to their own account, can you just add them to the bucket policy?&#8221;</p>



<p class="wp-block-paragraph">It is a one-line change. It works on the first try. And it is the grant most likely to still be sitting there long after that analytics project shipped, covering a wider set of identities than the person who wrote it ever had in mind.</p>



<p class="wp-block-paragraph">That asymmetry is the whole problem. A denied cross-account request is a nuisance: someone pings you, you read the policy, you fix it before lunch. A cross-account grant that succeeds, keeps succeeding, and quietly reaches further than intended produces no error, no alert, and no ticket. It surfaces months later during an audit, when someone asks who has been reading that prefix and the honest answer is &#8220;anyone the other account decided to let in.&#8221;</p>



<p class="wp-block-paragraph">This post covers AWS cross-account data sharing as a set of failure families rather than a service tour. How the delegation model actually behaves, where encryption keys break or widen access without warning, who ends up owning the bytes after a handoff, why a copy is worse than a reference, and which guardrails catch the mistake you are eventually going to make. Real policy snippets and commands throughout, plus the denials that are hardest to read.</p>



<h2 class="wp-block-heading">AWS cross-account data sharing is a delegation, not a permission</h2>



<p class="wp-block-paragraph">Start with the mechanic that everything else hangs off. For a request that crosses an account boundary, two separate accounts have to say yes. The resource-based policy in the account that owns the data has to allow the caller, and an identity-based policy in the calling account has to allow the action. Neither side alone is enough.</p>



<p class="wp-block-paragraph">That is why &#8220;I added them to the bucket policy and it still fails&#8221; is such a routine ticket. The producer did their half. Nobody attached the matching IAM policy on the consumer side.</p>



<p class="wp-block-paragraph">Now the part that bites. Look at how most cross-account bucket policies get written:</p>



<pre class="wp-block-code"><code>{
  "Sid": "AllowAnalyticsAccount",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:root" },
  "Action": ["s3:GetObject", "s3:ListBucket"],
  "Resource": [
    "arn:aws:s3:::shared-events-bucket",
    "arn:aws:s3:::shared-events-bucket/exports/*"
  ]
}</code></pre>



<p class="wp-block-paragraph">Despite how it reads, that <code>:root</code> suffix does not mean the root user. It means the account. You have not granted access to a principal at all. You have delegated the guest list to the administrators of account 111122223333, and they can hand that access to any principal they like, including a role somebody creates next quarter for a purpose you will never hear about.</p>



<p class="wp-block-paragraph">Sometimes that delegation is exactly what you want, because the consumer team manages its own roles and you do not want to be in the loop for every change. Usually you want the narrower version:</p>



<pre class="wp-block-code"><code>{
  "Sid": "AllowAnalyticsRoleOnly",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:role/analytics-reader" },
  "Action": ["s3:GetObject", "s3:ListBucket"],
  "Resource": [
    "arn:aws:s3:::shared-events-bucket",
    "arn:aws:s3:::shared-events-bucket/exports/*"
  ],
  "Condition": {
    "StringEquals": { "aws:PrincipalOrgID": "o-abc123def4" }
  }
}</code></pre>



<p class="wp-block-paragraph">Both halves earn their place. The named ARN stops the wrong role in the right account. The organization condition stops the right role name in the wrong account, which matters if the statement is ever copied, templated, or widened by someone in a hurry. One protects against internal drift, the other against the policy escaping your organization entirely.</p>



<p class="wp-block-paragraph">Be clear-eyed about the trade-off, though. Naming ARNs is precise and brittle: every new consumer role is a policy change, a pull request, and a review. Organization-wide conditions scale beautifully and narrow nothing inside a member account. If your organization contains twelve accounts and forty teams, <code>aws:PrincipalOrgID</code> on its own is a fairly wide door.</p>



<h3 class="wp-block-heading">The grant that stops working after a rebuild</h3>



<p class="wp-block-paragraph">When you name a role ARN as a principal, IAM stores the role&#8217;s internal unique ID behind the scenes. Delete that role and the policy will start displaying an opaque <code>AROA...</code> string where the ARN used to be, because the ARN no longer resolves to anything. Recreate a role with the identical name and the access does not come back, because the new role has a new unique ID.</p>



<p class="wp-block-paragraph">A <code>terraform destroy</code> followed by <code>terraform apply</code> on the consumer side reproduces this perfectly. If you are staring at a bucket policy that contains a principal starting with AROA, that is your answer. Rewrite the statement with the current ARN.</p>



<h2 class="wp-block-heading">The encryption boundary nobody tests until it is live</h2>



<p class="wp-block-paragraph">Second failure family: the key. Encrypting shared data is the easy decision. What people miss is that a KMS key is a second, fully independent authorization surface, with its own resource policy, evaluated separately from the bucket.</p>



<ul class="wp-block-list">
<li>Downloading an object encrypted with SSE-KMS requires <code>kms:Decrypt</code> on the key.</li>

<li>Uploading one requires <code>kms:GenerateDataKey</code>.</li>

<li>A multipart upload requires both.</li>
</ul>



<p class="wp-block-paragraph">The AWS managed key for S3 cannot carry you here, because you cannot edit its key policy. Sharing KMS-encrypted objects across accounts means a customer managed key, with a statement naming the consumer:</p>



<pre class="wp-block-code"><code>{
  "Sid": "AllowAnalyticsRoleToDecrypt",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:role/analytics-reader" },
  "Action": ["kms:Decrypt", "kms:DescribeKey"],
  "Resource": "*",
  "Condition": {
    "StringEquals": { "kms:ViaService": "s3.us-east-1.amazonaws.com" }
  }
}</code></pre>



<p class="wp-block-paragraph">The <code>kms:ViaService</code> condition is cheap insurance. It restricts use of the key to requests that arrive through S3 in that region, so a leaked set of consumer credentials cannot call KMS directly and start decrypting ciphertext it obtained some other way. And the two-key rule applies to the key exactly as it does to the bucket: the consumer&#8217;s own IAM policy has to allow the KMS actions too.</p>



<p class="wp-block-paragraph">Now the sharp edge, which is worth reading twice. If a KMS key is specified by <em>alias</em> rather than by fully qualified ARN, the alias is resolved in the requester&#8217;s account. In a cross-account write that means data landing in your bucket can end up encrypted under a key belonging to the writer. You own the object and cannot read it, and no error is raised at write time. Use full key ARNs in anything that crosses an account boundary.</p>



<h2 class="wp-block-heading">Who owns the bytes after the handoff</h2>



<p class="wp-block-paragraph">Third family: ownership, and it depends entirely on which direction the data moves.</p>



<p class="wp-block-paragraph">New buckets are created with Object Ownership set to Bucket owner enforced. ACLs are off, and the bucket owner owns every object written into the bucket regardless of who wrote it. That default removed the single most confusing trap in S3 sharing.</p>



<p class="wp-block-paragraph">Buckets created before that default are the ones to check. If a bucket still sits on the Object writer setting, an object uploaded by another account is owned by that account, and you, holding the bucket, may not be able to read it. Nothing fails at write time. It fails later, when a job in your own account tries to read your own bucket and gets a 403.</p>



<p class="wp-block-paragraph">Check and fix it with the ownership controls API:</p>



<pre class="wp-block-code"><code>aws s3api get-bucket-ownership-controls --bucket shared-events-bucket

aws s3api put-bucket-ownership-controls 
  --bucket shared-events-bucket 
  --ownership-controls '{"Rules":[{"ObjectOwnership":"BucketOwnerEnforced"}]}'</code></pre>



<p class="wp-block-paragraph">Order matters when you make that change on a live bucket. Once ACLs are disabled, any PUT that still carries an ACL is rejected with <code>AccessControlListNotSupported</code>. Requests that send no ACL, or send <code>bucket-owner-full-control</code>, keep working. So the sequence is:</p>



<ol class="wp-block-list">
<li>Find every writer that sets an ACL, including old scripts using <code>--acl</code> and any SDK call passing an ACL argument.</li>

<li>Update those clients so they stop sending one.</li>

<li>Confirm from CloudTrail that ACL-bearing PUTs have stopped.</li>

<li>Flip the bucket to Bucket owner enforced.</li>
</ol>



<p class="wp-block-paragraph">Given a free choice of design, I would rather the consumer pull from a bucket I own than push into a bucket I do not. Pull keeps ownership, lifecycle rules, encryption configuration, and access logging in one account, under one team&#8217;s control. Push scatters them.</p>



<h2 class="wp-block-heading">Copies you cannot recall</h2>



<p class="wp-block-paragraph">Fourth family, and the one with the longest tail. There is a large difference between granting a reference and shipping a copy. A reference can be revoked on a Tuesday afternoon. A copy is simply gone.</p>



<p class="wp-block-paragraph">Three reference-shaped patterns are worth knowing before you reach for a bulk export.</p>



<h3 class="wp-block-heading">S3 Access Points for object-level sharing</h3>



<p class="wp-block-paragraph">Instead of a bucket policy that grows a new statement every time a team asks for something, you write the bucket policy once to delegate to access points, then give each consumer their own access point with its own focused policy. The delegation is expressed with the <code>s3:DataAccessPointAccount</code> condition. Access points can also be owned by the consumer&#8217;s account, so they manage their own detail without touching your bucket policy.</p>



<p class="wp-block-paragraph">The gotcha: when a request goes through an access point, both the access point policy and the underlying bucket policy have to allow it. A 403 here is easy to misdiagnose because the bucket policy looks correct and is correct. The access point is the layer that said no.</p>



<h3 class="wp-block-heading">Lake Formation for table-level sharing</h3>



<p class="wp-block-paragraph">When the thing being shared is a table rather than a prefix, Lake Formation is the better fit, because it lets you grant on databases, tables, columns, and row filters instead of on object paths. Cross-account grants ride on AWS Resource Access Manager, so the recipient gets an invitation that a data lake administrator has to accept.</p>



<pre class="wp-block-code"><code>aws ram get-resource-share-invitations

aws ram accept-resource-share-invitation 
  --resource-share-invitation-arn arn:aws:ram:us-east-1:111122223333:resource-share-invitation/EXAMPLE</code></pre>



<p class="wp-block-paragraph">After the invitation is accepted, the shared database and tables are visible but still not queryable from Athena or Redshift Spectrum until someone creates a <em>resource link</em>, which behaves roughly like a symlink in the recipient&#8217;s Data Catalog. Missing resource links account for most of the &#8220;the share went through but I cannot see the table&#8221; messages.</p>



<p class="wp-block-paragraph">One setting to check on the producer side is the cross-account data sharing version. Newer versions collapse many grants into far fewer RAM resource shares, which matters once you are sharing at any scale, and they allow granting directly to IAM principals in another account rather than only at account level. Read the current version with:</p>



<pre class="wp-block-code"><code>aws lakeformation get-data-lake-settings</code></pre>



<h3 class="wp-block-heading">When a copy really is the answer</h3>



<p class="wp-block-paragraph">Sometimes the partner genuinely needs the data on their own infrastructure: a warehouse you have no reach into, a VPS at a provider like Contabo or InterServer, or in the worst case an analyst&#8217;s laptop. That is a legitimate outcome, but be honest with yourself and with them that your controls stop at the handoff.</p>



<p class="wp-block-paragraph">Practical hygiene for that case: scope the extract to the columns actually needed rather than the whole table, put an expiry date in the agreement rather than in your head, and write down what happens to the copy when the engagement ends. If the copy lives on machines you or the client control, verified erasure with something like O&amp;O SafeErase reads a great deal better in an audit than &#8220;we deleted the folder.&#8221; And if the recipient has no AWS account at all, a time-limited presigned URL is usually a better answer than minting an IAM user with long-lived keys.</p>



<h2 class="wp-block-heading">The guardrail that catches the mistake you will make</h2>



<p class="wp-block-paragraph">Everything above describes what a careful engineer does per grant. Guardrails are what happens when somebody is not careful, which over a long enough window includes all of us.</p>



<p class="wp-block-paragraph">Resource control policies are the strongest lever if you run AWS Organizations. An RCP sets a ceiling on what any resource-based policy inside the organization can grant. The canonical statement is &#8220;no principal outside my organization may access S3 buckets in my accounts, regardless of what an individual bucket policy says.&#8221; That inverts the risk model: a careless bucket policy is no longer an exposure, it is just an ineffective statement.</p>



<p class="wp-block-paragraph">Two limits are worth stating plainly. RCPs cover a specific set of services rather than everything in AWS, so check the current supported list rather than assuming coverage. And they constrain your resources, not a partner&#8217;s behavior once data has legitimately left. An RCP is a wall around your side of the fence.</p>



<p class="wp-block-paragraph">Service control policies do the mirror-image job on the consumer side, limiting which external accounts your own principals are allowed to assume roles into. Account-level S3 Block Public Access remains a sensible floor underneath both.</p>



<p class="wp-block-paragraph">For sharing with a third party rather than an internal account, the external ID is not optional in my book:</p>



<pre class="wp-block-code"><code>{
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::444455556666:role/vendor-connector" },
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringEquals": { "sts:ExternalId": "a-value-the-vendor-generated-for-you" }
  }
}</code></pre>



<p class="wp-block-paragraph">The external ID exists because of the confused deputy problem. A vendor holds credentials that can assume roles across many customers, and without a per-customer secret in the condition, customer A can trick the vendor into pointing that access at customer B. If a vendor asks you to create a role for them with no external ID and no conditions at all, that is worth a conversation before it is worth a role.</p>



<h2 class="wp-block-heading">Verifying that the grant is what you think it is</h2>



<p class="wp-block-paragraph">Reading policies is not the same as knowing what they permit. IAM Access Analyzer answers the question &#8220;what is currently reachable from outside my zone of trust&#8221; using automated reasoning rather than pattern matching, which is why it catches combinations a human reviewer skims past.</p>



<pre class="wp-block-code"><code>aws accessanalyzer create-analyzer 
  --analyzer-name external-access 
  --type ACCOUNT

aws accessanalyzer list-findings 
  --analyzer-arn arn:aws:access-analyzer:us-east-1:999988887777:analyzer/external-access</code></pre>



<p class="wp-block-paragraph">Scope the zone of trust to your organization where you have one, then archive the findings that represent intentional shares, so that a new entry in the list actually means something.</p>



<p class="wp-block-paragraph">The same service exposes custom policy checks you can run in a pipeline before a policy change merges, including checks for public access, for specific actions being granted, and for whether an updated policy grants more than the one it replaces. One thing to understand about them: they evaluate the document you hand them and nothing else. They do not read your account state, so a public-access check can fail on a policy in an account that has Block Public Access enabled. For a merge gate that is correct behavior, because the policy should not be relying on a distant safety net.</p>



<p class="wp-block-paragraph">Finally, CloudTrail tells you who actually used a grant, which is the input to the only cleanup that ever really works: delete the statements nobody has exercised. Routing that history into CloudWatch, or into a Grafana Cloud dashboard alongside the rest of your platform telemetry, turns &#8220;is this still needed&#8221; from an argument into a query.</p>



<h2 class="wp-block-heading">Troubleshooting the denials that are hard to read</h2>



<ul class="wp-block-list">
<li><strong>You can list the bucket but every GET returns 403.</strong> That pattern points at KMS, not S3. Listing does not touch the key; reading does. Check <code>kms:Decrypt</code> on both the key policy and the consumer&#8217;s identity policy.</li>

<li><strong>403 through an access point while the bucket policy is clearly correct.</strong> The access point has its own policy and it also has to allow the calling principal. Read that one next.</li>

<li><strong>AssumeRole is denied although the trust policy names the caller.</strong> Cross-account needs both halves: the caller&#8217;s identity policy must permit <code>sts:AssumeRole</code> against that role ARN. Then check the external ID value and any SCP or permissions boundary in the calling account.</li>

<li><strong>A principal in the policy displays as AROA followed by random characters.</strong> The referenced role was deleted. Recreating it under the same name will not restore access.</li>

<li><strong>A Lake Formation share appears in the catalog but queries fail.</strong> Usually a missing resource link in the recipient account, or a RAM invitation that was never accepted.</li>

<li><strong>Access worked yesterday and stopped today with nothing changed locally.</strong> Look at the organization layer before re-reading the bucket policy. A newly attached RCP or SCP denies in a way that no amount of resource-policy editing will fix.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Granting to an account principal when a role ARN would have done, then never revisiting it.</li>

<li>Treating <code>aws:PrincipalOrgID</code> as least privilege. It narrows the organization, not the account.</li>

<li>Sharing a bucket and forgetting the key, then debugging S3 for an hour over a KMS denial.</li>

<li>Referring to KMS keys by alias in cross-account configuration instead of by full ARN.</li>

<li>Letting a partner push into your bucket without confirming the ownership setting first.</li>

<li>Exporting a full table when the consumer needed four columns, because the export was easier to write.</li>

<li>Creating a vendor role with no external ID and no conditions because the vendor&#8217;s setup guide said to.</li>

<li>Granting access with no expiry, no owner, and no note in the policy Sid explaining why it exists.</li>
</ul>



<h2 class="wp-block-heading">Best practices worth the effort</h2>



<ul class="wp-block-list">
<li>Give every cross-account grant a descriptive <code>Sid</code>. It is the only free documentation slot IAM offers, and future-you will read it.</li>

<li>Prefer one dedicated consumer role per relationship over a shared role used by several integrations. Revocation becomes a single delete.</li>

<li>Pair a specific principal with a broad condition, rather than relying on either alone.</li>

<li>Put policy checks in CI so a widened grant fails a pull request instead of a quarterly review.</li>

<li>Keep an inventory of active shares with an owner and a review date, even if it is a table in a repository README.</li>

<li>Enable an external access analyzer scoped to your organization and treat new findings as work, not noise.</li>

<li>Watch cross-region and internet data transfer when a consumer sits somewhere unexpected. Cost visibility tooling such as Vantage or CloudZero will attribute it faster than reading a bill.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">What is the safest way to share an S3 bucket with another AWS account?</h3>



<p class="wp-block-paragraph">Name a specific role ARN in the bucket policy, add an organization or account condition alongside it, scope the resource to a prefix rather than the whole bucket, and make sure the consumer&#8217;s identity policy is equally narrow. If the bucket is encrypted with a customer managed key, grant on the key as well. If the list of consumers is going to grow, move to access points rather than growing the bucket policy.</p>



<h3 class="wp-block-heading">Do I need both a bucket policy and an IAM policy?</h3>



<p class="wp-block-paragraph">For cross-account access, yes. The resource-based policy in the owning account and the identity-based policy in the calling account both have to allow the action. That is the single most common reason a correct-looking bucket policy still returns Access Denied.</p>



<h3 class="wp-block-heading">Why do I get Access Denied when the bucket policy clearly allows the other account?</h3>



<p class="wp-block-paragraph">Work through it in this order: the consumer&#8217;s identity policy, then the KMS key policy if the objects are encrypted with a customer managed key, then any access point policy in the path, then organization-level SCPs and RCPs. Denials from the organization layer are the ones people waste the most time on, because nothing in either account&#8217;s policies looks wrong.</p>



<h3 class="wp-block-heading">Can I share data with an account outside my AWS Organization?</h3>



<p class="wp-block-paragraph">You can, and organization conditions will not help you there. Use a named role ARN, an external ID on any assumed role, a tightly scoped resource, and a defined end date. Also confirm whether an RCP in your organization already blocks external principals for that service, because that is a common reason a partner grant refuses to work no matter how carefully it is written.</p>



<h3 class="wp-block-heading">Should I use S3 Access Points or Lake Formation?</h3>



<p class="wp-block-paragraph">It depends on the unit of sharing. If the consumer wants objects under a prefix, access points are simpler and cheaper to operate. If the consumer wants a table, and especially if you need column or row filtering, Lake Formation is the right tool and access points would force you to fake fine-grained control with prefix layout. Plenty of environments run both.</p>



<h3 class="wp-block-heading">How do I revoke cross-account access cleanly?</h3>



<p class="wp-block-paragraph">Remove the statement from the resource policy, remove the KMS key grant, and check for any access point, RAM share, or resource link that also carries permission. Then confirm with CloudTrail that the principal has stopped appearing. Revocation feels done long before it is done, which is exactly why an inventory of active shares pays for itself.</p>



<h3 class="wp-block-heading">Does cross-account access change anything about cost?</h3>



<p class="wp-block-paragraph">The request charges land on the bucket owner by default, and data transfer depends on whether the consumer is in the same region. A consumer that quietly starts scanning a large prefix from another region is a billing event as much as a security one, which is a good argument for reviewing access patterns and spend together rather than in separate meetings.</p>



<h2 class="wp-block-heading">The one thing to remember</h2>



<p class="wp-block-paragraph">Getting AWS cross-account data sharing right is less about knowing which service to reach for and more about noticing which grants are quietly larger than they read. An account principal delegates your guest list. An organization condition narrows the wrong dimension. A copy leaves your perimeter permanently and no policy edit brings it back.</p>



<p class="wp-block-paragraph">Grant references rather than copies, name principals rather than accounts, remember that the key is a second door, and put something in place that watches the perimeter for you. The cross-account grant worth worrying about is never the one that failed. It is the one that worked, and kept working, long after anybody was still looking at it.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on your cross-account setup?</h2>



<p class="wp-block-paragraph">Most of the cross-account problems I see are not exotic. They are a policy written in a hurry two years ago that nobody has read since. If that sounds familiar, here is where I can help:</p>



<ul class="wp-block-list">
<li>Auditing existing bucket, key, and role trust policies to find grants that are wider than intended, and rewriting them to named principals with conditions.</li>

<li>Designing a producer and consumer pattern for a specific data handoff, including whether access points, Lake Formation, or a plain scoped role is the right shape.</li>

<li>Setting up IAM Access Analyzer with a sensible zone of trust and archive rules, so the findings list stays worth reading.</li>

<li>Adding policy checks to your pipeline so a widened grant blocks a merge rather than surviving to production.</li>

<li>Building an organization-level perimeter with resource control policies and service control policies without breaking the integrations you already depend on.</li>

<li>Untangling stubborn cross-account denials, including the KMS, access point, and organization-layer cases that hide well.</li>
</ul>



<p class="wp-block-paragraph">Send me a bucket policy, a trust policy, or the CloudTrail entry for the denial you cannot explain, and I will tell you what it actually permits.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/aws-cross-account-data-sharing/">AWS Cross-Account Data Sharing Done Safely: Share Access, Not Copies</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
