A client’s auditor asks one question: list every person at your company who can sign in to our AWS account, and show us how you know. If the honest answer starts with “whoever has the shared vault,” that isn’t an answer. It’s a hope.<\/p>\n\n\n\n
That question is what actually bites when you run infrastructure for twenty clients. Not a breach. An audit, or a resignation. Someone gives notice on a Friday and you spend Monday working out which accounts they still hold keys to, because the credential everybody shared was never attached to a person. You can’t revoke a person from a password. You can only rotate it and hope you got all twenty.<\/p>\n\n\n\n
This post covers how to design client AWS account access so that humans authenticate once against something you control, every client account holds a role that trusts exactly one thing, and every API call in a client’s CloudTrail can be traced back to a named engineer. It’s organised by failure family, because the mechanics are well documented and the failure modes are not.<\/p>\n\n\n\n
The instinct that causes all the trouble is putting credentials in the middle. Client gives you a login, you store it somewhere shared, engineers take it out when they need it. Everything downstream of that decision is damage control.<\/p>\n\n\n\n
The shape that works is boring. One AWS account whose only job is identity. Your engineers exist there, once, in IAM Identity Center or federated from whatever directory you already pay for. Each client account holds an IAM role whose trust policy names one principal in that account and requires an external ID. Nobody stores a client credential, because there is no client credential. There’s a role your engineers can ask STS for a short-lived session against, and a client who can delete that role whenever they like.<\/p>\n\n\n\n
There is a second design worth knowing about: inviting client accounts into your own AWS Organization. It’s genuinely better in some ways. You get service control policies, resource control policies, and centralised root access management across the estate, all of which are organisation-scoped features you simply cannot use on an account you don’t own. It’s also a much bigger commitment. You take on consolidated billing, the client’s spend flows through you, and pulling an account back out of an organisation at the end of a contract is a real project rather than a policy change.<\/p>\n\n\n\n
For most consultancies and managed service providers, the client keeps their own organisation and you get a role. That’s the model the rest of this assumes.<\/p>\n\n\n\n
The shared IAM user with an access key is the default because it works immediately. It keeps working right up until one of three moments: an engineer leaves, a client asks who did something, or a laptop gets stolen.<\/p>\n\n\n\n
Access keys have no expiry. They’re bearer tokens with an indefinite lifetime, and the only way to invalidate one is to go into each account and delete it. At twenty accounts that’s twenty deliberate actions performed under time pressure, and the one you forget is the one that matters. A role session, by contrast, expires on its own. That’s the entire argument.<\/p>\n\n\n\n
A shared password manager still earns its place, just not for admin keys. Bitwarden or 1Password is the right home for per-client external IDs, account numbers, role ARNs and the client-side contacts you need when something breaks at two in the morning. Metadata, not credentials.<\/p>\n\n\n\n
This is the invisible one. The role gets created, the access works, everyone moves on, and the trust policy quietly grants far more than anyone intended.<\/p>\n\n\n\n
Two mistakes travel together. The first is naming your account root in the principal. Writing The second is skipping the external ID. AWS calls the problem it solves the confused deputy: you are a third party that holds the ability to assume roles in many customer accounts, and without a shared secret in the condition, one customer who learns another customer’s role ARN could persuade your systems to act against it. The external ID is the value that makes each trust relationship specific rather than generic.<\/p>\n\n\n\n Generate it yourself, one per client, as a UUID or similar unguessable value. The client must not be the one choosing it, because if they can choose it they can choose someone else’s, and the control does nothing.<\/p>\n\n\n\n The principal names one role, not the account. If that role is compromised you have one thing to fix. If the account is named, you have an inventory problem you didn’t know you had.<\/p>\n\n\n\n You’ve done everything above and a client emails asking who deleted the staging load balancer. You open their CloudTrail and every event attributes to the same assumed role. The role is shared by your whole team by design, so the log tells you your company did it. That’s not what they asked.<\/p>\n\n\n\n There are two knobs and they are not equivalent. The role session name is chosen freely by whoever calls AssumeRole, appears in the assumed-role ARN, and is trivially set to anything. It’s useful, but it’s a label, not evidence. Source identity is the one that carries weight: once set on a session it cannot be changed, it persists across chained role sessions, and it lands in CloudTrail in the session context of every subsequent call.<\/p>\n\n\n\n The catch is that the role’s trust policy has to permit If your engineers arrive through SAML federation, the identity provider can populate source identity from a directory attribute such as email, which means nobody has to remember to pass it. For direct CLI use it’s an explicit flag:<\/p>\n\n\n\n Getting this right is what turns “we take security seriously” into a sentence you can back with a log query. It is also, in my experience of what clients actually ask for, the single most persuasive thing on the list.<\/p>\n\n\n\n A hub account is a concentration of risk. That’s the trade you’re making, and it’s worth making, but only if the hub is held to a higher standard than the accounts it reaches into.<\/p>\n\n\n\n Practically: not every engineer gets permission to assume every client’s role. Group permissions in the hub so the people on the Acme engagement reach Acme and nothing else. Use per-client role names rather than one shared name across all twenty, so a policy in the hub can’t wildcard its way into a client you stopped working with a year ago.<\/p>\n\n\n\n Tier the roles by what they’re for, not by who’s asking. A read-only role for investigation covers most of the work most days. A change role carries the write permissions the engagement actually needs. Break-glass sits separate, with a short maximum session duration and an alarm on every use.<\/p>\n\n\n\n Two extras are worth knowing about even if you don’t reach for them immediately. Session policies let the caller narrow a session further at assume time, which is useful for automation that only needs one bucket. Permissions boundaries cap what any role or user created inside the client account can do, which matters when part of your job is creating IAM there.<\/p>\n\n\n\n If your team already routes through a business VPN or zero trust gateway with stable egress addresses, a source IP condition on the trust policy is a cheap extra layer. NordLayer, Tailscale and Cloudflare’s zero trust products all give you predictable egress. Be honest about the cost, though: the moment something runs from CI, a client’s network or a laptop on hotel wifi, an IP condition becomes a support ticket. I’d add it once everything else works, not before.<\/p>\n\n\n\n Somewhere in the twenty accounts there is a role with AdministratorAccess attached, created during a migration, described in the ticket as temporary. It’s been eighteen months.<\/p>\n\n\n\n Here’s the honest part: AWS does not give you a native just-in-time approval workflow for cross-account role assumption. There’s no built-in “request four hours of write access, get approved in Slack, access expires automatically” for this pattern. You either build that on top of STS and your ticketing system, or you accept standing access and compensate with tight session durations, aggressive monitoring and a genuine periodic review. Both are defensible. Pretending you have JIT when you have a wiki page is not.<\/p>\n\n\n\n The cheap version that actually helps: set the maximum session duration on client roles to something short, alert on assumption of the break-glass role, and run a quarterly pass over service last accessed data to strip permissions nobody has used. IAM can generate that report per role.<\/p>\n\n\n\n Run that in each client account and the conversation about narrowing permissions stops being theoretical. You’re not guessing what the engagement needs; you’re reading it.<\/p>\n\n\n\n The whole point of the hub is that offboarding is one action. Disable the person in your directory and every path into every client account closes at once, because there was never a credential in a client account with their name on it.<\/p>\n\n\n\n That’s true only if nothing crept in sideways. Things that creep in sideways:<\/p>\n\n\n\n IAM Access Analyzer is the tool that finds most of the first four. Its external access findings show every principal outside the account that can reach into it, which is exactly the list a client wants to see, and it’s also the list that reveals the monitoring vendor nobody remembers signing up for. For the laptop, whatever your hardware disposal process is, make sure it involves an actual wipe rather than a factory reset; tools like O&O SafeErase exist for that reason and are cheap next to the alternative.<\/p>\n\n\n\n Doing this by hand twenty times guarantees twenty subtly different configurations. Make it a template and a checklist.<\/p>\n\n\n\n The generated profile chains from your federated session into the client role. The AWS CLI reads these from the shared config file and handles the assume-role call for you:<\/p>\n\n\n\n Then the two-second sanity check that has saved me from more mistakes than any other command:<\/p>\n\n\n\n It returns the account ID and the assumed-role ARN. Read the account number before you run anything destructive. Twenty profiles in one config file is exactly the situation where muscle memory picks the wrong one.<\/p>\n\n\n\n For scheduled work that runs without a human, the same trust relationship serves a Lambda function or an EC2 instance role in your hub. A low-cost VPS from Contabo or InterServer will also do the job, but note the trade: an instance profile inside AWS gets rotated credentials automatically, while a box outside AWS needs a long-lived credential to bootstrap from, which reintroduces exactly what this design removes. Inside AWS is the cleaner answer.<\/p>\n\n\n\n Three separate causes produce nearly the same message, and they need different fixes. Either the client’s trust policy doesn’t name your principal, or the external ID you sent doesn’t match what the condition expects, or your own identity in the hub lacks permission to call AssumeRole on that role ARN. Check them in that order, because the first two require the client and the third doesn’t.<\/p>\n\n\n\n This one wastes an afternoon if you don’t know it. The console’s switch role feature does not support roles that require an external ID. It’s a documented limitation, not a bug in your policy. Engineers who need console access to those accounts should reach them through federation rather than the switch role box.<\/p>\n\n\n\n When one role session assumes another role, that’s role chaining, and a chained session is capped at one hour regardless of the role’s maximum session duration setting. Since your engineers assume the client role from an already-assumed session in the hub, you’re chaining by definition. Plan around the hour rather than fighting it.<\/p>\n\n\n\n Some denials come back with a long encoded blob attached. STS will decode it if your identity has permission to, and the decoded version usually names the exact statement that denied you.<\/p>\n\n\n\n You can, and it’ll work on day one. The problem is day four hundred. That user’s access key never expires, it isn’t attached to any individual on your team, and revoking it means someone in the client’s organisation remembering to act. A role costs the client the same five minutes to create and hands them a control they can exercise unilaterally.<\/p>\n\n\n\n Yes. A shared external ID across your client base defeats the purpose, because the value’s entire job is to be specific to one trust relationship. Generate a UUID per client, treat it as a secret, and never let the client choose it.<\/p>\n\n\n\n Only if you’re also taking on their billing and the commercial relationship supports it. Inside your organisation you get service control policies, resource control policies and centralised root access management, which are real advantages. Outside it, those stay the client’s tools and your job is to recommend them well. Most consulting engagements sit outside.<\/p>\n\n\n\n Show the directory record disabling them, then show that no IAM user or access key in the client account carries their name, then show Access Analyzer external access findings listing only your operations role as an external principal. Three artefacts, all exportable. That’s the evidence pack.<\/p>\n\n\n\n Don’t assume it does. The reliable place to enforce MFA is where the human authenticates in your identity plane, and the reliable control at the client boundary is the combination of a narrowly scoped principal and the external ID condition. If you want an MFA condition in the client’s trust policy as well, test it end to end against your actual federation path before you rely on it, because the request context differs between direct assumption and chained sessions.<\/p>\n\n\n\n A hub account with federated engineers, one role per client scoped to a named principal with a unique external ID, and source identity required. Skip the tiering, skip the IP conditions, skip the JIT ambitions. Those three things give you revocation, attribution and a clean answer for the auditor, which is most of the value.<\/p>\n\n\n\n Client AWS account access is not really a permissions problem. It’s a revocation and attribution problem wearing a permissions costume. Any design where a credential exists that isn’t tied to a specific human will fail you at exactly the wrong moment, and no amount of policy tuning fixes that, because the gap isn’t in the policy.<\/p>\n\n\n\n Put the humans in one place you control. Put a narrowly scoped, externally identified role in each client account. Require source identity so the log names a person. Everything else on this page is refinement; those three are the structure. And the pleasant side effect is that “here’s how you revoke us, unilaterally, in one action” is a genuinely good thing to be able to say in a sales conversation.<\/p>\n\n\n\n This is the kind of work I do for consultancies and managed service providers who’ve grown past the point where a shared vault is survivable. Specifically:<\/p>\n\n\n\n If you want a concrete starting point, send me one trust policy and the AccessDenied message it produces. That’s usually enough to tell where the model is leaking.<\/p>\n\n\n\narn:aws:iam::111122223333:root<\/code> does not mean “the root user of my account.” It means any principal in that account that has permission to call AssumeRole. Every engineer, every Lambda function, every CI runner, every future role you haven’t created yet. It delegates the decision to your own account’s IAM, which is fine when your IAM is tidy and quietly catastrophic when it isn’t.<\/p>\n\n\n\n{\n \"Version\": \"2012-10-17\",\n \"Statement\": [\n {\n \"Effect\": \"Allow\",\n \"Principal\": {\n \"AWS\": \"arn:aws:iam::111122223333:role\/MspOperationsAccess\"\n },\n \"Action\": \"sts:AssumeRole\",\n \"Condition\": {\n \"StringEquals\": {\n \"sts:ExternalId\": \"6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02\"\n }\n }\n }\n ]\n}<\/code><\/pre>\n\n\n\nFailure family 3: CloudTrail says the role, not the human<\/h2>\n\n\n\n
sts:SetSourceIdentity<\/code> alongside sts:AssumeRole<\/code>, and you can require it with a condition so a session without one is refused outright.<\/p>\n\n\n\n\"Action\": [\n \"sts:AssumeRole\",\n \"sts:SetSourceIdentity\"\n],\n\"Condition\": {\n \"StringEquals\": {\n \"sts:ExternalId\": \"6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02\"\n },\n \"StringLike\": {\n \"sts:SourceIdentity\": \"*@yourcompany.com\"\n }\n}<\/code><\/pre>\n\n\n\naws sts assume-role \n --role-arn arn:aws:iam::444455556666:role\/MspEngineerAccess \n --role-session-name acme-investigation \n --external-id 6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02 \n --source-identity jane.doe@yourcompany.com<\/code><\/pre>\n\n\n\nFailure family 4: one identity, twenty blast radii<\/h2>\n\n\n\n
Failure family 5: standing administrator access nobody remembers granting<\/h2>\n\n\n\n
# Kick off the report, then fetch it with the returned job ID\naws iam generate-service-last-accessed-details \n --arn arn:aws:iam::444455556666:role\/MspEngineerAccess\n\naws iam get-service-last-accessed-details --job-id JOB_ID_FROM_ABOVE<\/code><\/pre>\n\n\n\nFailure family 6: offboarding that only half happens<\/h2>\n\n\n\n
\n
Rolling client AWS account access out across twenty accounts<\/h2>\n\n\n\n
\n
# ~\/.aws\/config\n[profile msp-hub]\nsso_session = company\nsso_account_id = 111122223333\nsso_role_name = MspOperationsAccess\nregion = eu-west-1\n\n[profile acme-readonly]\nrole_arn = arn:aws:iam::444455556666:role\/MspEngineerReadOnly\nsource_profile = msp-hub\nexternal_id = 6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02\nrole_session_name = jane-doe\nduration_seconds = 3600\nregion = eu-west-1<\/code><\/pre>\n\n\n\naws sts get-caller-identity --profile acme-readonly<\/code><\/pre>\n\n\n\n
\n\n\n\nTroubleshooting the failures you will actually hit<\/h2>\n\n\n\n
AccessDenied on AssumeRole<\/h3>\n\n\n\n
Switch Role in the console silently fails<\/h3>\n\n\n\n
Sessions expiring after an hour when you asked for longer<\/h3>\n\n\n\n
An authorization message you can’t read<\/h3>\n\n\n\n
aws sts decode-authorization-message --encoded-message <encoded-blob><\/code><\/pre>\n\n\n\nCommon mistakes<\/h2>\n\n\n\n
\n
Best practices worth the effort<\/h2>\n\n\n\n
\n
FAQ<\/h2>\n\n\n\n
Can’t I just ask each client to create an IAM user for me?<\/h3>\n\n\n\n
Do I need a different external ID for every client?<\/h3>\n\n\n\n
Should I bring client accounts into my own AWS Organization?<\/h3>\n\n\n\n
How do I prove a departed engineer no longer has access?<\/h3>\n\n\n\n
Does MFA carry across the cross-account role assumption?<\/h3>\n\n\n\n
What’s the minimum viable version if I only have three clients?<\/h3>\n\n\n\n
The one thing to take away<\/h2>\n\n\n\n
\n\n\n\nNeed a second pair of eyes on your client access model?<\/h2>\n\n\n\n
\n