They probably never will, and that is not a criticism of the product. SageMaker Unified Studio is a front door. It is not a runtime, it is not a billing boundary, and it does not replace your IAM design. Teams who adopt it expecting consolidation do get consolidation, but only of attention. Everything underneath stays where it was, on the same services, on the same bill, behind the same permission model.

This post covers what it genuinely replaces, what it leaves untouched no matter how the migration deck was worded, and what it quietly adds to your operational surface that you will own indefinitely. If you are signing off on an adoption plan, the third part is the one that matters.

SageMaker Unified Studio is a front door, not a runtime

Start here, because every surprise downstream follows from this. When you run a query in the SQL editor, Athena or Redshift runs it. When you build a visual ETL job, Glue runs it. When you train a model, SageMaker AI runs it. When you orchestrate something, Airflow runs it. The studio decides who may open which tool, inside which project, against which data. It executes nothing itself.

AWS is explicit about this in its access control documentation: opening a project is equivalent to signing into an account in a specific Region and assuming one of that project’s IAM roles. The tools you then see are the tools that role can reach. There is no new execution layer. There is a new authorisation layer, and a new set of nouns to describe it. Hold that model and the rest is predictable.

What it actually replaces

The pile of standalone studios and query editors

This is the real win, worth stating plainly before the caveats start. AWS assembled the studio by pulling together the separate studios, query editors and visual tools that previously lived inside Athena, EMR, Glue, Redshift, Managed Workflows for Apache Airflow and the existing SageMaker Studio, then added a Bedrock-backed environment for generative AI work.

The problem it solves is not tab count. It is context loss. A data engineer moving from a Glue job, to the table it wrote, to the notebook consuming that table crossed three consoles, three permission surfaces and three mental models of what a “workspace” meant. That handoff is where mistakes lived. Collapsing it into one project boundary is the reason to adopt at all.

Per-team domain sprawl

The older pattern for isolating business units was a SageMaker domain each, then reconciling identity, networking and tagging across all of them by hand. Projects replace most of that. A project is a multi-user boundary with its own roles, storage and members, and it can span accounts, which the old shared-space model could not do cleanly.

The access request spreadsheet

There are two supported ways to give a project access to data: attach IAM permissions directly to the project role, or publish the asset to the catalog and let members subscribe through a request and approval flow.

The second one is the point of the product. If you are only ever going to use the first, you have bought a portal and skipped the governance engine you are now paying to run.

What it doesn’t replace

Your bill, or your ability to explain it

There is no standalone charge for the studio itself. AWS bills you for each underlying service you reach through it, plus usage of SageMaker Catalog, which is metered separately on metadata ingestion and related operations. Nothing consolidates.

What changes, and this is the part that bites, is attribution. Before, an engineer created a Glue job and it inherited whatever tags your pipeline applied. Now a project profile deploys a bundle of resources on that engineer’s behalf, and unless you have deliberately configured project-level custom tags, those resources land in your cost report under a project name that means nothing to finance.

Sort this out before migration. Decide your tag keys, wire them into the project configuration, and confirm they appear in Cost Explorer and the Cost and Usage Report before you onboard a second team. If you run a cost platform such as CloudZero or Vantage on top of the CUR, check that your allocation rules cope with resources whose creator is a service role rather than a person, because that is the shape everything arrives in now.

Your IAM and Lake Formation design

Project roles are ordinary IAM roles. Lake Formation grants are ordinary Lake Formation grants. Bucket policies still apply. If your data access is broken at any of those layers, the studio will surface the same denial you always got, with slightly less context about which principal was being used at the time.

Nobody in the adoption meeting will say “this fixes our permissions mess.” Plenty of people will quietly assume it. Say it out loud early.

Your infrastructure as code

This one catches DevOps teams hardest. AWS ships an open-source CI/CD CLI for the studio that promotes applications across dev, test and production by mapping each pipeline stage to an isolated project and substituting stage-specific values at deploy time. It is a good tool and it slots into GitHub Actions, GitLab CI or Jenkins without much argument.

It also does not create domains or projects. The documentation says so directly and points you at CloudFormation, CDK or Terraform for the infrastructure underneath, and the same limitations page notes that native notebook deployment is not covered. The substrate is still yours to build and version. You get an application deployment tool, not a platform provisioning tool. Plan for both.

SageMaker AI

The original SageMaker was renamed SageMaker AI and remains available on its own. If your team’s work is model training and inference and nothing else, you are allowed to keep using it directly and skip the governance layer entirely. That is a supported choice, not a holdout position.

What SageMaker Unified Studio quietly adds

Here is the section that rarely makes it into the migration plan.

You now operate DataZone, intentionally or not

SageMaker Catalog is built on Amazon DataZone, and the governance model comes with it: domains, domain units, projects, project profiles, blueprints, environments, glossaries, metadata forms, subscription requests. Domain identifiers still carry a dzd prefix, and the console for creating one still lives under the DataZone console path.

You can see the shape of what you own from the CLI, which is worth doing on day one. These commands read the DataZone control plane sitting behind the studio:

# Every domain in the account and Region, with its status
aws datazone list-domains 
  --query 'items[].{id:id,name:name,status:status}' 
  --output table

# Projects inside one domain (domain IDs look like dzd_xxxxxxxx)
aws datazone list-projects 
  --domain-identifier dzd_EXAMPLE 
  --output table

# The project profiles an admin has published, which is the
# real menu of what users can create
aws datazone list-project-profiles 
  --domain-identifier dzd_EXAMPLE 
  --output table

# The environments actually deployed under one project
aws datazone list-environments 
  --domain-identifier dzd_EXAMPLE 
  --project-identifier PROJECT_ID

Run these monthly against a domain someone else administers and you will learn more about your real governance posture than any dashboard tells you.

Project creation is a CloudFormation deploy wearing a friendly hat

Blueprints are backed by CloudFormation templates. Project profiles bundle blueprint configurations, and choosing a profile triggers a stack deployment that provisions the environment behind the project.

This is the invisible failure mode. To a user, project creation is a form and a spinner. To you, it is a stack that can roll back on a missing permission, an unavailable subnet, a service quota or a name collision. When somebody reports that project creation “just doesn’t work,” do not reproduce it in the UI. Read the stack events:

# Find recently failed or rolled-back stacks
aws cloudformation list-stacks 
  --stack-status-filter CREATE_FAILED ROLLBACK_COMPLETE ROLLBACK_IN_PROGRESS 
  --query 'StackSummaries[].{name:StackName,status:StackStatus}' 
  --output table

# Then read the actual reason, which is almost always
# in the first CREATE_FAILED event, not the last
aws cloudformation describe-stack-events 
  --stack-name STACK_NAME 
  --query 'StackEvents[?ResourceStatus==`CREATE_FAILED`].[LogicalResourceId,ResourceStatusReason]' 
  --output table

The first failed resource causes the rollback. Everything after it is noise. This one habit turns most “the platform is broken” tickets into five-minute answers.

A domain-type fork you will live with for a long time

Domains can authenticate through IAM or through IAM Identity Center. AWS positions the Identity Center configuration as the governance-heavy option, preserving individual user identity through the session and supporting the formal publish and subscribe approval workflow with identity-based audit trails. The IAM configuration is positioned toward lighter-weight developer productivity.

Parity between the two has moved over time and is still moving. Before committing an organisation to one, check the current documentation for the specific capabilities you care about, in the Region you are deploying to. Do not take a blog post’s word for it, including this one. This is the decision that is genuinely expensive to reverse, because it is entangled with how every user signs in.

Where adoption actually goes wrong

“It works in my console but not in the project”

This is the single most common support ticket, and it is almost never a bug. The engineer tested with their own federated role, which has broad permissions, then opened a project running under a narrow project role. Two different principals, two different answers.

Confirm which principal you are before debugging anything else. Run this from a terminal inside the project environment, not from your laptop:

aws sts get-caller-identity

If the ARN is not the one you assumed, stop. You have found the problem.

The bill went up after a migration that was supposed to save money

Usually because the migration created things rather than moving them. New project environments provision new resources, catalog metadata ingestion is metered, and per-project orchestration environments are not free just because nobody chose them explicitly. Compare like for like by tag rather than by service total, and give it a full billing cycle before drawing conclusions.

Nobody can find the data that is definitely there

Existing in a Glue database is not the same as being published to the catalog. If an asset was never published, it will not appear in search, and no amount of IAM permission will make it discoverable. Check publication state before you check permissions.

Common mistakes

• Treating it as a migration rather than an addition. Nothing is decommissioned. Your Glue jobs, Athena workgroups and Redshift clusters stay where they were.
• Letting users self-serve projects on day one. Every project is a stack deploy and a set of billable resources. Gate it until profiles and tagging are settled.
• Choosing the domain authentication type in a hurry. It is the hardest thing here to change later.
• Skipping the catalog. Grant everything through direct IAM attachments and you carry the governance overhead with none of the benefit.
• Assuming the CI/CD CLI provisions your platform. It deploys applications into projects that already exist.
• Not tagging project resources from the start. Retrofitting attribution across dozens of blueprint-created resources is miserable work.

How I’d decide

A rough order of operations for anyone weighing this up:

1. Ask what problem you are solving. If it is “our teams cannot discover or safely share each other’s data,” this is the right tool. If it is “we have too many consoles,” it is an expensive way to buy fewer bookmarks.
2. Count your teams. The value curve is steep with cross-team sharing and nearly flat for one team in its own account. One team, one account, mostly model training? Stay on SageMaker AI.
3. Decide the authentication model first. Identity Center for per-user audit trails and approval workflows. IAM if you are optimising for a small group moving fast and accept a different feature set.
4. Write the project profiles yourself. The defaults are demonstrations. Production profiles should encode your networking, tags and quotas.
5. Prove the teardown path before the build path. Create a project, use it, delete it, confirm nothing is orphaned. Do this before any real team touches it.
6. Keep the old consoles. They are your escape hatch when the abstraction hides something you need to see, and cost nothing to leave enabled.

One more piece of judgement: wire your existing observability in from the start rather than relying on the studio’s own views. CloudWatch, Amazon Managed Grafana or a Grafana Cloud stack you already run, it does not much matter which. The failures that matter here are stack failures and job failures, and both are visible in the places you already watch.

Frequently asked questions

Does SageMaker Unified Studio replace SageMaker Studio?

It supersedes it as the recommended experience, and AWS provides an upgrade path that imports existing domains, user profiles and spaces without redeploying infrastructure. JupyterLab and Code Editor spaces, training jobs, pipelines, models and endpoints are preserved, and administrators control when access to the older experience is turned off. It is a migration you schedule, not one that happens to you.

Do I still need Amazon DataZone separately?

Not as a separate purchase. The catalog is built on DataZone, and existing DataZone domains can be upgraded into unified domains, which requires establishing the domain execution and service roles the new model expects. Functionally you are already running DataZone; the studio is the interface on top of it.

How much does SageMaker Unified Studio cost?

The interface has no standalone charge. You pay for each AWS service you use through it on that service’s normal terms, plus SageMaker Catalog usage, metered on metadata ingestion and related operations. Model it by listing what a project profile actually provisions and applying current rates to each item, rather than hunting for a per-seat number that does not exist.

Can I manage domains and projects with Terraform?

AWS’s own documentation directs you to CloudFormation, CDK or Terraform for creating the domain and project substrate, since the CI/CD CLI deliberately does not do it. Coverage in each tool tracks the underlying API, so check the current provider or construct documentation for the specific resources you need before designing around it.

What happens to my Glue jobs and Athena queries?

Nothing. They keep running as Glue jobs and Athena queries. The studio gives you a different place to author and run them and a governance layer that decides who may do so. The execution, the quotas and the billing are unchanged.

Is it worth adopting for a small team?

Often not. The governance machinery is the product, and a team of three sharing one account has little to govern. The threshold is roughly where you have more than one team, data one team owns that another needs, and a real answer required to who approved that access.

Can I run both the old and new experiences at once?

Yes, and you should during transition. The upgrade path lets teams keep access to both while they adapt, with the administrator deciding when to close the older door. Running in parallel for a full billing cycle is also the cleanest way to see what the change actually costs.

The one thing worth remembering

SageMaker Unified Studio replaces an interface problem. It does not replace an infrastructure problem, a permissions problem, or a cost attribution problem, and it adds a governance system you are now responsible for operating.

That trade is worth it when you have data crossing team boundaries and no credible answer to who approved access to it. It is not worth it when the pitch was “fewer tabs.” The teams who are happy a year in went in knowing they were buying a governance engine and budgeted the operational time to run it. The unhappy ones expected a consolidation that was never on offer.

Ask which one you are before the first project gets created, not after the fortieth.

Need a second opinion on your adoption plan?

This is a decision that is cheap to get right at the start and expensive to unwind two quarters later. Things I can help with directly:

• Reviewing a domain design before rollout, especially the authentication model choice and how it meets your existing identity setup
• Writing production project profiles and blueprints that encode your VPC layout, tagging standard and quotas instead of the defaults
• Building the cost attribution layer so project-created resources land in reporting with tags finance can actually use
• Wiring the CI/CD CLI into an existing GitHub Actions, GitLab CI or Jenkins pipeline, plus the CloudFormation or Terraform layer underneath that the CLI does not cover
• Diagnosing failed project creations and access denials that trace back to project roles or Lake Formation grants
• Sanity-checking a migration plan from SageMaker Studio or an existing DataZone domain, including what runs in parallel and for how long

Send me a project profile, a failed stack event log, or a tagged cost export and I will tell you what I see.

The post SageMaker Unified Studio: What It Replaces, and What It Quietly Doesn’t appeared first on John Nessime.

That question is what actually bites when you run infrastructure for twenty clients. Not a breach. An audit, or a resignation. Someone gives notice on a Friday and you spend Monday working out which accounts they still hold keys to, because the credential everybody shared was never attached to a person. You can’t revoke a person from a password. You can only rotate it and hope you got all twenty.

This post covers how to design client AWS account access so that humans authenticate once against something you control, every client account holds a role that trusts exactly one thing, and every API call in a client’s CloudTrail can be traced back to a named engineer. It’s organised by failure family, because the mechanics are well documented and the failure modes are not.

The shape: one identity plane, twenty separate trust decisions

The instinct that causes all the trouble is putting credentials in the middle. Client gives you a login, you store it somewhere shared, engineers take it out when they need it. Everything downstream of that decision is damage control.

The shape that works is boring. One AWS account whose only job is identity. Your engineers exist there, once, in IAM Identity Center or federated from whatever directory you already pay for. Each client account holds an IAM role whose trust policy names one principal in that account and requires an external ID. Nobody stores a client credential, because there is no client credential. There’s a role your engineers can ask STS for a short-lived session against, and a client who can delete that role whenever they like.

There is a second design worth knowing about: inviting client accounts into your own AWS Organization. It’s genuinely better in some ways. You get service control policies, resource control policies, and centralised root access management across the estate, all of which are organisation-scoped features you simply cannot use on an account you don’t own. It’s also a much bigger commitment. You take on consolidated billing, the client’s spend flows through you, and pulling an account back out of an organisation at the end of a contract is a real project rather than a policy change.

For most consultancies and managed service providers, the client keeps their own organisation and you get a role. That’s the model the rest of this assumes.

Failure family 1: credentials you can’t revoke

The shared IAM user with an access key is the default because it works immediately. It keeps working right up until one of three moments: an engineer leaves, a client asks who did something, or a laptop gets stolen.

Access keys have no expiry. They’re bearer tokens with an indefinite lifetime, and the only way to invalidate one is to go into each account and delete it. At twenty accounts that’s twenty deliberate actions performed under time pressure, and the one you forget is the one that matters. A role session, by contrast, expires on its own. That’s the entire argument.

A shared password manager still earns its place, just not for admin keys. Bitwarden or 1Password is the right home for per-client external IDs, account numbers, role ARNs and the client-side contacts you need when something breaks at two in the morning. Metadata, not credentials.

Failure family 2: a trust policy that trusts more than you meant

This is the invisible one. The role gets created, the access works, everyone moves on, and the trust policy quietly grants far more than anyone intended.

Two mistakes travel together. The first is naming your account root in the principal. Writing arn:aws:iam::111122223333:root does not mean “the root user of my account.” It means any principal in that account that has permission to call AssumeRole. Every engineer, every Lambda function, every CI runner, every future role you haven’t created yet. It delegates the decision to your own account’s IAM, which is fine when your IAM is tidy and quietly catastrophic when it isn’t.

The second is skipping the external ID. AWS calls the problem it solves the confused deputy: you are a third party that holds the ability to assume roles in many customer accounts, and without a shared secret in the condition, one customer who learns another customer’s role ARN could persuade your systems to act against it. The external ID is the value that makes each trust relationship specific rather than generic.

Generate it yourself, one per client, as a UUID or similar unguessable value. The client must not be the one choosing it, because if they can choose it they can choose someone else’s, and the control does nothing.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111122223333:role/MspOperationsAccess"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02"
        }
      }
    }
  ]
}

The principal names one role, not the account. If that role is compromised you have one thing to fix. If the account is named, you have an inventory problem you didn’t know you had.

Failure family 3: CloudTrail says the role, not the human

You’ve done everything above and a client emails asking who deleted the staging load balancer. You open their CloudTrail and every event attributes to the same assumed role. The role is shared by your whole team by design, so the log tells you your company did it. That’s not what they asked.

There are two knobs and they are not equivalent. The role session name is chosen freely by whoever calls AssumeRole, appears in the assumed-role ARN, and is trivially set to anything. It’s useful, but it’s a label, not evidence. Source identity is the one that carries weight: once set on a session it cannot be changed, it persists across chained role sessions, and it lands in CloudTrail in the session context of every subsequent call.

The catch is that the role’s trust policy has to permit sts:SetSourceIdentity alongside sts:AssumeRole, and you can require it with a condition so a session without one is refused outright.

"Action": [
  "sts:AssumeRole",
  "sts:SetSourceIdentity"
],
"Condition": {
  "StringEquals": {
    "sts:ExternalId": "6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02"
  },
  "StringLike": {
    "sts:SourceIdentity": "*@yourcompany.com"
  }
}

If your engineers arrive through SAML federation, the identity provider can populate source identity from a directory attribute such as email, which means nobody has to remember to pass it. For direct CLI use it’s an explicit flag:

aws sts assume-role 
  --role-arn arn:aws:iam::444455556666:role/MspEngineerAccess 
  --role-session-name acme-investigation 
  --external-id 6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02 
  --source-identity jane.doe@yourcompany.com

Getting this right is what turns “we take security seriously” into a sentence you can back with a log query. It is also, in my experience of what clients actually ask for, the single most persuasive thing on the list.

Failure family 4: one identity, twenty blast radii

A hub account is a concentration of risk. That’s the trade you’re making, and it’s worth making, but only if the hub is held to a higher standard than the accounts it reaches into.

Practically: not every engineer gets permission to assume every client’s role. Group permissions in the hub so the people on the Acme engagement reach Acme and nothing else. Use per-client role names rather than one shared name across all twenty, so a policy in the hub can’t wildcard its way into a client you stopped working with a year ago.

Tier the roles by what they’re for, not by who’s asking. A read-only role for investigation covers most of the work most days. A change role carries the write permissions the engagement actually needs. Break-glass sits separate, with a short maximum session duration and an alarm on every use.

Two extras are worth knowing about even if you don’t reach for them immediately. Session policies let the caller narrow a session further at assume time, which is useful for automation that only needs one bucket. Permissions boundaries cap what any role or user created inside the client account can do, which matters when part of your job is creating IAM there.

If your team already routes through a business VPN or zero trust gateway with stable egress addresses, a source IP condition on the trust policy is a cheap extra layer. NordLayer, Tailscale and Cloudflare’s zero trust products all give you predictable egress. Be honest about the cost, though: the moment something runs from CI, a client’s network or a laptop on hotel wifi, an IP condition becomes a support ticket. I’d add it once everything else works, not before.

Failure family 5: standing administrator access nobody remembers granting

Somewhere in the twenty accounts there is a role with AdministratorAccess attached, created during a migration, described in the ticket as temporary. It’s been eighteen months.

Here’s the honest part: AWS does not give you a native just-in-time approval workflow for cross-account role assumption. There’s no built-in “request four hours of write access, get approved in Slack, access expires automatically” for this pattern. You either build that on top of STS and your ticketing system, or you accept standing access and compensate with tight session durations, aggressive monitoring and a genuine periodic review. Both are defensible. Pretending you have JIT when you have a wiki page is not.

The cheap version that actually helps: set the maximum session duration on client roles to something short, alert on assumption of the break-glass role, and run a quarterly pass over service last accessed data to strip permissions nobody has used. IAM can generate that report per role.

# Kick off the report, then fetch it with the returned job ID
aws iam generate-service-last-accessed-details 
  --arn arn:aws:iam::444455556666:role/MspEngineerAccess

aws iam get-service-last-accessed-details --job-id JOB_ID_FROM_ABOVE

Run that in each client account and the conversation about narrowing permissions stops being theoretical. You’re not guessing what the engagement needs; you’re reading it.

Failure family 6: offboarding that only half happens

The whole point of the hub is that offboarding is one action. Disable the person in your directory and every path into every client account closes at once, because there was never a credential in a client account with their name on it.

That’s true only if nothing crept in sideways. Things that creep in sideways:

• IAM users created directly in a client account during an incident, because it was faster
• Access keys minted under a role by automation and never inventoried
• A client’s own console user shared with your team “just for the migration”
• Third party tools connected with their own cross-account roles that outlived the trial
• A departing engineer’s laptop with cached SSO tokens and stale profiles still on disk

IAM Access Analyzer is the tool that finds most of the first four. Its external access findings show every principal outside the account that can reach into it, which is exactly the list a client wants to see, and it’s also the list that reveals the monitoring vendor nobody remembers signing up for. For the laptop, whatever your hardware disposal process is, make sure it involves an actual wipe rather than a factory reset; tools like O&O SafeErase exist for that reason and are cheap next to the alternative.

Rolling client AWS account access out across twenty accounts

Doing this by hand twenty times guarantees twenty subtly different configurations. Make it a template and a checklist.

1. Write one CloudFormation template that creates the role, its trust policy and its permissions, parameterised by external ID and your operations account ID.
2. Generate a unique external ID per client and store it with the client’s account metadata, not in the template.
3. Send the client a launch stack link rather than asking for credentials. They deploy it in their own account, in their own console, and can read exactly what they’re granting.
4. Record the account ID, role ARN, external ID location, engagement scope and review date in a single inventory file in version control.
5. Generate everyone’s local AWS profiles from that inventory so nobody hand-edits a config file.
6. Verify each one with a call that proves identity and nothing else before you touch anything real.

The generated profile chains from your federated session into the client role. The AWS CLI reads these from the shared config file and handles the assume-role call for you:

# ~/.aws/config
[profile msp-hub]
sso_session = company
sso_account_id = 111122223333
sso_role_name = MspOperationsAccess
region = eu-west-1

[profile acme-readonly]
role_arn = arn:aws:iam::444455556666:role/MspEngineerReadOnly
source_profile = msp-hub
external_id = 6f1c2e94-3a77-4c2b-9f0d-8e5a1b7c4d02
role_session_name = jane-doe
duration_seconds = 3600
region = eu-west-1

Then the two-second sanity check that has saved me from more mistakes than any other command:

aws sts get-caller-identity --profile acme-readonly

It returns the account ID and the assumed-role ARN. Read the account number before you run anything destructive. Twenty profiles in one config file is exactly the situation where muscle memory picks the wrong one.

For scheduled work that runs without a human, the same trust relationship serves a Lambda function or an EC2 instance role in your hub. A low-cost VPS from Contabo or InterServer will also do the job, but note the trade: an instance profile inside AWS gets rotated credentials automatically, while a box outside AWS needs a long-lived credential to bootstrap from, which reintroduces exactly what this design removes. Inside AWS is the cleaner answer.

Troubleshooting the failures you will actually hit

AccessDenied on AssumeRole

Three separate causes produce nearly the same message, and they need different fixes. Either the client’s trust policy doesn’t name your principal, or the external ID you sent doesn’t match what the condition expects, or your own identity in the hub lacks permission to call AssumeRole on that role ARN. Check them in that order, because the first two require the client and the third doesn’t.

Switch Role in the console silently fails

This one wastes an afternoon if you don’t know it. The console’s switch role feature does not support roles that require an external ID. It’s a documented limitation, not a bug in your policy. Engineers who need console access to those accounts should reach them through federation rather than the switch role box.

Sessions expiring after an hour when you asked for longer

When one role session assumes another role, that’s role chaining, and a chained session is capped at one hour regardless of the role’s maximum session duration setting. Since your engineers assume the client role from an already-assumed session in the hub, you’re chaining by definition. Plan around the hour rather than fighting it.

An authorization message you can’t read

Some denials come back with a long encoded blob attached. STS will decode it if your identity has permission to, and the decoded version usually names the exact statement that denied you.

aws sts decode-authorization-message --encoded-message <encoded-blob>

Common mistakes

• Naming the account root as the trust principal instead of the specific operations role
• Letting the client pick the external ID, or reusing one value across several clients
• Treating the role session name as an audit trail when it’s a caller-supplied string
• Asking a client for console credentials because a template felt like too much effort
• Granting AdministratorAccess for a migration and never revisiting it
• Building the hub properly and then leaving one legacy IAM user in one account as a fallback
• Assuming that removing someone from your directory also removed the keys they created by hand

Best practices worth the effort

• One identity plane, federated, with MFA enforced at the point the human authenticates
• A unique unguessable external ID per client, generated by you and stored with client metadata
• Source identity required in the trust policy so CloudTrail names a person, not a role
• Roles tiered by purpose: read-only for investigation, change for the engagement, break-glass alarmed
• Client-owned policies, deployed by the client from your template, revocable by the client in one action
• A version-controlled inventory of every account, role, scope and review date
• Access Analyzer external access findings reviewed on a schedule, in every client account

FAQ

Can’t I just ask each client to create an IAM user for me?

You can, and it’ll work on day one. The problem is day four hundred. That user’s access key never expires, it isn’t attached to any individual on your team, and revoking it means someone in the client’s organisation remembering to act. A role costs the client the same five minutes to create and hands them a control they can exercise unilaterally.

Do I need a different external ID for every client?

Yes. A shared external ID across your client base defeats the purpose, because the value’s entire job is to be specific to one trust relationship. Generate a UUID per client, treat it as a secret, and never let the client choose it.

Should I bring client accounts into my own AWS Organization?

Only if you’re also taking on their billing and the commercial relationship supports it. Inside your organisation you get service control policies, resource control policies and centralised root access management, which are real advantages. Outside it, those stay the client’s tools and your job is to recommend them well. Most consulting engagements sit outside.

How do I prove a departed engineer no longer has access?

Show the directory record disabling them, then show that no IAM user or access key in the client account carries their name, then show Access Analyzer external access findings listing only your operations role as an external principal. Three artefacts, all exportable. That’s the evidence pack.

Does MFA carry across the cross-account role assumption?

Don’t assume it does. The reliable place to enforce MFA is where the human authenticates in your identity plane, and the reliable control at the client boundary is the combination of a narrowly scoped principal and the external ID condition. If you want an MFA condition in the client’s trust policy as well, test it end to end against your actual federation path before you rely on it, because the request context differs between direct assumption and chained sessions.

What’s the minimum viable version if I only have three clients?

A hub account with federated engineers, one role per client scoped to a named principal with a unique external ID, and source identity required. Skip the tiering, skip the IP conditions, skip the JIT ambitions. Those three things give you revocation, attribution and a clean answer for the auditor, which is most of the value.

The one thing to take away

Client AWS account access is not really a permissions problem. It’s a revocation and attribution problem wearing a permissions costume. Any design where a credential exists that isn’t tied to a specific human will fail you at exactly the wrong moment, and no amount of policy tuning fixes that, because the gap isn’t in the policy.

Put the humans in one place you control. Put a narrowly scoped, externally identified role in each client account. Require source identity so the log names a person. Everything else on this page is refinement; those three are the structure. And the pleasant side effect is that “here’s how you revoke us, unilaterally, in one action” is a genuinely good thing to be able to say in a sales conversation.

Need a second pair of eyes on your client access model?

This is the kind of work I do for consultancies and managed service providers who’ve grown past the point where a shared vault is survivable. Specifically:

• Reviewing existing cross-account trust policies for over-broad principals and missing external IDs
• Designing a hub account and federation model that scales past a handful of clients without per-client special cases
• Building the CloudFormation or Terraform template clients deploy themselves, so onboarding stops involving credentials
• Turning on source identity end to end, including the identity provider side, so CloudTrail attributes to named engineers
• Producing the offboarding evidence pack clients and auditors ask for, and automating it so it isn’t a scramble
• Narrowing long-standing administrator roles down using service last accessed data, without breaking the engagement

If you want a concrete starting point, send me one trust policy and the AccessDenied message it produces. That’s usually enough to tell where the model is leaking.

The post Twenty Client AWS Accounts and Not One Shared Admin Credential appeared first on John Nessime.