So you sketch the obvious thing. Amazon Cognito for sign-in, Amazon S3 for the documents, a small front end in between. It’s three services and a weekend, and it works on the first try, which is exactly the problem. A secure patient document portal built this way passes every test you think to run, because the tests you think to run are all “can Alice download Alice’s file?” The one that matters is “what can Alice see about Bob?” and that one has an answer you won’t like.

This post covers the parts that go wrong after the demo. Where to put the authorization boundary, how to scope the S3 policy so one patient can’t enumerate another, why your download links expire earlier than you configured, what encryption actually gates, and what your audit trail looks like on the day someone asks for it. Everything here is mechanism, not a compliance checklist. Getting a Business Associate Agreement in place is a separate conversation with your lawyer and your AWS account team.

The failure that never pages you

Here’s the shape of it. You write an IAM policy that lets a signed-in user read objects under their own prefix. Something like arn:aws:s3:::clinic-docs/patients/${cognito-identity.amazonaws.com:sub}/*. You test it. Alice gets her file. Alice cannot get Bob’s file. Ship it.

Then, because the front end needs to render a file list, someone adds s3:ListBucket. And s3:ListBucket is not an object-level action. It applies to the bucket ARN, not the object ARN, so the ${cognito-identity.amazonaws.com:sub} substitution in your resource string does nothing for it. Unless you add a separate condition on the request, a signed-in patient can list every key in the bucket.

They still can’t download anything. Every GetObject outside their prefix returns a denial. But they can read the key names. And key names in a clinic bucket are rarely neutral. patients/8f2a.../oncology-referral-2024.pdf tells you something about a person even if the bytes stay locked.

Nothing alerts on this. No error appears in any log you’re watching, because nothing errored. The listing succeeded. That’s the failure mode worth building around: in a document portal, the metadata leak is the one that doesn’t announce itself.

The fix is a condition key on the list action, not a resource string:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListOwnPrefixOnly",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::clinic-docs",
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "patients/${cognito-identity.amazonaws.com:sub}/*"
          ]
        }
      }
    },
    {
      "Sid": "ReadOwnObjectsOnly",
      "Effect": "Allow",
      "Action": "s3:GetObject",
      "Resource": "arn:aws:s3:::clinic-docs/patients/${cognito-identity.amazonaws.com:sub}/*"
    }
  ]
}

Two statements, two different jobs. The first constrains what the caller may ask the bucket to enumerate. The second constrains what bytes the caller may pull. Write only the second one and you’ve secured the download path while leaving the discovery path wide open.

One trap in that variable. ${cognito-identity.amazonaws.com:sub} resolves to the identity ID from the Cognito identity pool, not the sub claim from the user pool. They are different values for the same human. If you name your prefixes with the user pool sub because that’s what your application code has handy, the policy will match nothing and every request will be denied, which at least fails loudly.

Decide where the authorization boundary lives

There are two honest architectures here and the choice shapes everything downstream. Pick deliberately.

Pattern A: browser holds AWS credentials

The user signs in to a Cognito user pool. The application exchanges that token with a Cognito identity pool for temporary AWS credentials, and the browser calls S3 directly using the AWS SDK.

It’s less code and there’s no server to run. The authorization boundary is entirely the IAM role policy attached to the identity pool. That policy is the whole security model, so it had better be right, and it’s harder to reason about than application code because the enforcement happens in a service you can’t step through with a debugger.

The real limitation is expressiveness. IAM policy conditions do not know that Bob’s referral was superseded, that Alice revoked consent last Tuesday, or that a document should only be visible after a clinician releases it. Anything with business logic in it has to be encoded as prefix structure or object tags, and that gets brittle fast.

Pattern B: backend brokers every object

The browser never touches AWS credentials. It sends the Cognito access token to your own API. The API validates the token, checks its own records for who may see what, and returns a presigned URL for that one object. S3 is reachable only by the API’s execution role.

You now have a place to put logic, a place to log intent alongside access, and a single choke point to rate limit. The cost is that you own a service, its uptime, and its dependencies.

For a small clinic build, this is the one I reach for first. Not because Pattern A is insecure, but because the questions clients ask in month three (“can we hide results until the doctor reviews them?”) are answerable in Pattern B and require a re-architecture in Pattern A. The identity-pool approach earns its keep when access really is a pure function of who you are, which in healthcare it rarely stays.

If you do want attribute-driven access without a backend, Cognito identity pools can map claims from the token onto principal tags on the temporary session, which you then reference in policy conditions. It works, and it is the right tool for coarse distinctions like patient versus staff. It is not the right tool for per-document consent state.

Your download links expire before you think they do

This one generates support tickets on week two. You set a presigned URL to last a few hours. Patients report links dying in under one. Nothing in your code changed.

A presigned URL carries a signature made with whatever credentials signed it, and it cannot outlive those credentials. AWS documents the ceilings clearly: with long-lived IAM user keys and Signature Version 4 the maximum is seven days; from the S3 console the range is one minute to twelve hours; with temporary credentials from an assumed role, the URL dies when the role session does, regardless of the expiry you asked for. A default AssumeRole session is one hour.

Your Lambda or container signing those URLs is using temporary credentials. So the effective lifetime is the shorter of your requested expiry and the remaining time on that role session, and the remaining time depends on when the runtime last refreshed. That’s the source of the flapping. You can watch it yourself:

# Sign a URL and inspect what was actually encoded into it.
aws s3 presign s3://clinic-docs/patients/example/scan.pdf --expires-in 3600

# The returned URL contains X-Amz-Date, X-Amz-Expires and X-Amz-Credential.
# X-Amz-Expires is the window you asked for. If X-Amz-Credential shows an
# ASIA... key id, you signed with temporary credentials and the real ceiling
# is the session, not the window.

The honest resolution is to stop fighting it. Sign short, on demand, when the user clicks. Fifteen minutes is generous for a download the patient is standing in front of. Long links are not a convenience feature in this context anyway, because a presigned URL is a bearer credential: whoever holds the string can fetch the object, signed in or not. Forwarding the email forwards the access. If your portal mails links, you have moved the authorization boundary into someone’s inbox.

Mail a notification that says a document is ready and link to the portal login. Never mail the signed URL itself.

Encryption that actually gates something

New S3 objects are encrypted at rest by default with S3-managed keys. That satisfies the sentence “data is encrypted at rest” and gates precisely nothing, because anyone with s3:GetObject gets plaintext back. It is a storage property, not an access control.

Server-side encryption with an AWS KMS customer managed key is different in kind. Reading an object now requires two independent grants: the S3 permission and kms:Decrypt on the key. Those live in different policies and can be owned by different people. A bucket policy mistake no longer hands over plaintext on its own, and revoking the key grant cuts off reads without touching a single S3 permission.

You also get a second audit stream. Every decrypt is a KMS API call, recorded separately from the S3 call, which is useful precisely because an attacker who tampers with one trail probably didn’t think about the other.

The costs are real and worth stating. KMS charges per request, so a portal that fetches many small objects will notice. S3 Bucket Keys reduce that request volume substantially by caching a bucket-level key, at the price of coarser detail in the KMS audit records. For a document portal with modest traffic and high sensitivity, I’d take the per-request cost and keep the granularity. For a thumbnail-heavy application, the opposite.

The audit trail you don’t have yet

Assume someone will eventually ask: who downloaded this specific file, and when? If you have not explicitly turned something on, the answer is that you don’t know.

CloudTrail is on for management events by default, and management events are control-plane operations: creating the bucket, changing its policy, attaching a role. Object-level reads and writes are data events, and data events are off by default because of volume and cost. GetObject on a patient scan is a data event. Unless you configured a trail with data events for that bucket, no record of the download exists.

# Log S3 object-level activity for one bucket on an existing trail.
# Advanced event selectors let you scope to a single bucket prefix rather
# than every object in the account.
aws cloudtrail put-event-selectors 
  --trail-name clinic-audit-trail 
  --advanced-event-selectors '[
    {
      "Name": "PHI bucket object access",
      "FieldSelectors": [
        {"Field": "eventCategory", "Equals": ["Data"]},
        {"Field": "resources.type", "Equals": ["AWS::S3::Object"]},
        {"Field": "resources.ARN", "StartsWith":
          ["arn:aws:s3:::clinic-docs/patients/"]}
      ]
    }
  ]'

S3 server access logs are the other option and they are not equivalent. They record HTTP-level request detail and are delivered on a best-effort basis with no delivery guarantee, which makes them useful for traffic analysis and weak as evidence. CloudTrail data events carry caller identity, which is the field you actually need when the question is “who.”

Two things to sort out while you’re in there. Send the trail to a bucket in a different account, or at minimum a bucket the portal’s roles cannot write to, so that compromising the application doesn’t compromise its own log. And put a lifecycle rule on the log bucket, because data event volume grows with usage and nobody budgets for it. If you already run centralized dashboards on something like Grafana Cloud or CloudWatch, wire an alert to unusual read volume per identity now rather than after the incident.

Cutting off access when someone leaves

Disabling a Cognito user does not immediately end their session, and this surprises people. Tokens are self-contained. An access token already in a browser stays cryptographically valid until it expires, typically about an hour.

Refresh tokens are the longer exposure. The default is thirty days, and app clients can be configured anywhere from sixty minutes to ten years. Ten-year refresh tokens on a portal holding medical records is not a hypothetical misconfiguration; it’s a checkbox someone ticked to stop users complaining about re-login.

Cognito gives you revocation. Token revocation is enabled by default on newly created app clients. AdminUserGlobalSignOut revokes all of a user’s refresh, ID, and access tokens from the server side, and the /oauth2/revoke endpoint kills one refresh token and everything descended from it.

# Server-side, authorized with IAM credentials. Revokes the user's refresh,
# ID and access tokens.
aws cognito-idp admin-user-global-sign-out 
  --user-pool-id us-east-1_EXAMPLE 
  --username patient-example

# Confirm revocation is switched on for the app client before you rely on it.
aws cognito-idp describe-user-pool-client 
  --user-pool-id us-east-1_EXAMPLE 
  --client-id EXAMPLECLIENTID 
  --query 'UserPoolClient.EnableTokenRevocation'

Now the part that catches teams out. AWS documents that a revoked token still passes verification in a generic JWT library that only checks signature and expiry. Revocation is enforced by Cognito, not baked into the token. If your API validates tokens by fetching the JWKS and calling a verify function, a revoked token sails straight through until it expires on its own.

Which means short access token lifetimes are load-bearing. Revocation shortens the tail; it doesn’t close the window. Refresh token rotation, available on the Essentials and Plus feature tiers, replaces the refresh token on each exchange and narrows the window a stolen one is useful for. If you’re paying for Plus for threat protection features, you already have it.

Troubleshooting the usual suspects

Every request returns access denied after login

Nine times out of ten the prefix in your policy variable doesn’t match the prefix in your bucket. Print the identity ID your session actually resolved to and compare it character by character against a real key. Remember the identity pool ID and the user pool sub are different values.

Signature does not match on a presigned URL

Usually something modified the request after signing. A client library adding a header, a proxy rewriting the path, or an email gateway URL-encoding the query string. Compare the X-Amz-SignedHeaders list in the URL against what the client actually sent.

Uploads succeed but downloads fail

Check the KMS key policy separately from the IAM policy. A role can hold kms:GenerateDataKey (enough to write) without kms:Decrypt (needed to read). The S3 error will not say the word KMS prominently, which is why this one burns an afternoon.

Browser uploads fail before any request reaches S3

That’s CORS. The bucket needs a CORS configuration allowing your portal’s origin, the methods you use, and any headers the SDK sends. Browser preflight failures never appear in S3 logs, so check the browser console first.

Nothing appears in CloudTrail

Data events are not shown in CloudTrail Event History. Query the destination bucket directly, with Athena or CloudTrail Lake, and allow a few minutes for delivery.

Common mistakes

• Granting s3:ListBucket without an s3:prefix condition, so patients can enumerate every key even though downloads are blocked.
• Putting identifying information in object keys or file names. Assume the key will leak eventually and use opaque identifiers.
• Emailing presigned URLs. The link is the credential, and inboxes get forwarded, backed up, and breached.
• Extending refresh token lifetime to silence complaints about re-login, instead of implementing rotation.
• Assuming default encryption means access control. It means the disk is encrypted, nothing more.
• Enabling versioning without a lifecycle rule, so deleted documents linger indefinitely as noncurrent versions you forgot exist.
• Skipping CloudTrail data events to save money, then discovering the gap during an access review.

Best practices for a secure patient document portal

• Keep one bucket for patient documents and nothing else. Mixed-purpose buckets make policies ambiguous and audits slow.
• Turn on S3 Block Public Access at the account level, not just the bucket. Account level survives someone creating a new bucket in a hurry.
• Enable versioning and set a noncurrent version expiration rule, so an overwritten document is recoverable but not permanent.
• Use a KMS customer managed key so decrypt authority is separable from S3 authority.
• Require MFA for staff accounts. TOTP costs nothing to enable and staff accounts are the ones with breadth.
• Sign download URLs on demand with short expiry, and always from a backend that logged why it signed.
• Put the portal behind a WAF and rate limiting. Cloudflare in front of the application, or AWS WAF on the distribution, both work; the point is that credential stuffing against a login page is the most likely thing to actually happen to you.
• Restrict administrative access to the AWS console and to any bastion by network as well as identity, whether that’s IAM Identity Center with condition keys or a VPN such as NordVPN or Surfshark for a small team without central identity.
• Write down the retention period before launch, then encode it as a lifecycle rule. Retention decided later is retention never decided.

Frequently asked questions

Are S3 and Cognito enough for HIPAA compliance?

Both are covered under AWS’s HIPAA eligibility program, and Cognito’s documentation states it is HIPAA-BAA eligible. That makes them usable building blocks, not compliance. You need a signed Business Associate Agreement with AWS, and the configuration, access reviews, training, and documented policies remain yours. Service eligibility is a prerequisite, not an outcome.

Should I use a Cognito identity pool or a backend API?

Use an identity pool when access is a pure function of identity and the rules will not change. Use a backend when documents have states, when a clinician gates release, or when you need to log intent alongside access. Most clinic portals become the second within a year.

How long should a presigned URL last?

Minutes, not hours. Sign at click time. The maximum with long-lived credentials is seven days, but that ceiling is irrelevant here because the link is a bearer credential and a longer window is a longer leak.

Can patients upload documents back to the portal?

Yes, using presigned PUT or presigned POST. Presigned POST is worth knowing about because its policy document can constrain content type and size before the object lands, which presigned PUT cannot. Either way, write uploads into a quarantine prefix, scan them, then move them.

Does enabling CloudTrail data events cost much?

Data events are billed per event recorded with no free tier, unlike management events. For a portal serving a few thousand downloads a month the amount is small; for a bucket also serving application assets it is not. Scope your event selector to the patient prefix rather than the whole account and the question mostly goes away.

What happens to sessions when I disable a user?

Nothing immediate. Existing access tokens remain valid until they expire. Call AdminUserGlobalSignOut to revoke tokens, and keep access token lifetime short so the residual window is measured in minutes.

Do I need CloudFront in front of the bucket?

Not for correctness. It helps when patients are geographically spread, when you want signed cookies for a set of objects rather than one URL per object, or when you want a single edge to attach a WAF to. For a single-clinic portal, direct presigned URLs are simpler and one fewer thing to misconfigure.

The one thing to remember

A signed-in token proves who someone is. It does not decide what they may reach. In a secure patient document portal the boundary is the policy condition, the KMS grant, and the code that decides whether to sign a URL at all, and every one of those is something you have to write on purpose.

The demo passes either way. The difference only shows up when someone reads a key name they shouldn’t have seen, and nothing in your system considers that an error.

If you build one of these, spend an hour before launch on exactly two things: sign in as a test patient and try to list the bucket, then ask your logs who downloaded a specific file last week. If either answer is uncomfortable, you’ve found your next task.

Need a second pair of eyes on your portal?

This is the kind of build where the gap between working and safe is invisible from the outside. I help small teams close it.

• Reviewing S3 bucket policies and Cognito identity pool roles for prefix isolation gaps, including the list-versus-get split most policies miss.
• Designing the boundary: identity pool credentials in the browser versus a backend that brokers presigned URLs, with the trade-offs written down for your stakeholders.
• Setting up KMS customer managed keys, key policies, and grant separation so decrypt authority is not implied by storage authority.
• Turning on scoped CloudTrail data events, building the Athena tables to query them, and alerting on abnormal per-identity read volume.
• Hardening the Cognito side: MFA, token lifetimes, refresh token rotation, and a revocation path that actually ends sessions.
• Writing the whole thing as Terraform or OpenTofu so the next change is a reviewed pull request rather than a console click.

Send me your bucket policy and your identity pool role, with the account IDs redacted, and I’ll tell you what a signed-in patient can actually see.

The post Building a Secure Patient Document Portal on S3 and Cognito appeared first on John Nessime.

So you open the audit table. And you find it records who requested a link to the document. Not who opened it. The actual downloads never touched your application at all, because you handed out a signed storage URL and the browser went straight to the bucket. Your log is a record of intentions.

That gap is typical of how property SaaS document storage goes wrong. The failures are rarely dramatic. Nobody dumps a bucket publicly. The guarantees you thought you had just turn out to be enforced in application code that ran once, long before the file was actually read. What follows is five failure families worth designing against when a platform stores tenant ID scans, signed leases, deposit certificates, right-to-rent evidence and referencing packs.

What’s actually in the pile

Before architecture, be honest about the contents. A property platform’s document store is at least four risk classes wearing the same file extension:

• Identity evidence. Passport and driving licence scans, visa documents, sometimes a selfie held next to the document. Highest blast radius, lowest genuine need for long retention.
• Contracts. Tenancy agreements, addenda, guarantor deeds, signed inventories. Legally driven retention, plus an integrity requirement: proving the file hasn’t changed matters more than hiding it.
• Financial and referencing material. Bank statements, payslips, credit reference reports. Sensitive, third-party sourced, and often covered by contractual restrictions from the referencing provider.
• Operational clutter. Inspection photos, safety certificates, maintenance invoices. Low sensitivity, high volume, and the thing that dominates your storage bill.

These want different retention clocks, different access rules and arguably different buckets. Treating them as one “documents” table is where most of the later pain originates.

Failure one: the access check and the access happen at different times

Presigned URLs are the right pattern. They keep the bucket private, take the transfer off your servers, and mean you never hand storage credentials to a browser. I reach for them first.

The problem is what a presigned URL is. It’s a request you already authorised, signed, and then gave away. Anyone holding it can replay it until it expires. Your permission logic ran once and never runs again. Which means all of these keep working after the fact:

• A letting agent who left the company yesterday and whose account you disabled this morning.
• A link forwarded into a WhatsApp group, pasted into a support ticket, or sitting in browser history on a shared office machine.
• A document the tenant asked you to delete, if the object is still there behind a delete marker.

Make the URL die when the session does

Amazon’s own guidance is clearer on this than most people realise. A presigned URL created with temporary credentials expires when those credentials expire, even if you asked for a longer expiry. It also stops working if the signing credential is revoked, deleted or deactivated.

That’s a lever, not a footnote. Sign with a long-lived IAM user key and the seven day maximum the SDK allows is a real seven days. Sign inside a short-lived assumed-role session and the URL cannot outlive that session, whatever number you passed in. So: never sign document URLs with static credentials. Assume a role per request, tag the session with the tenant, sign inside it.

import boto3

sts = boto3.client("sts")

# The tag key here must match the principal tag key used in the role's policy.
session = sts.assume_role(
    RoleArn="arn:aws:iam::111122223333:role/TenantDocumentAccess",
    RoleSessionName="doc-download",
    Tags=[{"Key": "TenantID", "Value": tenant_id}],
    DurationSeconds=900,
)["Credentials"]

s3 = boto3.client(
    "s3",
    aws_access_key_id=session["AccessKeyId"],
    aws_secret_access_key=session["SecretAccessKey"],
    aws_session_token=session["SessionToken"],
)

url = s3.generate_presigned_url(
    "get_object",
    Params={
        "Bucket": "prop-docs",
        "Key": f"tenants/{tenant_id}/identity/{doc_id}.pdf",
        "ResponseContentDisposition": 'attachment; filename="id-document.pdf"',
        "ResponseContentType": "application/octet-stream",
    },
    ExpiresIn=120,
)

Two things are doing work. DurationSeconds caps the real lifetime of the link regardless of ExpiresIn. And ResponseContentDisposition overrides how the object is served on this request, which matters for the serving problem further down.

The trade-off is real: an STS call per download adds latency, adds a dependency, and STS has request limits you can hit at volume. AWS’s prescriptive guidance says as much. For identity documents I’d pay that cost. For inspection photos I wouldn’t.

Log where the read actually happens

Your application log records issuance. To record retrieval you need logging at the storage layer: CloudTrail data events for object-level operations, or S3 server access logging. Data events are billed per event, so scope them to the identity and contract prefixes rather than the bucket holding thousands of inspection photos.

Failure two: isolation that lives in your ORM instead of your IAM policy

Ask most teams how tenant isolation works and you get a description of a WHERE clause. The key is built as tenants/{tenant_id}/... and the tenant ID comes from the session. Fine until one code path takes it from a request parameter, or a background job iterates without scoping, or someone writes an admin endpoint in a hurry.

Move the boundary below your code. With attribute-based access control the tenant becomes a session tag and the IAM policy interpolates it into the resource ARN. One role serves every tenant, and the policy count doesn’t grow as you onboard.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadWriteOwnTenantPrefix",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::prop-docs/tenants/${aws:PrincipalTag/TenantID}/*"
    },
    {
      "Sid": "ListOnlyOwnTenantPrefix",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::prop-docs",
      "Condition": {
        "StringLike": {
          "s3:prefix": "tenants/${aws:PrincipalTag/TenantID}/*"
        }
      }
    }
  ]
}

Two statements because S3 splits the problem. The first governs the object, where the tenant prefix is part of the resource ARN. The second governs listing, a bucket-level action, so the prefix has to be constrained through a condition instead. Skip it and a tenant can enumerate every key in the bucket even though they can’t read any of them, and key names in this domain leak plenty on their own.

The failure mode worth internalising: if a code path forgets the session tag, ${aws:PrincipalTag/TenantID} resolves to nothing and the statement doesn’t match. That’s a deny, which is what you want. But it only holds if the role has no broader fallback statement. One well-meaning s3:GetObject on the whole bucket, added to unblock a migration script, quietly turns the scheme into decoration.

When to use a bucket per tenant instead

A bucket or an account per tenant is a genuinely harder boundary and I wouldn’t dismiss it. With a few dozen large agency customers it’s defensible and makes offboarding trivially provable. It stops being defensible once you onboard self-serve landlords, because bucket and account counts run into service quotas and the operational surface grows with every signup. Pick on tenant count, not on which sounds more secure. And note that ABAC is only as trustworthy as the tenant claim: if the caller can influence the value reaching the tagging call, you’ve built an elegant no-op.

Failure three: the upload and serve path

Uploads here come from people you have no relationship with yet. Prospective tenants, guarantors, contractors. That’s an untrusted channel by definition, and it produces two separate problems.

Constrain the upload itself

A presigned PUT URL lets the holder write whatever they like to that key, at whatever size. Presigned POST is the better tool for public-facing uploads: the signature covers a policy document, and the policy carries conditions the browser cannot alter.

post = s3.generate_presigned_post(
    Bucket="prop-docs",
    Key=f"tenants/{tenant_id}/inbox/{upload_id}",
    Fields={"Content-Type": "application/pdf"},
    Conditions=[
        {"Content-Type": "application/pdf"},
        ["content-length-range", 1, 15728640],
    ],
    ExpiresIn=600,
)

The content-length-range condition is the one people skip, and it’s the one that stops a stranger using your referencing portal as free storage for a 40 GB file. A declared Content-Type is a claim, not a fact, so land uploads in a quarantine prefix, verify the actual bytes, and only then move them to the tenant prefix.

Scan on that path too. ClamAV in a container is the zero-cost option and covers known malware; GuardDuty Malware Protection for S3 and hosted APIs such as Cloudmersive trade money for not maintaining it. What you’re protecting is mostly the letting agent who opens the attachment on a Windows machine, so this is a control for your customers rather than your servers.

Never serve tenant documents from your application origin

This is the one that turns a document store into an account takeover. SVG and HTML are formats browsers execute. If a file uploaded by a prospective tenant is served inline from the same origin as your app, any script inside it runs with your application’s cookies and session context. A long stream of real advisories follows exactly this shape, particularly in self-hosted deployments where the object store sits on the app’s own domain.

Three controls, and they stack rather than substitute:

1. Force download. Serve with Content-Disposition: attachment and X-Content-Type-Options: nosniff. Set these as object metadata at upload time and override them per request. The override protects links you mint; the stored metadata protects anything that bypasses your minting path.
2. Use a separate origin. Put user content on a distinct domain, not a subdomain of the app. GitHub’s separate apex for user content is the reference implementation. Same-origin policy then contains the blast radius if control one is missed somewhere.
3. Normalise where you can. Rendering an uploaded ID to a flattened image server side removes the executable content entirely. It costs CPU and fidelity. For identity documents, where you rarely need the original bytes back, it’s the right trade.

Failure four: deleting the document doesn’t delete the document

A tenant asks you to erase their passport scan. Your app sets a flag, the file disappears from the UI, and you reply confirming deletion. Then count the copies that still exist.

• The previous object versions, if versioning is on. A delete without a version ID just writes a delete marker.
• Cross-region replicas with their own versioning and lifecycle rules, plus any backup or snapshot taken before the deletion.
• Derivatives you generated and forgot: thumbnails, page previews, OCR text in a search index. Run a passport through Amazon Textract and the extracted fields are a copy of the sensitive data in a different shape.
• Copies that left the platform. A document attached to a CRM record in something like GoHighLevel, forwarded by email, or downloaded to a letting agent’s laptop.

The ICO’s position is that a valid erasure request extends to backups as well as live systems, and that where backup data cannot be immediately overwritten it must be put beyond use: isolated, unused for anything else, deleted when that backup cycles. Workable, but only if you know which backup sets contain documents and how long they cycle. If you can’t answer that, you can’t answer the request.

Crypto-shredding, and what it actually costs

The one technique that makes deletion provable across copies you can’t reach is a per-tenant KMS key. Encrypt that tenant’s objects under their own customer managed key; when they offboard, schedule the key for deletion. Every ciphertext copy anywhere, backups included, becomes undecryptable. You delete the data by deleting the ability to read it. The honest costs:

• KMS bills per key per month plus per request, so per-tenant keys are a real line item. Check current rates and quotas first: there’s an account limit on customer managed keys that a self-serve signup flow will eventually meet.
• Key deletion has a mandatory waiting period. Crypto-shredding is not instant, and your erasure response has to say so.
• It’s all-or-nothing per key. If a tenant asks you to erase one document rather than offboard, the key doesn’t help.

Object Lock fights erasure, on purpose

S3 Object Lock is tempting for signed tenancy agreements, because write-once storage is a strong integrity story. But compliance mode means nobody, root included, can delete the object until retention lapses. That is the feature. It is also a direct collision with an erasure request, and you cannot engineer your way out of it afterwards.

My line: Object Lock on executed contracts, where a statutory retention basis overrides erasure, and never on identity documents or referencing material. Different buckets, different rules, decided before anything is written. For per-document erasure elsewhere, lifecycle rules that expire noncurrent versions on the sensitive prefixes are worth setting up before you need them.

Failure five: storing what you never needed to store

The cheapest control on this list is not storing the file. The ICO is clear that UK GDPR sets no fixed retention periods and puts the burden on you to justify how long you keep personal data in identifiable form. That’s a design instruction, not a legal footnote.

For identity documents, ask what your platform needs once the check is done. Usually it’s the result, not the evidence: document type, issuing country, expiry date, a check reference, who performed it and when. If your customer’s regulator requires the image itself, that’s a retention rule with a stated basis, which is a different conversation from keeping it because deleting felt risky.

Practical version: give every document class a retention clock wired to a business event rather than upload date, and let lifecycle rules do the deleting. Tenancy end plus a defined period. Check completion plus a defined period. Not “whenever someone remembers.”

Troubleshooting the breakages you’ll actually hit

Everything broke after someone enabled S3 Bucket Keys

This one catches teams who did the sophisticated thing. If your IAM or KMS key policies use the object ARN as the encryption context to scope access, enabling an S3 Bucket Key changes that context to the bucket ARN and those policies stop matching. AWS documents it explicitly as something to fix before flipping the setting.

The quieter side effect: KMS CloudTrail events then log the bucket ARN instead of the object ARN, and there are far fewer of them because S3 caches the bucket-level key. If you were treating KMS CloudTrail volume as a proxy for document access, that signal degrades without anything appearing to break. Bucket Keys are still worth enabling for the request cost reduction. Move the conditions first, then re-check what your dashboards count.

The presigned URL expires far sooner than requested

Almost always the signing credential is a role session shorter than the expiry you asked for. This is documented behaviour, not a bug. Either raise the session duration within the role’s maximum, or accept the shorter window as the security property it is.

Listing works, reading returns Access Denied

Classic split between the bucket-level and object-level statements. Check both exist and reference the same tag key spelling. Session tag keys are case sensitive, and a tenantId in the code against a TenantID in the policy fails silently as a non-match.

Access denied only for encrypted objects

An SSE-KMS object needs two grants, not one. The S3 action on the object, and kms:Decrypt on the key it was encrypted under. In a per-tenant key setup that second grant is usually also tag-conditioned, so a missing or mismatched tag on the key itself produces a denial that looks like an S3 problem and isn’t.

Common mistakes

• Signing document URLs with a long-lived IAM user access key, which makes every link independently long-lived.
• Granting the tenant role a broad bucket-wide read statement alongside the tag-scoped one, usually to unblock a one-off script that then stayed.
• Serving user uploads from a subdomain of the application and assuming that counts as origin separation. It doesn’t for cookie scope.
• Enabling versioning for safety and never adding a noncurrent version expiry rule, so every deletion is reversible forever.
• Building deletion that handles the object but not the OCR output, the thumbnail, or the search index entry.
• Putting Object Lock on the bucket that also holds identity documents.

Design rules worth holding to

1. Separate buckets by risk class, not by feature. Identity, contracts, financial, operational. Different encryption, retention, logging and lock settings follow naturally from that split.
2. Enforce isolation in IAM, with application checks as defence in depth rather than the boundary itself.
3. Sign short, sign from a session. Minutes, not days, and always from assumed-role credentials so revocation actually revokes.
4. Deny unencrypted writes at the bucket policy, using the server-side encryption request headers as conditions, so a misconfigured client fails loudly instead of writing plaintext.
5. Give every document class an expiry clock tied to a business event, implemented in lifecycle rules rather than a cron job someone maintains.
6. Write the deletion runbook before launch, listing every place a copy lands. If you can’t enumerate them, that’s the finding.
7. Test cross-tenant access in CI. A negative test that asserts tenant A’s session cannot read tenant B’s key is worth more than any amount of policy review.

Frequently asked questions

How long should a presigned URL for a tenant ID document last?

Short enough that the user clicks it immediately. A couple of minutes covers a click-to-download flow. If the UI needs longer, re-mint on demand rather than extend. And remember the ceiling is set by the signing credential, so a short assumed-role session enforces the limit even if a later code change passes a bigger number.

Do I need AWS, or will Backblaze B2 or Cloudflare R2 do?

They all speak the S3 API and presigned URLs work on all of them, so the core patterns transfer. Wasabi, Backblaze B2, Cloudflare R2 and DigitalOcean Spaces are reasonable for the operational tier, and R2’s egress model is attractive if agents download a lot. They differ in the depth of the IAM story: session-tag ABAC and per-tenant KMS keys are AWS-specific. If isolation and crypto-shredding are hard requirements, keep the sensitive prefixes on S3 even when the bulk photos live somewhere cheaper.

Can I store scanned passports on a normal VPS instead?

You can, and plenty of platforms run the app tier on a VPS from Contabo or InterServer while keeping documents in object storage. What you lose on local disk is the deletion story and the access log. Proving an image is gone from a filesystem, its backups and its snapshots is far harder than scheduling a KMS key for deletion. If documents stay local, budget real effort for secure disposal, including on staff machines where tools like O&O SafeErase have a place.

What about e-signed tenancy agreements?

With a provider like DocuSign, the executed document and its audit trail live there as well as in your platform. That’s a second copy under a second retention regime, and it belongs in your deletion runbook and data map. It also strengthens your integrity story, which is a fair reason to skip Object Lock on contracts you retrieve from the provider.

Should the CDN cache tenant documents?

Not the sensitive classes. A CDN in front of private documents adds a cache that can serve content after your access decision changed, and header handling at the edge is a common source of the inline-rendering bug. CloudFront with origin access control and signed URLs is legitimate if you need the performance, but for identity documents I’d go straight to the bucket. For inspection photos, cache away.

How do I prove any of this to a prospect’s security team?

Artefacts, not assertions. The bucket policy, the ABAC policy, CI output showing cross-tenant denial, a sample of storage-layer access logs, and a retention schedule with a clock per document class. That set answers most questionnaires without a call.

The one thing worth remembering

Good property SaaS document storage is mostly about closing the gap between when you check access and when access happens. Every failure family above is a version of that gap: the presigned link that outlives the permission, the isolation enforced once in code and never at the storage layer, the deletion that updates a flag while the bytes sit behind a delete marker.

Close it by pushing the decision down: into IAM policies that evaluate on every request, into credentials short enough that revocation is real, into lifecycle rules that delete without anyone remembering to. Then the answer to “show us who read this tenant’s ID document” is a query, not a project.

Need a second pair of eyes on your document storage?

Most of what I do on property and lettings platforms is this exact work. Things I’d help with here:

• Reviewing bucket and IAM policies for cross-tenant read paths, including the ones added to unblock a migration and never removed.
• Moving tenant isolation from application code into session-tag ABAC, with a CI test that fails the build on cross-tenant access.
• Rebuilding the upload path with presigned POST conditions, quarantine, content verification and scanning before anything lands in a tenant prefix.
• Fixing the serving path so uploaded files can’t execute against your origin, including header handling through a CDN.
• Designing retention clocks and lifecycle rules per document class, plus a deletion runbook that names every copy including derivatives and third-party systems.
• Setting up storage-layer access logging scoped to the sensitive prefixes so you can actually answer an access request.

If it’s useful, send me your bucket policy and the IAM policy attached to your document role, with the account IDs redacted, and I’ll tell you what a determined tenant could reach.

The post Property SaaS Document Storage: The Five Failures That Show Up in an Audit appeared first on John Nessime.

Nobody looks at the old link. If it was signed with a long expiry and mailed out, it is still live: in the client’s inbox, in the inbox of the bookkeeper on the CC line, and in whatever mail archive that firm runs. It keeps working for anyone holding the URL until the clock runs out, because a presigned S3 URL is a bearer token. AWS says exactly that in its own documentation. The URL does not know who you are. It knows that you have it.

That is the shape of most document vault failures at a small accounting firm. Not a wide-open bucket. Not a stolen partner password. An ordinary delivery mechanism that leaks quietly and writes no log line that looks wrong six months later.

This post covers how to build a secure client document vault for an accounting practice: what the regulation actually demands, the six failure families that decide whether it holds, and the control that closes each one. The examples are AWS because that is where most of these land, but the failure families are platform independent.

Let the regulation write your requirements

Most firms treat compliance as paperwork done after the build. That is backwards. For a US accounting or tax practice, the FTC Safeguards Rule at 16 CFR Part 314 is the closest thing you will get to a free requirements document.

It treats tax preparers, bookkeepers and accounting firms as financial institutions under the Gramm-Leach-Bliley Act. Size does not get you out of it. The parts that translate directly into architecture:

• Access controls limiting each user to the customer information they need, and limiting customers to their own information.
• Encryption in transit over external networks and at rest, with compensating controls only where encryption is infeasible and your Qualified Individual signs off in writing.
• Multi-factor authentication for any individual accessing any information system, subject to the same written-approval exception.
• Secure disposal, with an outer bound of two years after last use unless retention is required by law or necessary for legitimate business purposes.
• Logging of authorized user activity, plus a named Qualified Individual who owns the program.

One more clause should shape your design more than any other. The Rule requires notifying the FTC within thirty days of discovering a security event involving the unencrypted information of at least five hundred consumers, and it counts encrypted information as unencrypted if the key was accessed by an unauthorized person.

Read that twice. Encryption only helps at the breach-analysis stage if the key was genuinely out of reach. That is the entire argument for treating key custody as a control separate from storage.

Separately, IRS Publication 4557 expects paid preparers to keep a Written Information Security Plan, and PTIN renewal asks you to confirm you have one. Treat the WISP as the document describing the vault you built, not a substitute for building it.

Failure one: the link that outlives the engagement

Presigned URLs are the sensible answer to “let the client download this without an AWS account”. The problem is their expiry semantics, which almost nobody has read.

Two clocks apply and the shorter wins: the expiry you asked for, and the lifetime of the credential you signed with. Long-lived IAM user credentials under Signature Version 4 reach seven days. A role session from AssumeRole dies when the session ends, one hour by default, regardless of what you requested. An EC2 instance profile is bound by metadata credential rotation. The console caps at twelve hours.

So the two symptoms are opposites and both are common. Links dying early is a credential problem. Links living too long is a policy problem, and that is the one that hurts.

The fix is not a smaller number. It is changing what goes in the email.

1. Email a notification, not the file link. The client logs in with MFA and the portal mints a URL at click time.
2. Keep that expiry in minutes. The URL only has to survive a redirect and a download.
3. Scope every URL to one object and one verb.
4. If you truly need a long-lived share, use CloudFront signed URLs or signed cookies, where you can revoke by rotating the key group without touching the object.

# Sign a download URL valid for five minutes.
# --expires-in takes seconds; the default is 3600.
aws s3 presign s3://firm-client-vault/clients/acme-llc/workpapers.pdf 
    --expires-in 300

Run that under an assumed role, wait past the session expiry, and watch it fail well before five minutes. That is behaviour worth seeing once in a test rather than discovering during an incident review.

Failure two: everyone inside the firm sees everything

This shows up in nearly every small-firm build, and the reasoning is always the same. Nine people, everyone helps with everything during filing season, and compartmentalising access is friction exactly when friction is expensive.

The case for flat access is real, so say it out loud: a scoped model people fight gets worked around, and a workaround is worse than the flat model you started with. If your access control makes a senior accountant email a PDF to themselves at 11pm, you have made things less safe.

So scope by engagement rather than by person, and automate the grant so nobody files a request. Make the key prefix the boundary, then grant the prefix instead of the bucket. Two statements, two jobs: the first controls what someone can enumerate, the second what they can fetch.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListOnlyThisClient",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::firm-client-vault",
      "Condition": {
        "StringLike": { "s3:prefix": "clients/acme-llc/*" }
      }
    },
    {
      "Sid": "ReadWriteThisClient",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::firm-client-vault/clients/acme-llc/*"
    }
  ]
}

Without the prefix condition on ListBucket, a scoped user still gets a full inventory of your client list. Client names are confidential in this business. A directory listing that reads like a book of business is a disclosure even when no file opens.

The other half of this family is standing administrative access. Somebody has to fix things, so somebody ends up with a permanent s3:* on the whole bucket. That principal is the highest-value target in the firm and usually has the weakest MFA, because it is a service account nobody wants to touch.

Replace it with an assumable break-glass role: MFA required, short session, and an alert on every assumption. The alert is the point. Emergency access used twice a year is fine. Used weekly, it means your normal path is broken.

Failure three: encrypted at rest, key sitting right next to it

Default bucket encryption gets ticked in week one and forgotten. It satisfies the letter of “encrypted at rest” and protects against exactly one threat: somebody walking out of a data centre with a disk. That is not your threat.

Go back to the notification clause. If an attacker reaches a principal that can both read the object and use the key, your data is unencrypted for reporting purposes. The question is not “was the bucket encrypted”. It is “could whoever got in also use the key”.

• Use a customer managed KMS key. Only that gives you a key policy you control.
• Name the principals allowed to call Decrypt, and keep that list shorter than the list allowed to call GetObject. Two different lists means an attacker needs two wins.
• Use encryption context so grants can be conditioned per client, and so CloudTrail entries are readable later.
• Enable S3 Bucket Keys to cut KMS request volume. KMS bills per API call, so a vault serving many small reads generates a surprising line item.

Encryption in transit needs its own control, because IAM policies say nothing about the connection. A bucket policy does:

{
  "Sid": "DenyPlaintextTransport",
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": [
    "arn:aws:s3:::firm-client-vault",
    "arn:aws:s3:::firm-client-vault/*"
  ],
  "Condition": {
    "Bool": { "aws:SecureTransport": "false" }
  }
}

An explicit deny beats every allow, including one you add by mistake later. Five lines that permanently remove a category of misconfiguration, which makes it the best value item on this page.

Failure four: retention rules that fight your disposal duty

Here is the trap, and it is why I would not let an accounting firm switch on Object Lock without a conversation first.

Object Lock has two retention modes. In governance mode, a principal holding s3:BypassGovernanceRetention can remove or shorten the lock. In compliance mode, nobody can. Not an administrator, not the account root user. AWS is explicit that the only way to delete a compliance-locked object before its retention date is to close the AWS account.

Set that against the Safeguards Rule’s two-year outer bound on disposal of information you no longer need. Apply a blanket seven-year compliance-mode retention to everything landing in the vault and you have written a promise you cannot revoke, on documents you may later be obliged to destroy.

1. Classify by document type before client. Filed returns, engagement letters and supporting workpapers have real statutory retention arguments. Raw intake uploads and scratch scans usually do not.
2. Default to governance mode. It stops accidental and casual deletion, which is what you need, while keeping a documented and audited escape hatch.
3. Give s3:BypassGovernanceRetention to almost nobody and alert on every use through CloudTrail.
4. Reserve compliance mode for records where an external mandate genuinely requires immutability and you chose the period deliberately.
5. Use legal hold, not a retention period, for litigation. A legal hold has no end date and stays on until explicitly removed, which is right when you cannot know how long a matter runs.

Two details catch people out. Object Lock requires versioning and normally has to be enabled at bucket creation, so it is a design decision rather than a retrofit. And lifecycle expiration will not delete a version still under retention, so lifecycle and Object Lock are two layers that must agree, not alternatives.

# What lock configuration is this bucket actually running?
aws s3api get-object-lock-configuration --bucket firm-client-vault

# Freeze one object version for a matter with no known end date.
aws s3api put-object-legal-hold 
    --bucket firm-client-vault 
    --key clients/acme-llc/2023/return.pdf 
    --legal-hold Status=ON

For deletion itself, crypto-shredding is worth knowing. If a client’s documents sit under a per-client KMS key, scheduling that key for deletion renders every object unreadable without touching the objects. Clean, fast, provable, and irreversible once the waiting period ends, so it belongs behind two-person approval rather than in a script.

Failure five: no evidence when somebody asks

Every control above is worth nothing if you cannot show it held. The Rule expects logging of authorized user activity and an annual written report from the Qualified Individual.

The trap is that CloudTrail does not record S3 object-level reads and writes by default. Management events cover things like changing a bucket policy. Data events cover the actual GetObject calls, and they are opt-in and billed per event. Firms enable CloudTrail, see a trail in the console, assume they have a record of file access, then find during an incident that they have nothing at object level.

Turn on data events for the vault bucket. At firm scale the volume is manageable, and it is the difference between “we believe two people opened that file” and being able to show it.

Then decide where logs live. Access records for client financial data are themselves sensitive, so keeping them in the same account under the same administrators defeats the purpose. A separate logging account, or at minimum a separate bucket with its own key and retention, is the shape you want. If you already run Grafana Cloud or similar for infrastructure monitoring, pointing vault access logs there gives you dashboards and alerting without standing anything new up.

The alerts worth having are boring and few: any use of the break-glass role, any change to the bucket or key policy, any bulk download outside normal working patterns, and any failed decrypt. Four alerts that fire almost never beat forty that everyone mutes.

Failure six: the two ends of the pipeline

A vault only protects documents that are in it. The two ends are where firms quietly undo everything.

Intake

If clients can still email a W-2 as an attachment, they will, because it is easier. Each attachment becomes customer information sitting in a mailbox, a sent-items folder and a mail archive, each with a different retention policy and access model than your vault.

A policy memo will not fix this. Make the vault path faster than the email path: an upload link that works on a phone, no account creation for a one-off drop, and a mail rule that returns certain attachment types with the upload link instead. Then sweep the mailboxes you already have, because that backlog is real customer information and it counts.

Offboarding

Seasonal staff are normal here, so offboarding happens in batches and gets rushed. Disabling the SSO login is the part everyone does. The parts that get missed:

• Active role sessions and refresh tokens, which survive the account being disabled until they expire on their own.
• Presigned URLs that person generated, which stay valid because they are bound to the signing credential, not the login session.
• Named principals still sitting in the KMS key policy.
• Local copies on laptops and external drives. When hardware is retired, a certified erase tool such as O&O SafeErase is the difference between disposal and hoping.

The same applies to service providers. The Rule makes you responsible for selecting providers capable of maintaining appropriate safeguards and holding them to it contractually. If an outsourced bookkeeping team touches your document workflow, that relationship is in scope, and direct network access means they need MFA too. Cloudflare Zero Trust or a business VPN such as NordLayer in front of the portal gives you somewhere to enforce that without rebuilding the application.

Troubleshooting a secure client document vault

• Links expire far earlier than the expiry you set. You signed with temporary credentials that expired first. Sign from a principal whose credential outlives the link, or shorten the link and mint on demand.
• Uploads fail with access denied despite an allow on PutObject. With a customer managed key, both the IAM policy and the key policy must permit the caller. Check kms:GenerateDataKey, not just S3 actions.
• A file will not delete and no policy seems to block it. Look for Object Lock retention or a legal hold on that version. They are independent and either one alone blocks the delete.
• CloudTrail shows the bucket being created but no file access. Data events are not enabled. Management events will never show GetObject.
• A scoped user can see every client folder name. The prefix condition is missing from ListBucket, or was applied to the object ARN instead of the bucket ARN.
• Lifecycle rules appear to do nothing. Versions under retention are not expired by lifecycle. Confirm lock state before assuming the rule is wrong.

Common mistakes, and what to do instead

• Emailing the download link. Email a notification that a document is waiting, and mint the link after the client authenticates.
• Treating default bucket encryption as finished. Use a customer managed key and keep the decrypt list shorter than the read list.
• Granting bucket-wide list permissions. Condition ListBucket on the client prefix, or the roster leaks even when no document opens.
• Applying compliance-mode Object Lock broadly because it sounds safer. Write the retention schedule per document class first, get partners to sign it, then configure to match.
• Keeping a standing full-access principal. Replace it with an MFA-gated break-glass role that alerts on every assumption.
• Building a portal and leaving email intake open. Both channels then carry client data and only one is governed. Close the mail path and sweep the backlog.
• Storing access logs beside the data they describe. Ship S3 data events somewhere with different administrators and different keys.
• Never testing any of it. Verify a restore and a scoped-access denial quarterly. An unverified control is an assumption.

Frequently asked questions

Does a small accounting firm really fall under the FTC Safeguards Rule?

Almost certainly, if it prepares returns or handles client financial data in the United States. The Rule’s definition of a financial institution covers activities financial in nature, and the FTC’s own examples include tax preparation firms. A narrow exception at 16 CFR 314.6 covers institutions holding information on fewer than five thousand consumers, but it exempts only four specific provisions. Encryption, MFA, access controls and disposal still apply. Confirm your own position with counsel, not a blog post.

Should I build this or buy a client portal product?

Buy, if the product genuinely does what you need. A firm with no engineering capacity is safer on a maintained product than a bespoke stack nobody patches. Build when you need controls the product will not give you: per-client key isolation, your own retention logic, or integration with a pipeline you already run. The honest test is whether someone will still own the system in a year.

Can I self-host a secure client document vault instead of using AWS?

Yes. Nextcloud or similar on a well-run VPS from a provider such as InterServer or Contabo can meet the same requirements, and it is often simpler to reason about. What you take on is patching, backups, TLS renewal and log retention. The regulation does not care where the data sits. It cares that the controls exist and that you can show they held.

How long should client documents be retained?

There is no single number, which is why a schedule beats a default. Tax and professional standards drive minimums for filed returns and engagement documentation, and those vary by jurisdiction and engagement type. The Safeguards Rule pushes the other way. Write both directions down per document class, then configure to match.

Is a presigned URL safe to send to a client?

It is safe to use, not safe to send. Treat it as a bearer token with a short life, minted after the client authenticates and consumed immediately. What travels over email should be a notification, not the credential.

Does encrypting the bucket mean I avoid breach notification?

Only if the key stayed out of the attacker’s hands. The Rule treats encrypted customer information as unencrypted where the key was accessed by an unauthorized person. That is the whole argument for keeping key permissions narrower than storage permissions.

What is the minimum viable version for a two-person practice?

One bucket with per-client prefixes, a customer managed key, an explicit deny on non-TLS transport, MFA on every human account, short mint-on-click links, S3 data events enabled, and a written retention schedule. That is a weekend of work and it closes five of the six failure families above.

The one thing worth remembering

A secure client document vault is not defined by the storage you picked. It is defined by how narrow the path is between an attacker and a readable file, and by your ability to prove afterwards how many people walked it.

Most firms get the storage right and lose at the edges: a link that outlives the engagement, a key policy as wide as the bucket policy, a retention setting that cannot be undone, an audit trail nobody turned on. Fix those four and the rest is ordinary engineering.

Where I can help

I work with small firms and the MSPs that support them on exactly this kind of build. Things I take on regularly:

• Designing and building the vault itself: bucket layout, per-client prefix isolation, KMS key policy, and the portal glue that mints short-lived links.
• Reviewing an existing setup against the Safeguards Rule’s technical elements and producing a gap list your Qualified Individual can work from.
• Untangling retention: turning a vague policy into a per-document-class schedule, implemented with lifecycle rules, Object Lock and legal holds that do not conflict.
• Standing up the evidence layer: S3 data events, a separate logging account, dashboards, and the small set of alerts worth waking up for.
• Closing the email intake path without making life harder for clients, including mobile upload flows and a sweep of the historic mailbox backlog.
• Offboarding and access reviews: break-glass roles, session revocation, key policy hygiene, and a repeatable quarterly check.

If you want a second opinion, send the thing itself: a bucket policy, an IAM policy, a CloudTrail event selector, or a description of how documents reach your clients today. It is much easier to say something useful about a real configuration than about a general question.

The post Building a Secure Client Document Vault Your Accounting Firm Can Defend appeared first on John Nessime.

It is a one-line change. It works on the first try. And it is the grant most likely to still be sitting there long after that analytics project shipped, covering a wider set of identities than the person who wrote it ever had in mind.

That asymmetry is the whole problem. A denied cross-account request is a nuisance: someone pings you, you read the policy, you fix it before lunch. A cross-account grant that succeeds, keeps succeeding, and quietly reaches further than intended produces no error, no alert, and no ticket. It surfaces months later during an audit, when someone asks who has been reading that prefix and the honest answer is “anyone the other account decided to let in.”

This post covers AWS cross-account data sharing as a set of failure families rather than a service tour. How the delegation model actually behaves, where encryption keys break or widen access without warning, who ends up owning the bytes after a handoff, why a copy is worse than a reference, and which guardrails catch the mistake you are eventually going to make. Real policy snippets and commands throughout, plus the denials that are hardest to read.

AWS cross-account data sharing is a delegation, not a permission

Start with the mechanic that everything else hangs off. For a request that crosses an account boundary, two separate accounts have to say yes. The resource-based policy in the account that owns the data has to allow the caller, and an identity-based policy in the calling account has to allow the action. Neither side alone is enough.

That is why “I added them to the bucket policy and it still fails” is such a routine ticket. The producer did their half. Nobody attached the matching IAM policy on the consumer side.

Now the part that bites. Look at how most cross-account bucket policies get written:

{
  "Sid": "AllowAnalyticsAccount",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:root" },
  "Action": ["s3:GetObject", "s3:ListBucket"],
  "Resource": [
    "arn:aws:s3:::shared-events-bucket",
    "arn:aws:s3:::shared-events-bucket/exports/*"
  ]
}

Despite how it reads, that :root suffix does not mean the root user. It means the account. You have not granted access to a principal at all. You have delegated the guest list to the administrators of account 111122223333, and they can hand that access to any principal they like, including a role somebody creates next quarter for a purpose you will never hear about.

Sometimes that delegation is exactly what you want, because the consumer team manages its own roles and you do not want to be in the loop for every change. Usually you want the narrower version:

{
  "Sid": "AllowAnalyticsRoleOnly",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:role/analytics-reader" },
  "Action": ["s3:GetObject", "s3:ListBucket"],
  "Resource": [
    "arn:aws:s3:::shared-events-bucket",
    "arn:aws:s3:::shared-events-bucket/exports/*"
  ],
  "Condition": {
    "StringEquals": { "aws:PrincipalOrgID": "o-abc123def4" }
  }
}

Both halves earn their place. The named ARN stops the wrong role in the right account. The organization condition stops the right role name in the wrong account, which matters if the statement is ever copied, templated, or widened by someone in a hurry. One protects against internal drift, the other against the policy escaping your organization entirely.

Be clear-eyed about the trade-off, though. Naming ARNs is precise and brittle: every new consumer role is a policy change, a pull request, and a review. Organization-wide conditions scale beautifully and narrow nothing inside a member account. If your organization contains twelve accounts and forty teams, aws:PrincipalOrgID on its own is a fairly wide door.

The grant that stops working after a rebuild

When you name a role ARN as a principal, IAM stores the role’s internal unique ID behind the scenes. Delete that role and the policy will start displaying an opaque AROA... string where the ARN used to be, because the ARN no longer resolves to anything. Recreate a role with the identical name and the access does not come back, because the new role has a new unique ID.

A terraform destroy followed by terraform apply on the consumer side reproduces this perfectly. If you are staring at a bucket policy that contains a principal starting with AROA, that is your answer. Rewrite the statement with the current ARN.

The encryption boundary nobody tests until it is live

Second failure family: the key. Encrypting shared data is the easy decision. What people miss is that a KMS key is a second, fully independent authorization surface, with its own resource policy, evaluated separately from the bucket.

• Downloading an object encrypted with SSE-KMS requires kms:Decrypt on the key.
• Uploading one requires kms:GenerateDataKey.
• A multipart upload requires both.

The AWS managed key for S3 cannot carry you here, because you cannot edit its key policy. Sharing KMS-encrypted objects across accounts means a customer managed key, with a statement naming the consumer:

{
  "Sid": "AllowAnalyticsRoleToDecrypt",
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::111122223333:role/analytics-reader" },
  "Action": ["kms:Decrypt", "kms:DescribeKey"],
  "Resource": "*",
  "Condition": {
    "StringEquals": { "kms:ViaService": "s3.us-east-1.amazonaws.com" }
  }
}

The kms:ViaService condition is cheap insurance. It restricts use of the key to requests that arrive through S3 in that region, so a leaked set of consumer credentials cannot call KMS directly and start decrypting ciphertext it obtained some other way. And the two-key rule applies to the key exactly as it does to the bucket: the consumer’s own IAM policy has to allow the KMS actions too.

Now the sharp edge, which is worth reading twice. If a KMS key is specified by alias rather than by fully qualified ARN, the alias is resolved in the requester’s account. In a cross-account write that means data landing in your bucket can end up encrypted under a key belonging to the writer. You own the object and cannot read it, and no error is raised at write time. Use full key ARNs in anything that crosses an account boundary.

Who owns the bytes after the handoff

Third family: ownership, and it depends entirely on which direction the data moves.

New buckets are created with Object Ownership set to Bucket owner enforced. ACLs are off, and the bucket owner owns every object written into the bucket regardless of who wrote it. That default removed the single most confusing trap in S3 sharing.

Buckets created before that default are the ones to check. If a bucket still sits on the Object writer setting, an object uploaded by another account is owned by that account, and you, holding the bucket, may not be able to read it. Nothing fails at write time. It fails later, when a job in your own account tries to read your own bucket and gets a 403.

Check and fix it with the ownership controls API:

aws s3api get-bucket-ownership-controls --bucket shared-events-bucket

aws s3api put-bucket-ownership-controls 
  --bucket shared-events-bucket 
  --ownership-controls '{"Rules":[{"ObjectOwnership":"BucketOwnerEnforced"}]}'

Order matters when you make that change on a live bucket. Once ACLs are disabled, any PUT that still carries an ACL is rejected with AccessControlListNotSupported. Requests that send no ACL, or send bucket-owner-full-control, keep working. So the sequence is:

1. Find every writer that sets an ACL, including old scripts using --acl and any SDK call passing an ACL argument.
2. Update those clients so they stop sending one.
3. Confirm from CloudTrail that ACL-bearing PUTs have stopped.
4. Flip the bucket to Bucket owner enforced.

Given a free choice of design, I would rather the consumer pull from a bucket I own than push into a bucket I do not. Pull keeps ownership, lifecycle rules, encryption configuration, and access logging in one account, under one team’s control. Push scatters them.

Copies you cannot recall

Fourth family, and the one with the longest tail. There is a large difference between granting a reference and shipping a copy. A reference can be revoked on a Tuesday afternoon. A copy is simply gone.

Three reference-shaped patterns are worth knowing before you reach for a bulk export.

S3 Access Points for object-level sharing

Instead of a bucket policy that grows a new statement every time a team asks for something, you write the bucket policy once to delegate to access points, then give each consumer their own access point with its own focused policy. The delegation is expressed with the s3:DataAccessPointAccount condition. Access points can also be owned by the consumer’s account, so they manage their own detail without touching your bucket policy.

The gotcha: when a request goes through an access point, both the access point policy and the underlying bucket policy have to allow it. A 403 here is easy to misdiagnose because the bucket policy looks correct and is correct. The access point is the layer that said no.

Lake Formation for table-level sharing

When the thing being shared is a table rather than a prefix, Lake Formation is the better fit, because it lets you grant on databases, tables, columns, and row filters instead of on object paths. Cross-account grants ride on AWS Resource Access Manager, so the recipient gets an invitation that a data lake administrator has to accept.

aws ram get-resource-share-invitations

aws ram accept-resource-share-invitation 
  --resource-share-invitation-arn arn:aws:ram:us-east-1:111122223333:resource-share-invitation/EXAMPLE

After the invitation is accepted, the shared database and tables are visible but still not queryable from Athena or Redshift Spectrum until someone creates a resource link, which behaves roughly like a symlink in the recipient’s Data Catalog. Missing resource links account for most of the “the share went through but I cannot see the table” messages.

One setting to check on the producer side is the cross-account data sharing version. Newer versions collapse many grants into far fewer RAM resource shares, which matters once you are sharing at any scale, and they allow granting directly to IAM principals in another account rather than only at account level. Read the current version with:

aws lakeformation get-data-lake-settings

When a copy really is the answer

Sometimes the partner genuinely needs the data on their own infrastructure: a warehouse you have no reach into, a VPS at a provider like Contabo or InterServer, or in the worst case an analyst’s laptop. That is a legitimate outcome, but be honest with yourself and with them that your controls stop at the handoff.

Practical hygiene for that case: scope the extract to the columns actually needed rather than the whole table, put an expiry date in the agreement rather than in your head, and write down what happens to the copy when the engagement ends. If the copy lives on machines you or the client control, verified erasure with something like O&O SafeErase reads a great deal better in an audit than “we deleted the folder.” And if the recipient has no AWS account at all, a time-limited presigned URL is usually a better answer than minting an IAM user with long-lived keys.

The guardrail that catches the mistake you will make

Everything above describes what a careful engineer does per grant. Guardrails are what happens when somebody is not careful, which over a long enough window includes all of us.

Resource control policies are the strongest lever if you run AWS Organizations. An RCP sets a ceiling on what any resource-based policy inside the organization can grant. The canonical statement is “no principal outside my organization may access S3 buckets in my accounts, regardless of what an individual bucket policy says.” That inverts the risk model: a careless bucket policy is no longer an exposure, it is just an ineffective statement.

Two limits are worth stating plainly. RCPs cover a specific set of services rather than everything in AWS, so check the current supported list rather than assuming coverage. And they constrain your resources, not a partner’s behavior once data has legitimately left. An RCP is a wall around your side of the fence.

Service control policies do the mirror-image job on the consumer side, limiting which external accounts your own principals are allowed to assume roles into. Account-level S3 Block Public Access remains a sensible floor underneath both.

For sharing with a third party rather than an internal account, the external ID is not optional in my book:

{
  "Effect": "Allow",
  "Principal": { "AWS": "arn:aws:iam::444455556666:role/vendor-connector" },
  "Action": "sts:AssumeRole",
  "Condition": {
    "StringEquals": { "sts:ExternalId": "a-value-the-vendor-generated-for-you" }
  }
}

The external ID exists because of the confused deputy problem. A vendor holds credentials that can assume roles across many customers, and without a per-customer secret in the condition, customer A can trick the vendor into pointing that access at customer B. If a vendor asks you to create a role for them with no external ID and no conditions at all, that is worth a conversation before it is worth a role.

Verifying that the grant is what you think it is

Reading policies is not the same as knowing what they permit. IAM Access Analyzer answers the question “what is currently reachable from outside my zone of trust” using automated reasoning rather than pattern matching, which is why it catches combinations a human reviewer skims past.

aws accessanalyzer create-analyzer 
  --analyzer-name external-access 
  --type ACCOUNT

aws accessanalyzer list-findings 
  --analyzer-arn arn:aws:access-analyzer:us-east-1:999988887777:analyzer/external-access

Scope the zone of trust to your organization where you have one, then archive the findings that represent intentional shares, so that a new entry in the list actually means something.

The same service exposes custom policy checks you can run in a pipeline before a policy change merges, including checks for public access, for specific actions being granted, and for whether an updated policy grants more than the one it replaces. One thing to understand about them: they evaluate the document you hand them and nothing else. They do not read your account state, so a public-access check can fail on a policy in an account that has Block Public Access enabled. For a merge gate that is correct behavior, because the policy should not be relying on a distant safety net.

Finally, CloudTrail tells you who actually used a grant, which is the input to the only cleanup that ever really works: delete the statements nobody has exercised. Routing that history into CloudWatch, or into a Grafana Cloud dashboard alongside the rest of your platform telemetry, turns “is this still needed” from an argument into a query.

Troubleshooting the denials that are hard to read

• You can list the bucket but every GET returns 403. That pattern points at KMS, not S3. Listing does not touch the key; reading does. Check kms:Decrypt on both the key policy and the consumer’s identity policy.
• 403 through an access point while the bucket policy is clearly correct. The access point has its own policy and it also has to allow the calling principal. Read that one next.
• AssumeRole is denied although the trust policy names the caller. Cross-account needs both halves: the caller’s identity policy must permit sts:AssumeRole against that role ARN. Then check the external ID value and any SCP or permissions boundary in the calling account.
• A principal in the policy displays as AROA followed by random characters. The referenced role was deleted. Recreating it under the same name will not restore access.
• A Lake Formation share appears in the catalog but queries fail. Usually a missing resource link in the recipient account, or a RAM invitation that was never accepted.
• Access worked yesterday and stopped today with nothing changed locally. Look at the organization layer before re-reading the bucket policy. A newly attached RCP or SCP denies in a way that no amount of resource-policy editing will fix.

Common mistakes

• Granting to an account principal when a role ARN would have done, then never revisiting it.
• Treating aws:PrincipalOrgID as least privilege. It narrows the organization, not the account.
• Sharing a bucket and forgetting the key, then debugging S3 for an hour over a KMS denial.
• Referring to KMS keys by alias in cross-account configuration instead of by full ARN.
• Letting a partner push into your bucket without confirming the ownership setting first.
• Exporting a full table when the consumer needed four columns, because the export was easier to write.
• Creating a vendor role with no external ID and no conditions because the vendor’s setup guide said to.
• Granting access with no expiry, no owner, and no note in the policy Sid explaining why it exists.

Best practices worth the effort

• Give every cross-account grant a descriptive Sid. It is the only free documentation slot IAM offers, and future-you will read it.
• Prefer one dedicated consumer role per relationship over a shared role used by several integrations. Revocation becomes a single delete.
• Pair a specific principal with a broad condition, rather than relying on either alone.
• Put policy checks in CI so a widened grant fails a pull request instead of a quarterly review.
• Keep an inventory of active shares with an owner and a review date, even if it is a table in a repository README.
• Enable an external access analyzer scoped to your organization and treat new findings as work, not noise.
• Watch cross-region and internet data transfer when a consumer sits somewhere unexpected. Cost visibility tooling such as Vantage or CloudZero will attribute it faster than reading a bill.

Frequently asked questions

What is the safest way to share an S3 bucket with another AWS account?

Name a specific role ARN in the bucket policy, add an organization or account condition alongside it, scope the resource to a prefix rather than the whole bucket, and make sure the consumer’s identity policy is equally narrow. If the bucket is encrypted with a customer managed key, grant on the key as well. If the list of consumers is going to grow, move to access points rather than growing the bucket policy.

Do I need both a bucket policy and an IAM policy?

For cross-account access, yes. The resource-based policy in the owning account and the identity-based policy in the calling account both have to allow the action. That is the single most common reason a correct-looking bucket policy still returns Access Denied.

Why do I get Access Denied when the bucket policy clearly allows the other account?

Work through it in this order: the consumer’s identity policy, then the KMS key policy if the objects are encrypted with a customer managed key, then any access point policy in the path, then organization-level SCPs and RCPs. Denials from the organization layer are the ones people waste the most time on, because nothing in either account’s policies looks wrong.

Can I share data with an account outside my AWS Organization?

You can, and organization conditions will not help you there. Use a named role ARN, an external ID on any assumed role, a tightly scoped resource, and a defined end date. Also confirm whether an RCP in your organization already blocks external principals for that service, because that is a common reason a partner grant refuses to work no matter how carefully it is written.

Should I use S3 Access Points or Lake Formation?

It depends on the unit of sharing. If the consumer wants objects under a prefix, access points are simpler and cheaper to operate. If the consumer wants a table, and especially if you need column or row filtering, Lake Formation is the right tool and access points would force you to fake fine-grained control with prefix layout. Plenty of environments run both.

How do I revoke cross-account access cleanly?

Remove the statement from the resource policy, remove the KMS key grant, and check for any access point, RAM share, or resource link that also carries permission. Then confirm with CloudTrail that the principal has stopped appearing. Revocation feels done long before it is done, which is exactly why an inventory of active shares pays for itself.

Does cross-account access change anything about cost?

The request charges land on the bucket owner by default, and data transfer depends on whether the consumer is in the same region. A consumer that quietly starts scanning a large prefix from another region is a billing event as much as a security one, which is a good argument for reviewing access patterns and spend together rather than in separate meetings.

The one thing to remember

Getting AWS cross-account data sharing right is less about knowing which service to reach for and more about noticing which grants are quietly larger than they read. An account principal delegates your guest list. An organization condition narrows the wrong dimension. A copy leaves your perimeter permanently and no policy edit brings it back.

Grant references rather than copies, name principals rather than accounts, remember that the key is a second door, and put something in place that watches the perimeter for you. The cross-account grant worth worrying about is never the one that failed. It is the one that worked, and kept working, long after anybody was still looking at it.

Need a second pair of eyes on your cross-account setup?

Most of the cross-account problems I see are not exotic. They are a policy written in a hurry two years ago that nobody has read since. If that sounds familiar, here is where I can help:

• Auditing existing bucket, key, and role trust policies to find grants that are wider than intended, and rewriting them to named principals with conditions.
• Designing a producer and consumer pattern for a specific data handoff, including whether access points, Lake Formation, or a plain scoped role is the right shape.
• Setting up IAM Access Analyzer with a sensible zone of trust and archive rules, so the findings list stays worth reading.
• Adding policy checks to your pipeline so a widened grant blocks a merge rather than surviving to production.
• Building an organization-level perimeter with resource control policies and service control policies without breaking the integrations you already depend on.
• Untangling stubborn cross-account denials, including the KMS, access point, and organization-layer cases that hide well.

Send me a bucket policy, a trust policy, or the CloudTrail entry for the denial you cannot explain, and I will tell you what it actually permits.

The post AWS Cross-Account Data Sharing Done Safely: Share Access, Not Copies appeared first on John Nessime.