So you open the audit table. And you find it records who requested a link to the document. Not who opened it. The actual downloads never touched your application at all, because you handed out a signed storage URL and the browser went straight to the bucket. Your log is a record of intentions.

That gap is typical of how property SaaS document storage goes wrong. The failures are rarely dramatic. Nobody dumps a bucket publicly. The guarantees you thought you had just turn out to be enforced in application code that ran once, long before the file was actually read. What follows is five failure families worth designing against when a platform stores tenant ID scans, signed leases, deposit certificates, right-to-rent evidence and referencing packs.

What’s actually in the pile

Before architecture, be honest about the contents. A property platform’s document store is at least four risk classes wearing the same file extension:

• Identity evidence. Passport and driving licence scans, visa documents, sometimes a selfie held next to the document. Highest blast radius, lowest genuine need for long retention.
• Contracts. Tenancy agreements, addenda, guarantor deeds, signed inventories. Legally driven retention, plus an integrity requirement: proving the file hasn’t changed matters more than hiding it.
• Financial and referencing material. Bank statements, payslips, credit reference reports. Sensitive, third-party sourced, and often covered by contractual restrictions from the referencing provider.
• Operational clutter. Inspection photos, safety certificates, maintenance invoices. Low sensitivity, high volume, and the thing that dominates your storage bill.

These want different retention clocks, different access rules and arguably different buckets. Treating them as one “documents” table is where most of the later pain originates.

Failure one: the access check and the access happen at different times

Presigned URLs are the right pattern. They keep the bucket private, take the transfer off your servers, and mean you never hand storage credentials to a browser. I reach for them first.

The problem is what a presigned URL is. It’s a request you already authorised, signed, and then gave away. Anyone holding it can replay it until it expires. Your permission logic ran once and never runs again. Which means all of these keep working after the fact:

• A letting agent who left the company yesterday and whose account you disabled this morning.
• A link forwarded into a WhatsApp group, pasted into a support ticket, or sitting in browser history on a shared office machine.
• A document the tenant asked you to delete, if the object is still there behind a delete marker.

Make the URL die when the session does

Amazon’s own guidance is clearer on this than most people realise. A presigned URL created with temporary credentials expires when those credentials expire, even if you asked for a longer expiry. It also stops working if the signing credential is revoked, deleted or deactivated.

That’s a lever, not a footnote. Sign with a long-lived IAM user key and the seven day maximum the SDK allows is a real seven days. Sign inside a short-lived assumed-role session and the URL cannot outlive that session, whatever number you passed in. So: never sign document URLs with static credentials. Assume a role per request, tag the session with the tenant, sign inside it.

import boto3

sts = boto3.client("sts")

# The tag key here must match the principal tag key used in the role's policy.
session = sts.assume_role(
    RoleArn="arn:aws:iam::111122223333:role/TenantDocumentAccess",
    RoleSessionName="doc-download",
    Tags=[{"Key": "TenantID", "Value": tenant_id}],
    DurationSeconds=900,
)["Credentials"]

s3 = boto3.client(
    "s3",
    aws_access_key_id=session["AccessKeyId"],
    aws_secret_access_key=session["SecretAccessKey"],
    aws_session_token=session["SessionToken"],
)

url = s3.generate_presigned_url(
    "get_object",
    Params={
        "Bucket": "prop-docs",
        "Key": f"tenants/{tenant_id}/identity/{doc_id}.pdf",
        "ResponseContentDisposition": 'attachment; filename="id-document.pdf"',
        "ResponseContentType": "application/octet-stream",
    },
    ExpiresIn=120,
)

Two things are doing work. DurationSeconds caps the real lifetime of the link regardless of ExpiresIn. And ResponseContentDisposition overrides how the object is served on this request, which matters for the serving problem further down.

The trade-off is real: an STS call per download adds latency, adds a dependency, and STS has request limits you can hit at volume. AWS’s prescriptive guidance says as much. For identity documents I’d pay that cost. For inspection photos I wouldn’t.

Log where the read actually happens

Your application log records issuance. To record retrieval you need logging at the storage layer: CloudTrail data events for object-level operations, or S3 server access logging. Data events are billed per event, so scope them to the identity and contract prefixes rather than the bucket holding thousands of inspection photos.

Failure two: isolation that lives in your ORM instead of your IAM policy

Ask most teams how tenant isolation works and you get a description of a WHERE clause. The key is built as tenants/{tenant_id}/... and the tenant ID comes from the session. Fine until one code path takes it from a request parameter, or a background job iterates without scoping, or someone writes an admin endpoint in a hurry.

Move the boundary below your code. With attribute-based access control the tenant becomes a session tag and the IAM policy interpolates it into the resource ARN. One role serves every tenant, and the policy count doesn’t grow as you onboard.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadWriteOwnTenantPrefix",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::prop-docs/tenants/${aws:PrincipalTag/TenantID}/*"
    },
    {
      "Sid": "ListOnlyOwnTenantPrefix",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::prop-docs",
      "Condition": {
        "StringLike": {
          "s3:prefix": "tenants/${aws:PrincipalTag/TenantID}/*"
        }
      }
    }
  ]
}

Two statements because S3 splits the problem. The first governs the object, where the tenant prefix is part of the resource ARN. The second governs listing, a bucket-level action, so the prefix has to be constrained through a condition instead. Skip it and a tenant can enumerate every key in the bucket even though they can’t read any of them, and key names in this domain leak plenty on their own.

The failure mode worth internalising: if a code path forgets the session tag, ${aws:PrincipalTag/TenantID} resolves to nothing and the statement doesn’t match. That’s a deny, which is what you want. But it only holds if the role has no broader fallback statement. One well-meaning s3:GetObject on the whole bucket, added to unblock a migration script, quietly turns the scheme into decoration.

When to use a bucket per tenant instead

A bucket or an account per tenant is a genuinely harder boundary and I wouldn’t dismiss it. With a few dozen large agency customers it’s defensible and makes offboarding trivially provable. It stops being defensible once you onboard self-serve landlords, because bucket and account counts run into service quotas and the operational surface grows with every signup. Pick on tenant count, not on which sounds more secure. And note that ABAC is only as trustworthy as the tenant claim: if the caller can influence the value reaching the tagging call, you’ve built an elegant no-op.

Failure three: the upload and serve path

Uploads here come from people you have no relationship with yet. Prospective tenants, guarantors, contractors. That’s an untrusted channel by definition, and it produces two separate problems.

Constrain the upload itself

A presigned PUT URL lets the holder write whatever they like to that key, at whatever size. Presigned POST is the better tool for public-facing uploads: the signature covers a policy document, and the policy carries conditions the browser cannot alter.

post = s3.generate_presigned_post(
    Bucket="prop-docs",
    Key=f"tenants/{tenant_id}/inbox/{upload_id}",
    Fields={"Content-Type": "application/pdf"},
    Conditions=[
        {"Content-Type": "application/pdf"},
        ["content-length-range", 1, 15728640],
    ],
    ExpiresIn=600,
)

The content-length-range condition is the one people skip, and it’s the one that stops a stranger using your referencing portal as free storage for a 40 GB file. A declared Content-Type is a claim, not a fact, so land uploads in a quarantine prefix, verify the actual bytes, and only then move them to the tenant prefix.

Scan on that path too. ClamAV in a container is the zero-cost option and covers known malware; GuardDuty Malware Protection for S3 and hosted APIs such as Cloudmersive trade money for not maintaining it. What you’re protecting is mostly the letting agent who opens the attachment on a Windows machine, so this is a control for your customers rather than your servers.

Never serve tenant documents from your application origin

This is the one that turns a document store into an account takeover. SVG and HTML are formats browsers execute. If a file uploaded by a prospective tenant is served inline from the same origin as your app, any script inside it runs with your application’s cookies and session context. A long stream of real advisories follows exactly this shape, particularly in self-hosted deployments where the object store sits on the app’s own domain.

Three controls, and they stack rather than substitute:

1. Force download. Serve with Content-Disposition: attachment and X-Content-Type-Options: nosniff. Set these as object metadata at upload time and override them per request. The override protects links you mint; the stored metadata protects anything that bypasses your minting path.
2. Use a separate origin. Put user content on a distinct domain, not a subdomain of the app. GitHub’s separate apex for user content is the reference implementation. Same-origin policy then contains the blast radius if control one is missed somewhere.
3. Normalise where you can. Rendering an uploaded ID to a flattened image server side removes the executable content entirely. It costs CPU and fidelity. For identity documents, where you rarely need the original bytes back, it’s the right trade.

Failure four: deleting the document doesn’t delete the document

A tenant asks you to erase their passport scan. Your app sets a flag, the file disappears from the UI, and you reply confirming deletion. Then count the copies that still exist.

• The previous object versions, if versioning is on. A delete without a version ID just writes a delete marker.
• Cross-region replicas with their own versioning and lifecycle rules, plus any backup or snapshot taken before the deletion.
• Derivatives you generated and forgot: thumbnails, page previews, OCR text in a search index. Run a passport through Amazon Textract and the extracted fields are a copy of the sensitive data in a different shape.
• Copies that left the platform. A document attached to a CRM record in something like GoHighLevel, forwarded by email, or downloaded to a letting agent’s laptop.

The ICO’s position is that a valid erasure request extends to backups as well as live systems, and that where backup data cannot be immediately overwritten it must be put beyond use: isolated, unused for anything else, deleted when that backup cycles. Workable, but only if you know which backup sets contain documents and how long they cycle. If you can’t answer that, you can’t answer the request.

Crypto-shredding, and what it actually costs

The one technique that makes deletion provable across copies you can’t reach is a per-tenant KMS key. Encrypt that tenant’s objects under their own customer managed key; when they offboard, schedule the key for deletion. Every ciphertext copy anywhere, backups included, becomes undecryptable. You delete the data by deleting the ability to read it. The honest costs:

• KMS bills per key per month plus per request, so per-tenant keys are a real line item. Check current rates and quotas first: there’s an account limit on customer managed keys that a self-serve signup flow will eventually meet.
• Key deletion has a mandatory waiting period. Crypto-shredding is not instant, and your erasure response has to say so.
• It’s all-or-nothing per key. If a tenant asks you to erase one document rather than offboard, the key doesn’t help.

Object Lock fights erasure, on purpose

S3 Object Lock is tempting for signed tenancy agreements, because write-once storage is a strong integrity story. But compliance mode means nobody, root included, can delete the object until retention lapses. That is the feature. It is also a direct collision with an erasure request, and you cannot engineer your way out of it afterwards.

My line: Object Lock on executed contracts, where a statutory retention basis overrides erasure, and never on identity documents or referencing material. Different buckets, different rules, decided before anything is written. For per-document erasure elsewhere, lifecycle rules that expire noncurrent versions on the sensitive prefixes are worth setting up before you need them.

Failure five: storing what you never needed to store

The cheapest control on this list is not storing the file. The ICO is clear that UK GDPR sets no fixed retention periods and puts the burden on you to justify how long you keep personal data in identifiable form. That’s a design instruction, not a legal footnote.

For identity documents, ask what your platform needs once the check is done. Usually it’s the result, not the evidence: document type, issuing country, expiry date, a check reference, who performed it and when. If your customer’s regulator requires the image itself, that’s a retention rule with a stated basis, which is a different conversation from keeping it because deleting felt risky.

Practical version: give every document class a retention clock wired to a business event rather than upload date, and let lifecycle rules do the deleting. Tenancy end plus a defined period. Check completion plus a defined period. Not “whenever someone remembers.”

Troubleshooting the breakages you’ll actually hit

Everything broke after someone enabled S3 Bucket Keys

This one catches teams who did the sophisticated thing. If your IAM or KMS key policies use the object ARN as the encryption context to scope access, enabling an S3 Bucket Key changes that context to the bucket ARN and those policies stop matching. AWS documents it explicitly as something to fix before flipping the setting.

The quieter side effect: KMS CloudTrail events then log the bucket ARN instead of the object ARN, and there are far fewer of them because S3 caches the bucket-level key. If you were treating KMS CloudTrail volume as a proxy for document access, that signal degrades without anything appearing to break. Bucket Keys are still worth enabling for the request cost reduction. Move the conditions first, then re-check what your dashboards count.

The presigned URL expires far sooner than requested

Almost always the signing credential is a role session shorter than the expiry you asked for. This is documented behaviour, not a bug. Either raise the session duration within the role’s maximum, or accept the shorter window as the security property it is.

Listing works, reading returns Access Denied

Classic split between the bucket-level and object-level statements. Check both exist and reference the same tag key spelling. Session tag keys are case sensitive, and a tenantId in the code against a TenantID in the policy fails silently as a non-match.

Access denied only for encrypted objects

An SSE-KMS object needs two grants, not one. The S3 action on the object, and kms:Decrypt on the key it was encrypted under. In a per-tenant key setup that second grant is usually also tag-conditioned, so a missing or mismatched tag on the key itself produces a denial that looks like an S3 problem and isn’t.

Common mistakes

• Signing document URLs with a long-lived IAM user access key, which makes every link independently long-lived.
• Granting the tenant role a broad bucket-wide read statement alongside the tag-scoped one, usually to unblock a one-off script that then stayed.
• Serving user uploads from a subdomain of the application and assuming that counts as origin separation. It doesn’t for cookie scope.
• Enabling versioning for safety and never adding a noncurrent version expiry rule, so every deletion is reversible forever.
• Building deletion that handles the object but not the OCR output, the thumbnail, or the search index entry.
• Putting Object Lock on the bucket that also holds identity documents.

Design rules worth holding to

1. Separate buckets by risk class, not by feature. Identity, contracts, financial, operational. Different encryption, retention, logging and lock settings follow naturally from that split.
2. Enforce isolation in IAM, with application checks as defence in depth rather than the boundary itself.
3. Sign short, sign from a session. Minutes, not days, and always from assumed-role credentials so revocation actually revokes.
4. Deny unencrypted writes at the bucket policy, using the server-side encryption request headers as conditions, so a misconfigured client fails loudly instead of writing plaintext.
5. Give every document class an expiry clock tied to a business event, implemented in lifecycle rules rather than a cron job someone maintains.
6. Write the deletion runbook before launch, listing every place a copy lands. If you can’t enumerate them, that’s the finding.
7. Test cross-tenant access in CI. A negative test that asserts tenant A’s session cannot read tenant B’s key is worth more than any amount of policy review.

Frequently asked questions

How long should a presigned URL for a tenant ID document last?

Short enough that the user clicks it immediately. A couple of minutes covers a click-to-download flow. If the UI needs longer, re-mint on demand rather than extend. And remember the ceiling is set by the signing credential, so a short assumed-role session enforces the limit even if a later code change passes a bigger number.

Do I need AWS, or will Backblaze B2 or Cloudflare R2 do?

They all speak the S3 API and presigned URLs work on all of them, so the core patterns transfer. Wasabi, Backblaze B2, Cloudflare R2 and DigitalOcean Spaces are reasonable for the operational tier, and R2’s egress model is attractive if agents download a lot. They differ in the depth of the IAM story: session-tag ABAC and per-tenant KMS keys are AWS-specific. If isolation and crypto-shredding are hard requirements, keep the sensitive prefixes on S3 even when the bulk photos live somewhere cheaper.

Can I store scanned passports on a normal VPS instead?

You can, and plenty of platforms run the app tier on a VPS from Contabo or InterServer while keeping documents in object storage. What you lose on local disk is the deletion story and the access log. Proving an image is gone from a filesystem, its backups and its snapshots is far harder than scheduling a KMS key for deletion. If documents stay local, budget real effort for secure disposal, including on staff machines where tools like O&O SafeErase have a place.

What about e-signed tenancy agreements?

With a provider like DocuSign, the executed document and its audit trail live there as well as in your platform. That’s a second copy under a second retention regime, and it belongs in your deletion runbook and data map. It also strengthens your integrity story, which is a fair reason to skip Object Lock on contracts you retrieve from the provider.

Should the CDN cache tenant documents?

Not the sensitive classes. A CDN in front of private documents adds a cache that can serve content after your access decision changed, and header handling at the edge is a common source of the inline-rendering bug. CloudFront with origin access control and signed URLs is legitimate if you need the performance, but for identity documents I’d go straight to the bucket. For inspection photos, cache away.

How do I prove any of this to a prospect’s security team?

Artefacts, not assertions. The bucket policy, the ABAC policy, CI output showing cross-tenant denial, a sample of storage-layer access logs, and a retention schedule with a clock per document class. That set answers most questionnaires without a call.

The one thing worth remembering

Good property SaaS document storage is mostly about closing the gap between when you check access and when access happens. Every failure family above is a version of that gap: the presigned link that outlives the permission, the isolation enforced once in code and never at the storage layer, the deletion that updates a flag while the bytes sit behind a delete marker.

Close it by pushing the decision down: into IAM policies that evaluate on every request, into credentials short enough that revocation is real, into lifecycle rules that delete without anyone remembering to. Then the answer to “show us who read this tenant’s ID document” is a query, not a project.

Need a second pair of eyes on your document storage?

Most of what I do on property and lettings platforms is this exact work. Things I’d help with here:

• Reviewing bucket and IAM policies for cross-tenant read paths, including the ones added to unblock a migration and never removed.
• Moving tenant isolation from application code into session-tag ABAC, with a CI test that fails the build on cross-tenant access.
• Rebuilding the upload path with presigned POST conditions, quarantine, content verification and scanning before anything lands in a tenant prefix.
• Fixing the serving path so uploaded files can’t execute against your origin, including header handling through a CDN.
• Designing retention clocks and lifecycle rules per document class, plus a deletion runbook that names every copy including derivatives and third-party systems.
• Setting up storage-layer access logging scoped to the sensitive prefixes so you can actually answer an access request.

If it’s useful, send me your bucket policy and the IAM policy attached to your document role, with the account IDs redacted, and I’ll tell you what a determined tenant could reach.

The post Property SaaS Document Storage: The Five Failures That Show Up in an Audit appeared first on John Nessime.

Nobody looks at the old link. If it was signed with a long expiry and mailed out, it is still live: in the client’s inbox, in the inbox of the bookkeeper on the CC line, and in whatever mail archive that firm runs. It keeps working for anyone holding the URL until the clock runs out, because a presigned S3 URL is a bearer token. AWS says exactly that in its own documentation. The URL does not know who you are. It knows that you have it.

That is the shape of most document vault failures at a small accounting firm. Not a wide-open bucket. Not a stolen partner password. An ordinary delivery mechanism that leaks quietly and writes no log line that looks wrong six months later.

This post covers how to build a secure client document vault for an accounting practice: what the regulation actually demands, the six failure families that decide whether it holds, and the control that closes each one. The examples are AWS because that is where most of these land, but the failure families are platform independent.

Let the regulation write your requirements

Most firms treat compliance as paperwork done after the build. That is backwards. For a US accounting or tax practice, the FTC Safeguards Rule at 16 CFR Part 314 is the closest thing you will get to a free requirements document.

It treats tax preparers, bookkeepers and accounting firms as financial institutions under the Gramm-Leach-Bliley Act. Size does not get you out of it. The parts that translate directly into architecture:

• Access controls limiting each user to the customer information they need, and limiting customers to their own information.
• Encryption in transit over external networks and at rest, with compensating controls only where encryption is infeasible and your Qualified Individual signs off in writing.
• Multi-factor authentication for any individual accessing any information system, subject to the same written-approval exception.
• Secure disposal, with an outer bound of two years after last use unless retention is required by law or necessary for legitimate business purposes.
• Logging of authorized user activity, plus a named Qualified Individual who owns the program.

One more clause should shape your design more than any other. The Rule requires notifying the FTC within thirty days of discovering a security event involving the unencrypted information of at least five hundred consumers, and it counts encrypted information as unencrypted if the key was accessed by an unauthorized person.

Read that twice. Encryption only helps at the breach-analysis stage if the key was genuinely out of reach. That is the entire argument for treating key custody as a control separate from storage.

Separately, IRS Publication 4557 expects paid preparers to keep a Written Information Security Plan, and PTIN renewal asks you to confirm you have one. Treat the WISP as the document describing the vault you built, not a substitute for building it.

Failure one: the link that outlives the engagement

Presigned URLs are the sensible answer to “let the client download this without an AWS account”. The problem is their expiry semantics, which almost nobody has read.

Two clocks apply and the shorter wins: the expiry you asked for, and the lifetime of the credential you signed with. Long-lived IAM user credentials under Signature Version 4 reach seven days. A role session from AssumeRole dies when the session ends, one hour by default, regardless of what you requested. An EC2 instance profile is bound by metadata credential rotation. The console caps at twelve hours.

So the two symptoms are opposites and both are common. Links dying early is a credential problem. Links living too long is a policy problem, and that is the one that hurts.

The fix is not a smaller number. It is changing what goes in the email.

1. Email a notification, not the file link. The client logs in with MFA and the portal mints a URL at click time.
2. Keep that expiry in minutes. The URL only has to survive a redirect and a download.
3. Scope every URL to one object and one verb.
4. If you truly need a long-lived share, use CloudFront signed URLs or signed cookies, where you can revoke by rotating the key group without touching the object.

# Sign a download URL valid for five minutes.
# --expires-in takes seconds; the default is 3600.
aws s3 presign s3://firm-client-vault/clients/acme-llc/workpapers.pdf 
    --expires-in 300

Run that under an assumed role, wait past the session expiry, and watch it fail well before five minutes. That is behaviour worth seeing once in a test rather than discovering during an incident review.

Failure two: everyone inside the firm sees everything

This shows up in nearly every small-firm build, and the reasoning is always the same. Nine people, everyone helps with everything during filing season, and compartmentalising access is friction exactly when friction is expensive.

The case for flat access is real, so say it out loud: a scoped model people fight gets worked around, and a workaround is worse than the flat model you started with. If your access control makes a senior accountant email a PDF to themselves at 11pm, you have made things less safe.

So scope by engagement rather than by person, and automate the grant so nobody files a request. Make the key prefix the boundary, then grant the prefix instead of the bucket. Two statements, two jobs: the first controls what someone can enumerate, the second what they can fetch.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ListOnlyThisClient",
      "Effect": "Allow",
      "Action": "s3:ListBucket",
      "Resource": "arn:aws:s3:::firm-client-vault",
      "Condition": {
        "StringLike": { "s3:prefix": "clients/acme-llc/*" }
      }
    },
    {
      "Sid": "ReadWriteThisClient",
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:PutObject"],
      "Resource": "arn:aws:s3:::firm-client-vault/clients/acme-llc/*"
    }
  ]
}

Without the prefix condition on ListBucket, a scoped user still gets a full inventory of your client list. Client names are confidential in this business. A directory listing that reads like a book of business is a disclosure even when no file opens.

The other half of this family is standing administrative access. Somebody has to fix things, so somebody ends up with a permanent s3:* on the whole bucket. That principal is the highest-value target in the firm and usually has the weakest MFA, because it is a service account nobody wants to touch.

Replace it with an assumable break-glass role: MFA required, short session, and an alert on every assumption. The alert is the point. Emergency access used twice a year is fine. Used weekly, it means your normal path is broken.

Failure three: encrypted at rest, key sitting right next to it

Default bucket encryption gets ticked in week one and forgotten. It satisfies the letter of “encrypted at rest” and protects against exactly one threat: somebody walking out of a data centre with a disk. That is not your threat.

Go back to the notification clause. If an attacker reaches a principal that can both read the object and use the key, your data is unencrypted for reporting purposes. The question is not “was the bucket encrypted”. It is “could whoever got in also use the key”.

• Use a customer managed KMS key. Only that gives you a key policy you control.
• Name the principals allowed to call Decrypt, and keep that list shorter than the list allowed to call GetObject. Two different lists means an attacker needs two wins.
• Use encryption context so grants can be conditioned per client, and so CloudTrail entries are readable later.
• Enable S3 Bucket Keys to cut KMS request volume. KMS bills per API call, so a vault serving many small reads generates a surprising line item.

Encryption in transit needs its own control, because IAM policies say nothing about the connection. A bucket policy does:

{
  "Sid": "DenyPlaintextTransport",
  "Effect": "Deny",
  "Principal": "*",
  "Action": "s3:*",
  "Resource": [
    "arn:aws:s3:::firm-client-vault",
    "arn:aws:s3:::firm-client-vault/*"
  ],
  "Condition": {
    "Bool": { "aws:SecureTransport": "false" }
  }
}

An explicit deny beats every allow, including one you add by mistake later. Five lines that permanently remove a category of misconfiguration, which makes it the best value item on this page.

Failure four: retention rules that fight your disposal duty

Here is the trap, and it is why I would not let an accounting firm switch on Object Lock without a conversation first.

Object Lock has two retention modes. In governance mode, a principal holding s3:BypassGovernanceRetention can remove or shorten the lock. In compliance mode, nobody can. Not an administrator, not the account root user. AWS is explicit that the only way to delete a compliance-locked object before its retention date is to close the AWS account.

Set that against the Safeguards Rule’s two-year outer bound on disposal of information you no longer need. Apply a blanket seven-year compliance-mode retention to everything landing in the vault and you have written a promise you cannot revoke, on documents you may later be obliged to destroy.

1. Classify by document type before client. Filed returns, engagement letters and supporting workpapers have real statutory retention arguments. Raw intake uploads and scratch scans usually do not.
2. Default to governance mode. It stops accidental and casual deletion, which is what you need, while keeping a documented and audited escape hatch.
3. Give s3:BypassGovernanceRetention to almost nobody and alert on every use through CloudTrail.
4. Reserve compliance mode for records where an external mandate genuinely requires immutability and you chose the period deliberately.
5. Use legal hold, not a retention period, for litigation. A legal hold has no end date and stays on until explicitly removed, which is right when you cannot know how long a matter runs.

Two details catch people out. Object Lock requires versioning and normally has to be enabled at bucket creation, so it is a design decision rather than a retrofit. And lifecycle expiration will not delete a version still under retention, so lifecycle and Object Lock are two layers that must agree, not alternatives.

# What lock configuration is this bucket actually running?
aws s3api get-object-lock-configuration --bucket firm-client-vault

# Freeze one object version for a matter with no known end date.
aws s3api put-object-legal-hold 
    --bucket firm-client-vault 
    --key clients/acme-llc/2023/return.pdf 
    --legal-hold Status=ON

For deletion itself, crypto-shredding is worth knowing. If a client’s documents sit under a per-client KMS key, scheduling that key for deletion renders every object unreadable without touching the objects. Clean, fast, provable, and irreversible once the waiting period ends, so it belongs behind two-person approval rather than in a script.

Failure five: no evidence when somebody asks

Every control above is worth nothing if you cannot show it held. The Rule expects logging of authorized user activity and an annual written report from the Qualified Individual.

The trap is that CloudTrail does not record S3 object-level reads and writes by default. Management events cover things like changing a bucket policy. Data events cover the actual GetObject calls, and they are opt-in and billed per event. Firms enable CloudTrail, see a trail in the console, assume they have a record of file access, then find during an incident that they have nothing at object level.

Turn on data events for the vault bucket. At firm scale the volume is manageable, and it is the difference between “we believe two people opened that file” and being able to show it.

Then decide where logs live. Access records for client financial data are themselves sensitive, so keeping them in the same account under the same administrators defeats the purpose. A separate logging account, or at minimum a separate bucket with its own key and retention, is the shape you want. If you already run Grafana Cloud or similar for infrastructure monitoring, pointing vault access logs there gives you dashboards and alerting without standing anything new up.

The alerts worth having are boring and few: any use of the break-glass role, any change to the bucket or key policy, any bulk download outside normal working patterns, and any failed decrypt. Four alerts that fire almost never beat forty that everyone mutes.

Failure six: the two ends of the pipeline

A vault only protects documents that are in it. The two ends are where firms quietly undo everything.

Intake

If clients can still email a W-2 as an attachment, they will, because it is easier. Each attachment becomes customer information sitting in a mailbox, a sent-items folder and a mail archive, each with a different retention policy and access model than your vault.

A policy memo will not fix this. Make the vault path faster than the email path: an upload link that works on a phone, no account creation for a one-off drop, and a mail rule that returns certain attachment types with the upload link instead. Then sweep the mailboxes you already have, because that backlog is real customer information and it counts.

Offboarding

Seasonal staff are normal here, so offboarding happens in batches and gets rushed. Disabling the SSO login is the part everyone does. The parts that get missed:

• Active role sessions and refresh tokens, which survive the account being disabled until they expire on their own.
• Presigned URLs that person generated, which stay valid because they are bound to the signing credential, not the login session.
• Named principals still sitting in the KMS key policy.
• Local copies on laptops and external drives. When hardware is retired, a certified erase tool such as O&O SafeErase is the difference between disposal and hoping.

The same applies to service providers. The Rule makes you responsible for selecting providers capable of maintaining appropriate safeguards and holding them to it contractually. If an outsourced bookkeeping team touches your document workflow, that relationship is in scope, and direct network access means they need MFA too. Cloudflare Zero Trust or a business VPN such as NordLayer in front of the portal gives you somewhere to enforce that without rebuilding the application.

Troubleshooting a secure client document vault

• Links expire far earlier than the expiry you set. You signed with temporary credentials that expired first. Sign from a principal whose credential outlives the link, or shorten the link and mint on demand.
• Uploads fail with access denied despite an allow on PutObject. With a customer managed key, both the IAM policy and the key policy must permit the caller. Check kms:GenerateDataKey, not just S3 actions.
• A file will not delete and no policy seems to block it. Look for Object Lock retention or a legal hold on that version. They are independent and either one alone blocks the delete.
• CloudTrail shows the bucket being created but no file access. Data events are not enabled. Management events will never show GetObject.
• A scoped user can see every client folder name. The prefix condition is missing from ListBucket, or was applied to the object ARN instead of the bucket ARN.
• Lifecycle rules appear to do nothing. Versions under retention are not expired by lifecycle. Confirm lock state before assuming the rule is wrong.

Common mistakes, and what to do instead

• Emailing the download link. Email a notification that a document is waiting, and mint the link after the client authenticates.
• Treating default bucket encryption as finished. Use a customer managed key and keep the decrypt list shorter than the read list.
• Granting bucket-wide list permissions. Condition ListBucket on the client prefix, or the roster leaks even when no document opens.
• Applying compliance-mode Object Lock broadly because it sounds safer. Write the retention schedule per document class first, get partners to sign it, then configure to match.
• Keeping a standing full-access principal. Replace it with an MFA-gated break-glass role that alerts on every assumption.
• Building a portal and leaving email intake open. Both channels then carry client data and only one is governed. Close the mail path and sweep the backlog.
• Storing access logs beside the data they describe. Ship S3 data events somewhere with different administrators and different keys.
• Never testing any of it. Verify a restore and a scoped-access denial quarterly. An unverified control is an assumption.

Frequently asked questions

Does a small accounting firm really fall under the FTC Safeguards Rule?

Almost certainly, if it prepares returns or handles client financial data in the United States. The Rule’s definition of a financial institution covers activities financial in nature, and the FTC’s own examples include tax preparation firms. A narrow exception at 16 CFR 314.6 covers institutions holding information on fewer than five thousand consumers, but it exempts only four specific provisions. Encryption, MFA, access controls and disposal still apply. Confirm your own position with counsel, not a blog post.

Should I build this or buy a client portal product?

Buy, if the product genuinely does what you need. A firm with no engineering capacity is safer on a maintained product than a bespoke stack nobody patches. Build when you need controls the product will not give you: per-client key isolation, your own retention logic, or integration with a pipeline you already run. The honest test is whether someone will still own the system in a year.

Can I self-host a secure client document vault instead of using AWS?

Yes. Nextcloud or similar on a well-run VPS from a provider such as InterServer or Contabo can meet the same requirements, and it is often simpler to reason about. What you take on is patching, backups, TLS renewal and log retention. The regulation does not care where the data sits. It cares that the controls exist and that you can show they held.

How long should client documents be retained?

There is no single number, which is why a schedule beats a default. Tax and professional standards drive minimums for filed returns and engagement documentation, and those vary by jurisdiction and engagement type. The Safeguards Rule pushes the other way. Write both directions down per document class, then configure to match.

Is a presigned URL safe to send to a client?

It is safe to use, not safe to send. Treat it as a bearer token with a short life, minted after the client authenticates and consumed immediately. What travels over email should be a notification, not the credential.

Does encrypting the bucket mean I avoid breach notification?

Only if the key stayed out of the attacker’s hands. The Rule treats encrypted customer information as unencrypted where the key was accessed by an unauthorized person. That is the whole argument for keeping key permissions narrower than storage permissions.

What is the minimum viable version for a two-person practice?

One bucket with per-client prefixes, a customer managed key, an explicit deny on non-TLS transport, MFA on every human account, short mint-on-click links, S3 data events enabled, and a written retention schedule. That is a weekend of work and it closes five of the six failure families above.

The one thing worth remembering

A secure client document vault is not defined by the storage you picked. It is defined by how narrow the path is between an attacker and a readable file, and by your ability to prove afterwards how many people walked it.

Most firms get the storage right and lose at the edges: a link that outlives the engagement, a key policy as wide as the bucket policy, a retention setting that cannot be undone, an audit trail nobody turned on. Fix those four and the rest is ordinary engineering.

Where I can help

I work with small firms and the MSPs that support them on exactly this kind of build. Things I take on regularly:

• Designing and building the vault itself: bucket layout, per-client prefix isolation, KMS key policy, and the portal glue that mints short-lived links.
• Reviewing an existing setup against the Safeguards Rule’s technical elements and producing a gap list your Qualified Individual can work from.
• Untangling retention: turning a vague policy into a per-document-class schedule, implemented with lifecycle rules, Object Lock and legal holds that do not conflict.
• Standing up the evidence layer: S3 data events, a separate logging account, dashboards, and the small set of alerts worth waking up for.
• Closing the email intake path without making life harder for clients, including mobile upload flows and a sweep of the historic mailbox backlog.
• Offboarding and access reviews: break-glass roles, session revocation, key policy hygiene, and a repeatable quarterly check.

If you want a second opinion, send the thing itself: a bucket policy, an IAM policy, a CloudTrail event selector, or a description of how documents reach your clients today. It is much easier to say something useful about a real configuration than about a general question.

The post Building a Secure Client Document Vault Your Accounting Firm Can Defend appeared first on John Nessime.

That is the failure mode that actually bites when you build a medical lab data pipeline on AWS. Not the bucket you thought about. The seven places PHI ended up because a payload got logged, an error got queued, or a filename contained a medical record number.

This post walks through the design boundary by boundary: where lab data crosses from one trust zone into another, what leaks at each crossing, and which control closes it. It is aimed at engineers who have been handed an HL7 feed and a compliance checklist and told to make them meet. I am an engineer and not your compliance counsel, so treat the regulatory references here as pointers to read the primary text yourself.

Why laboratory data is a harder pipeline than it looks

Lab data has three properties that most ETL work does not.

First, results get corrected. A preliminary result goes out, the analyzer is recalibrated, and a corrected result follows hours or days later. Your pipeline is not append-only in the way a clickstream is. It has to carry amendment semantics all the way through, and an analytics table that silently keeps the first value is worse than no table at all.

Second, the wire format is old and the transport is older. Most laboratory information systems still emit HL7 version 2 messages, pipe-delimited segments carried over the Minimum Lower Layer Protocol, which is framed bytes over a raw TCP socket. AWS documents MLLP as the common transport standard for HL7 v2 interoperability in its Healthcare Industry Lens. It has no built-in authentication and no built-in encryption. Whatever security it has, you bolt on around it.

Third, the identifiers are load-bearing. Accession numbers, specimen IDs and medical record numbers are how the lab reconciles anything, so PHI is not a column you can drop. It is structural, and it will try to escape into every metadata surface you own.

Boundary one: getting messages out of the lab

The instrument or LIS sits on a network you probably do not control, run by a vendor who may not let you touch the configuration. That constrains the design more than anything on the AWS side.

You get three realistic shapes, and the LIS vendor usually picks for you.

• Live MLLP over a private link. An interface engine in the lab holds the TCP session and forwards over a Site-to-Site VPN or Direct Connect into a private subnet. Lowest latency, highest operational burden, because a dropped socket is a clinical incident and not a retry.
• Batch file drop over SFTP. The LIS writes flat files on a schedule and something picks them up. Far easier to reason about, far easier to audit, and acceptable whenever the downstream use is analytics rather than result delivery.
• An outbound API push. Newer systems will POST to an endpoint you expose. Best case, because you control authentication and you get a response code back, but rarest in practice.

Whichever shape you get, the same rule applies at this boundary: terminate the untrusted protocol as early as possible and convert to something you can authorize. A raw MLLP socket carries no identity. The moment those bytes land, wrap them in a message you can attribute, sign, and trace.

Two things go wrong here. First, people treat a network tunnel as authentication. A VPN tells you the packets came from the lab’s network. It does not tell you which system sent them, or that the sender was authorized to send that patient’s results. Second, people assume consumer VPN products fill the gap. Tools like NordVPN or Surfshark are fine for your own admin laptop on untrusted wifi, but they carry no business associate agreement and they are not a substitute for AWS Site-to-Site VPN or AWS Client VPN on a clinical path.

One piece of housekeeping that is cheap to do and expensive to skip: the business associate addendum. AWS makes this self-service through AWS Artifact, and it can be accepted across an entire organization. Only services on the current HIPAA Eligible Services Reference may hold PHI, and that list changes, so check it against the services you actually chose rather than your memory of it.

Boundary two: the landing zone

Raw messages land in S3. This part is well documented, so I will focus on the parts people configure once and never verify.

Make the bucket policy do the enforcing

Default bucket encryption means objects get encrypted. It does not mean a caller cannot write an object with different settings, and it does not stop a plaintext HTTP request. Two explicit denials cover the gap. The first protects the transport, the second protects the storage:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "DenyInsecureTransport",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:*",
      "Resource": [
        "arn:aws:s3:::lab-raw-inbound",
        "arn:aws:s3:::lab-raw-inbound/*"
      ],
      "Condition": {
        "Bool": { "aws:SecureTransport": "false" }
      }
    },
    {
      "Sid": "DenyWrongKey",
      "Effect": "Deny",
      "Principal": "*",
      "Action": "s3:PutObject",
      "Resource": "arn:aws:s3:::lab-raw-inbound/*",
      "Condition": {
        "StringNotEquals": {
          "s3:x-amz-server-side-encryption-aws-kms-key-id":
            "arn:aws:kms:REGION:ACCOUNT:key/KEY-ID"
        }
      }
    }
  ]
}

Use a customer managed KMS key, not the AWS managed one. The reason is not stronger cryptography, it is the key policy. A customer managed key gives you a second, independent authorization surface: even a principal with broad S3 permissions cannot read the object if the key policy does not grant them decrypt. That separation is what makes a blast radius argument credible to an assessor, and it is what makes crypto shredding possible later.

Your object keys are metadata, and metadata leaks

This is the one I would put on a poster. Object contents are encrypted. Object keys are not. They show up in CloudTrail data events, S3 server access logs, S3 Inventory reports, bucket listings, Lambda event payloads, and every console screenshot anyone pastes into a ticket.

# Leaks PHI into six places that are not the bucket
raw/2024/ORU_MRN4471902_SMITH_JOHN_A1C.hl7

# Same object, no identifiers in the key path
raw/dt=<partition>/src=lis01/msg=01HXYZ...ULID.hl7

Use an opaque, sortable identifier and keep the mapping from identifier to patient inside an encrypted store you actually control. Partition on ingest date and source system, never on anything derived from the patient. This one decision removes more PHI surface than any other single change in the pipeline.

Immutability, and the trap inside it

Clinical laboratories in the United States operating under CLIA have record retention obligations set out in 42 CFR 493.1105. The floor for test reports is at least two years after the date of reporting, with pathology test reports at ten years, and several state requirements sit above the federal floor. Retention is therefore not something you invent; go and read the section, then check your state.

S3 Object Lock gives you write-once-read-many enforcement, and it is the right tool. But understand the two modes before you commit, because one of them is genuinely irreversible.

• Governance mode blocks deletion for most principals, but anyone holding s3:BypassGovernanceRetention can override it by sending the x-amz-bypass-governance-retention:true header. Note the S3 console includes that header by default.
• Compliance mode cannot be overridden by anyone, including the account root. Per AWS documentation, the only way to delete an object under compliance retention before its date expires is to delete the AWS account.

A typo in a retention date under compliance mode is permanent, and you pay to store the mistake for its full term. Object Lock also has to be enabled at bucket creation, alongside versioning. Start in governance mode, prove the retention values against real traffic, then move to compliance mode deliberately rather than as a default.

# Indefinite hold for a specific object version, e.g. under litigation
aws s3api put-object-legal-hold 
  --bucket lab-raw-inbound 
  --key raw/dt=.../msg=01HXYZ.hl7 
  --legal-hold Status=ON

Boundary three: where PHI escapes into your observability

Here is the section that justifies the post. Everything above is on the compliance checklist. This is the part that is invisible until an audit or an incident makes it visible.

Your operational plane is a PHI store you did not declare. Specifically:

• Application logs. One print(record) left in a parser during debugging writes patient demographics into CloudWatch Logs, where the default is often no expiry at all.
• Exception traces. Unhandled parse errors frequently include the offending segment in the message. That segment is the PHI.
• Step Functions execution history. If you pass message bodies between states as payloads, that content is retained in the execution record and readable by anyone who can view executions.
• Dead letter queues. The whole point of a DLQ is to keep the failed message. A DLQ full of unparseable HL7 is a PHI archive with a different access policy than the bucket you designed so carefully.
• Third-party monitoring. Anything shipped to an external platform leaves your boundary. Grafana Cloud, Datadog and equivalents are perfectly reasonable choices for infrastructure telemetry, but a log line containing PHI going to a vendor without an executed BAA is a disclosure, not a metric.
• Notifications. An SNS topic that emails an on-call engineer “failed to process record for J. Smith, DOB …” has just sent PHI to a mail server nobody reviewed.

The pattern that fixes it: pass pointers, not payloads

Adopt a single rule and enforce it in review: PHI moves by reference between components, never by value.

1. The ingest function writes the raw message to the encrypted bucket and emits only an opaque object key plus a message ID.
2. Every downstream state, queue and event carries that pointer. Nothing carries the record body.
3. Error handling logs the pointer and a failure class, never the content that failed.
4. A DLQ therefore contains pointers to failures, and reprocessing means re-reading from the bucket under the same key policy as everything else.

The cost of this pattern is a slightly noisier debugging experience. You cannot read the failing record straight out of the queue. You have to go and fetch it, with credentials, and that fetch is logged. That inconvenience is the control working.

Then close the log surface itself

Log groups created implicitly by Lambda and other services often retain data indefinitely unless you set a policy. Find the ones nobody configured:

# List log groups with no retention configured
aws logs describe-log-groups 
  --query 'logGroups[?retentionInDays==null].logGroupName' 
  --output text

# Set an explicit retention on one of them
aws logs put-retention-policy 
  --log-group-name /aws/lambda/hl7-ingest 
  --retention-in-days 30

Run the first command as a scheduled check, not a one-off. New functions create new log groups, and the default comes back every time someone deploys.

Amazon Macie is worth pointing at your buckets as a detective backstop. It will not stop a leak, but it will tell you when something started landing where it should not.

Boundary four: from identified data to analytics

At some point someone wants to query the data. This is the crossing where teams are most likely to convince themselves they have done something they have not.

Amazon Comprehend Medical will detect PHI entities in unstructured clinical text through its DetectPHI operation, returning entity types such as name, address, identifier and date, each with a confidence score. It is a genuinely useful tool for narrative fields: pathology comments, specimen notes, microbiology free text.

aws comprehendmedical detect-phi 
  --text "Specimen received from the referring clinic on the stated date."

Now the part the marketing pages underplay, straight from the AWS developer guide: the entities Comprehend Medical detects do not map one to one to the identifier list specified by the Safe Harbor method, and AWS explicitly recommends additional human review or other methods to confirm accuracy for compliance use cases.

Read that as: PHI detection is not de-identification. Under the HIPAA Privacy Rule, de-identification has two defined routes, Safe Harbor and Expert Determination. An NLP model with confidence scores is an input to either, not a substitute for either. If your analytics tier is meant to hold de-identified data, someone has to own that determination, and it will not be your pipeline.

What works structurally:

• Put the analytics tier in a separate AWS account with its own KMS key. Not a separate bucket. A separate account, so the blast radius argument is enforced by IAM boundaries rather than by convention.
• Make the flow one directional. The de-identification job reads from the identified side and writes to the analytics side. Nothing in the analytics account holds permissions pointing back.
• Replace identifiers with pseudonyms rather than deleting them, keeping the crosswalk in the identified account. This preserves your ability to join across results and to re-identify under a documented process, which the research use case usually needs.
• Watch quasi-identifiers. Rare test panels, unusual reference ranges and precise timestamps re-identify people in small populations even after the obvious fields are gone. This is a statistics problem, not an IAM problem.
• Use Lake Formation or equivalent for column and row filtering if analysts need partial access, so the restriction lives with the catalog rather than in each query tool.

Boundary five: sending results back out

Delivery is where clinical correctness and security pull against each other, and security usually loses quietly.

The correction problem is the sharp edge. HL7 result messages carry a status that distinguishes preliminary, final and corrected results. If your delivery layer is idempotent on message ID alone, a corrected result with a new ID for the same observation will happily land beside the original instead of superseding it. Two conflicting values, both marked delivered, nobody alerted.

Build the idempotency key from the identity of the observation, which typically means the placer or filler order identifier combined with the specific observation identifier, and carry the result status as a first-class field so a correction overwrites rather than appends. Keep every version, expose the current one.

For the outbound path itself:

• Presigned URLs are convenient for report PDFs and they are also a bearer credential. Anyone holding the link has the document. Keep expiry short and treat generation as an auditable event.
• If you expose a provider-facing portal, put a WAF in front of it. AWS WAF and Cloudflare are both credible here; the deciding factor is usually where the rest of your edge already lives, not the rule engines.
• Never email a result. Email a notification that a result is available, behind authentication. The distinction sounds pedantic right up until someone forwards a mailbox.

Boundary six: deletion, and the conflict nobody plans for

Two obligations point in opposite directions. Retention rules say keep the record. Privacy rights and internal policy say be able to remove data. If you set compliance-mode Object Lock across everything, you have chosen one side without noticing.

Resolve it by classifying before you lock. Records that fall under laboratory retention obligations go into the immutable tier with a retention period derived from the applicable rule. Derived artifacts, caches, intermediate parquet, analytics extracts and enrichment outputs do not belong there. They belong in buckets with lifecycle rules, and they are what you actually delete.

Crypto shredding covers the middle ground: if a data set is encrypted under a key used for nothing else, scheduling that key for deletion renders the ciphertext unrecoverable without touching the objects. It is clean, it is verifiable, and it only works if you planned the key granularity up front. Retrofitting per-tenant or per-cohort keys onto a pipeline that used one key for everything is a full re-encryption exercise.

Do not forget the endpoints. Instrument workstations, the interface engine box, and the laptop somebody used to test the parser all accumulate copies. Cloud controls do nothing for physical media, and secure erase tooling such as O&O SafeErase exists for exactly that step in a decommissioning runbook.

Troubleshooting a medical lab data pipeline on AWS

Messages arrive but nothing appears downstream

Check the DLQ first, then check whether the DLQ itself has a consumer. A surprising number of pipelines have a correctly configured dead letter queue that nobody monitors, so failures accumulate silently and the only symptom is a gap in the data. Alarm on DLQ depth greater than zero, not on a threshold.

Access denied from a Lambda that has the right IAM policy

With a customer managed KMS key, IAM permission on S3 is only half the grant. The function’s role also needs decrypt permission on the key, and the key policy has to allow it. If you are crossing accounts, both the key policy and the role policy must permit the action. This is the intended behavior of the two-lock design, and it is the single most common support question on any pipeline built this way.

Deletion fails on an object you are certain you should be able to delete

Check for a legal hold before you check retention. A legal hold has no expiry and is independent of the retention period, so an object can be past its retention date and still undeletable. Also confirm which mode the retention uses, because governance and compliance produce the same error to a caller without bypass permission.

Counts do not reconcile with the LIS

Compare on observations, not messages. One message can carry several results, corrections create additional messages for the same observation, and a naive message count will diverge from the LIS by exactly the amount that matters clinically. Reconcile daily and alert on drift rather than investigating at quarter end.

Timestamps drift between systems

HL7 v2 timestamps do not always carry a timezone offset, and analyzers are often set to local time with no daylight saving handling. Capture the offset at ingest from the source system configuration and store everything in UTC with the original string preserved. Recovering an ambiguous timestamp after the fact is unpleasant, and around a daylight saving transition it may not be possible at all.

Common mistakes

• Treating “HIPAA eligible” as “compliant.” Eligibility means AWS will cover the service under the BAA. Configuration is still entirely yours.
• Putting identifiers in object keys, queue names, or log group names. Encrypted contents, plaintext metadata.
• Enabling compliance-mode Object Lock before validating retention values. There is no undo, and no support ticket that fixes it.
• Assuming automated PHI detection equals de-identification. AWS itself says the entity list does not map one to one to Safe Harbor identifiers.
• Running the analytics tier in the same account as identified data. A bucket boundary is a policy away from collapsing. An account boundary is not.
• Shipping application logs to an external platform without checking what the log lines contain. Test it by grepping your own log group for a known test patient name.
• Ignoring corrected results until an analyst notices. Amendment handling is a day-one requirement, not a phase two feature.

Best practices worth the effort

• Write the data flow diagram first and mark every trust boundary crossing. Every arrow that crosses one needs a named control. This exercise finds more problems than any scanner.
• Encode the eligible service list as policy. Service control policies at the organization level stop someone from putting PHI into a service you never assessed.
• Keep VPC endpoints on the PHI path. Interface and gateway endpoints keep traffic to AWS services off the public internet and give you an endpoint policy as an extra choke point. Check which of your chosen services offer one.
• Build with synthetic HL7 from day one. Nobody should need production PHI to develop a parser. A cheap VPS from Contabo or InterServer is fine for a synthetic-data development box, and keeping that environment entirely outside the PHI boundary is the point.
• Alarm on absence. A feed that stops is more dangerous than a feed that errors, because errors are loud. Alert when expected message volume for a source drops below its floor.
• Test restores, not backups. Restore into an isolated account, confirm the KMS grants work there, and document how long it took.
• Review CloudTrail for data events on the PHI buckets. Object-level logging costs money and is the only record of who read what.

FAQ

Do I need a BAA with AWS before I start building?

Before PHI touches the environment, yes. You can architect and test with synthetic data first. AWS provides the business associate addendum through AWS Artifact as a self-service acceptance that can be applied across an organization, and only services listed on the HIPAA Eligible Services Reference may process, store or transmit PHI.

Should I use AWS HealthLake or build on S3, Glue and Athena?

HealthLake is a managed FHIR datastore with FHIR APIs, medical NLP and query built in, and it is the shorter path if your consumers speak FHIR. It does not natively ingest HL7 v2; AWS points to partner tooling or your own transformation for non-FHIR input. If your consumers are analysts with SQL and the data is structured lab results rather than clinical documents, an S3 and Athena lakehouse is simpler and easier to keep inside a boundary you control. Decide on the consumer, not the source.

How long do I have to keep laboratory records?

For CLIA-regulated laboratories in the United States, 42 CFR 493.1105 sets the federal floor, including at least two years for test reports after the date of reporting and at least ten years for pathology test reports. State law frequently exceeds these minimums, so the applicable period is whichever is longer for your jurisdiction. Read the regulation and confirm with your compliance lead before you encode a number into a retention policy.

Can Amazon Comprehend Medical de-identify data for me?

It can detect PHI entities and give you confidence scores, which is a strong starting point for redaction workflows. It does not by itself satisfy HIPAA’s de-identification standard. AWS documents that the detected entities do not map one to one to the Safe Harbor identifier list and recommends human review for compliance use cases. Treat it as a detector feeding a documented Safe Harbor or Expert Determination process.

Is S3 Object Lock compliance mode required for lab records?

Not inherently. The argument for compliance mode is a threat model in which a compromised or malicious administrator could remove retention, not a regulation that names the mode. Governance mode plus tightly controlled bypass permissions and strong audit logging is a defensible position for many laboratories. Choose deliberately and write down the reasoning, because compliance mode commits you to the storage cost for the full term with no exit.

Where does PHI most often leak in a pipeline that looks correctly configured?

Logs, dead letter queues, workflow execution histories, notification messages and object key names. All five are metadata surfaces that sit outside the storage layer everyone reviews. Grep your own log groups for a known test patient identifier and you will find out quickly whether your pipeline has the problem.

Do I need a dedicated AWS account just for the lab pipeline?

Separate accounts for identified and de-identified data are worth it even at small scale, because the boundary is then enforced by IAM rather than by naming conventions. Whether the PHI workload also needs isolating from your other production workloads depends on who holds admin access there. If the answer is “broadly the same people,” separate it.

The one thing to take away

A secure medical lab data pipeline on AWS is not defined by the encryption on its storage layer. Encryption is the easy part, and it is the part everyone gets right. What separates a design that survives an audit from one that does not is whether PHI can travel by value through your operational plane: into logs, into queue bodies, into workflow histories, into filenames, into alert emails.

Move PHI by reference, keep identifiers out of every metadata surface, and put an account boundary between identified and de-identified data. Do those three things and the rest of the compliance work becomes documentation rather than redesign.

Work with me on healthcare data pipelines

I design and review AWS data platforms for regulated workloads, and lab data is a specific enough problem that generic cloud advice tends to miss it. Things I can help with:

• Mapping every trust boundary in an existing pipeline and finding the PHI surfaces nobody documented, including logs, queues and object key schemes.
• Designing HL7 v2 or file-drop ingestion into S3 with KMS key separation, VPC endpoints and bucket policies that enforce rather than suggest.
• Building the identified and de-identified account split, including the one-way transformation job and the pseudonym crosswalk.
• Getting amendment and correction handling right so corrected results supersede rather than duplicate, all the way to the analytics tables.
• Retention and deletion design: Object Lock modes, lifecycle policies, crypto shredding key granularity, and the runbook that ties them together.
• Observability that is useful without being a disclosure risk, including CloudWatch retention hygiene and safe alerting patterns.

If you have an architecture diagram, a Terraform plan, or a redacted sample of the messages you are receiving, send it over and I will tell you what I would change first.

The post Where PHI Escapes: Building a Secure Medical Lab Data Pipeline on AWS appeared first on John Nessime.