Legal Document Intelligence on AWS: A Secure Build

The demo was going well until someone asked where the third citation came from.

The answer on screen was good. Fluent, specific, correctly hedged, three sources listed underneath it. Then a partner in the room asked a simple question: which matter is source three from? Nobody could answer it in the room, and when we went and looked, it was from a matter that half the people watching the demo were formally walled off from.

Nothing had crashed. No alarm fired. The retrieval layer had done exactly what a similarity search does, which is return the nearest vectors, and the nearest vectors did not care about the ethical wall. That is the thing about building a legal document intelligence platform on AWS: the failures that matter almost never look like failures. They look like a confident answer with a citation attached.

This post is about the parts of that build that are hard, organised around the five boundaries that actually have to hold. The extraction pipeline is the easy half. I will cover it, but quickly, because the AWS documentation is good and the failure modes are visible. The other four boundaries fail quietly, and that is where the engineering goes.

The shape of the thing

Before the boundaries, the skeleton. A legal document intelligence platform on AWS almost always ends up looking like this, whether you plan it or arrive at it:

Documents land in Amazon S3, one prefix per matter, versioning on.
An S3 event triggers AWS Lambda, which starts an asynchronous Amazon Textract job.
Textract output lands back in S3 as JSON. Something normalises it into text plus page and bounding-box coordinates.
Amazon Comprehend or a foundation model classifies the document and pulls entities: parties, dates, governing law, clause types.
Chunks get embedded and written to a vector store, usually Amazon OpenSearch Serverless behind Amazon Bedrock Knowledge Bases.
An application layer, typically API Gateway plus Lambda, takes a question, retrieves, and calls a model on Amazon Bedrock.

AWS Step Functions is worth reaching for once you have more than three stages, because Textract jobs are long-running and Lambda timeouts are not a retry strategy. That is the whole architecture. You can stand it up in a fortnight. Then you spend six months on everything below.

Boundary one: isolation, and why metadata filtering is not optional

This is the one from the demo, and it is the one that ends engagements.

A vector index has no native concept of a matter, a client, or an ethical wall. If you pool every document into one index and rely on the prompt to keep things separate, you have built a system where a well-phrased question can pull privileged material across a wall. Prompt instructions are not an access control. They are a suggestion that usually works.

There are two shapes that do work, and the choice between them is a real trade-off rather than a best practice.

Pooled index with server-side filtering

Every chunk carries a matter_id and a client_id as metadata. Every retrieval call attaches a filter. With an S3 data source in Bedrock Knowledge Bases, the metadata comes from a sidecar JSON file that sits next to the document and carries a metadataAttributes object.

{
  "metadataAttributes": {
    "matter_id": "M-4417",
    "client_id": "C-108",
    "doc_type": "engagement_letter",
    "privileged": true
  }
}

The retrieval call then narrows the search before the model ever sees a chunk:

"retrievalConfiguration": {
  "vectorSearchConfiguration": {
    "filter": {
      "equals": { "key": "matter_id", "value": "M-4417" }
    }
  }
}

Two rules make this safe, and both are the kind of thing that gets skipped under deadline. First, the filter value is derived on the server from the authenticated caller’s claims, never accepted from the client. If the browser can send you a matter_id, the browser can send you a different one. Second, the filter is applied by code that no feature request can bypass. A helper that builds every retrieval request, and a code review rule that no other path may call the retrieve API directly.

The trap here is ordering. You cannot filter on an attribute the index does not have. Add matter_id after ingestion and every chunk already in the index is invisible to that filter, which means it either returns for everyone or for no one, depending on how your filter is written. Design the metadata schema before the first ingestion run, not after the first demo.

Separate collection per client

The heavier option. A separate OpenSearch Serverless collection per client, which buys you a separate AWS KMS key per client, separate index settings, and a failure mode where a bug in the filtering logic cannot reach across clients at all.

It costs you sprawl. Every collection is a resource to provision, monitor, patch policy on, and eventually delete. Ingestion jobs run per data source per knowledge base, so freshness guarantees fragment. For a firm with twelve institutional clients this is fine. For a platform onboarding a hundred small clients it becomes the main operational burden of the product.

My default is pooled with strict server-side filtering, and per-client collections only where the client’s own contract demands a dedicated encryption key. If you cannot say out loud which of those two you are running, you are running neither properly.

Boundary two: extraction, and the confidence score everyone ignores

Textract returns a confidence score between 0 and 100 with each prediction. AWS is explicit in its own best-practice guidance that applications sensitive to detection errors should enforce a minimum threshold and route anything below it for human scrutiny, and that the right threshold depends entirely on how the output gets used.

Legal documents sit at the harsh end of that. A misread date on an archival scan is a curiosity. A misread date on a limitation period is a problem with a name on it.

Practical notes from this layer:

Use the asynchronous operations for anything multipage. StartDocumentAnalysis and GetDocumentAnalysis handle long PDFs and TIFFs; the synchronous AnalyzeDocument path is for single pages and will fight you on a 400-page bundle.
Textract’s Queries feature earns its place on structured instruments. Instead of extracting everything and grepping, you ask the document a direct question and get a scoped answer with its own confidence.
Keep the page number and bounding box for every chunk. When a lawyer asks where a sentence came from, “page 14, second column” is an answer. “It’s in the corpus somewhere” is not, and the platform loses credibility the first time you say it.
Amazon Augmented AI wires low-confidence predictions into a human review queue with a private workforce, so the reviewers are your people rather than an anonymous pool. For privileged material that distinction is the entire point.

What I would skip early: building a custom entity recognition model. Comprehend supports custom entity recognition and it is genuinely useful, but you need labelled data you do not have yet, and a foundation model with a decent prompt gets you far enough to find out which entities the users actually care about. Train the custom model once the requirement has stopped moving.

Boundary three: retention, where deletion is a distributed problem

Here is the failure that will not show up in any test suite you write.

A client asks for a document to be removed. Someone deletes the S3 object. The ticket closes. The document is still fully searchable, because its chunks are sitting in the vector index and will stay there until the data source is re-synced. Meanwhile the Textract JSON output is in a second bucket, the extracted text may be in a database, and if model invocation logging is on, whole passages of the document are sitting in CloudWatch Logs or another S3 bucket entirely.

One delete, at least four places holding a copy. Write the deletion path as a real workflow with an assertion at the end, not as a single API call:

Delete the source object and, if versioning is on, its versions.
Delete the derived artefacts: Textract JSON, normalised text, any thumbnails or page images.
Trigger an ingestion job so the knowledge base drops the orphaned chunks.
Confirm removal by querying the index for the document identifier and expecting nothing back.
Deal with the logs, which means either a retention policy short enough to make the problem expire or a deliberate decision, written down, that logs are out of scope.

Step four is the one people leave out, and it is the only step that actually proves anything.

The opposite problem: things that must not be deleted

Legal work has the reverse requirement too. When a matter goes into litigation hold, the documents need to survive an administrator with delete permissions and a bad afternoon.

S3 Object Lock is the mechanism, and it has two independent controls that people routinely conflate. A retention period protects an object version until a fixed date, in either governance mode, which privileged users can override, or compliance mode, which nobody can override and where the period cannot be shortened. A legal hold has no date at all. It stays until someone with s3:PutObjectLegalHold explicitly removes it. The two can be active at once, and while either is active the object version cannot be deleted or overwritten.

# Place an indefinite hold on a specific object version
aws s3api put-object-legal-hold 
  --bucket matters-archive 
  --key M-4417/exhibit-c.pdf 
  --version-id 3sL7f2Qz9pXvB1kR 
  --legal-hold Status=ON

Two things bite here. Object Lock requires versioning and turns it on automatically, and once enabled on a bucket you cannot turn it off or suspend versioning again. And compliance mode is genuinely permanent: if you set a seven-year retention when you meant seven days, that is the answer, for everyone, including the account root. Test in governance mode first. The bypass path exists and is deliberately awkward, requiring both the s3:BypassGovernanceRetention permission and an explicit x-amz-bypass-governance-retention:true header on the request, which is exactly the level of friction you want on that operation.

Design the hold model before you design the deletion model, because holds win. A deletion request that collides with an active hold is a legal question, not an engineering one, and the platform’s job is to surface the collision clearly rather than resolve it silently.

Boundary four: disclosure, or what leaves the account

This is the boundary the client’s general counsel will ask about, usually in writing, usually before signature. Three separate controls, and they are not interchangeable.

The AI services opt-out policy

AWS Organizations has a governance policy type that opts your accounts out of having content processed by certain AI services stored and used for service improvement. The list includes Amazon Textract and Amazon Comprehend, which are precisely the two doing the reading in this architecture. The policy type has to be enabled at the organisation root before you can attach anything:

aws organizations enable-policy-type 
  --root-id r-example 
  --policy-type AISERVICES_OPT_OUT_POLICY

Read the AWS documentation on this one carefully rather than taking my summary as gospel, because there is a caveat in it that matters: the services may still need to store your data operationally even when you have opted out of it being used for improvement. Opting out is not the same as the data never existing outside your account. Say that plainly in the client conversation. It is a much better position than being asked about it later.

Bedrock’s data position, and the log that undoes it

Amazon Bedrock’s published position is that inputs and outputs are not shared with third-party model providers and are not used to train the base models, and that fine-tuning operates on a private copy. That is a strong starting point for privileged content and it is the main reason Bedrock rather than a direct provider API shows up in these builds.

Then there is model invocation logging. It captures full prompt and response payloads to CloudWatch Logs or S3, and you want it on, because without it you cannot reconstruct what the system told someone. But understand what you have just built: a log that contains privileged document text, at a per-region setting, in a destination that probably has looser access controls than the document store it came from. Encrypt it with the same KMS key discipline, restrict it harder than you think you need to, and set a retention period on purpose.

Worth separating in your head: CloudTrail records that an API call happened and who made it. Model invocation logging records what was in it. You need both, for different questions, and only one of them contains client confidences.

Network path

Interface VPC endpoints via AWS PrivateLink keep traffic to Bedrock, Textract and the rest on the AWS network rather than out through an internet gateway or NAT. Add a gateway endpoint for S3 while you are there.

Endpoint policies are the part that gets forgotten. An endpoint without a policy is a private path to the whole service, including into accounts that are not yours. Scope it to the operations and resources you actually use, and pair it with an S3 bucket policy that rejects requests not arriving through your endpoint. The first protects what leaves; the second protects what can be reached.

The parts that are not on the architecture diagram

Two of these have caught people out badly, and neither is an AWS control.

The first is the development environment. You do not want real client documents on a laptop or in a scratch account, so build the parsing and chunking code against synthetic documents on a small separate box. A cheap VPS from a provider like Contabo or InterServer is fine for this, and the separation is worth more than the convenience of iterating in the production account.

The second is the reviewers. If your human review loop involves contractors working remotely, their network path and their disks are part of your boundary whether or not you drew them. A managed VPN such as NordVPN or Surfshark handles the network side, and when a matter closes and local copies have to go, a dedicated erasure tool from something like O&O Software does what dragging a folder to the bin does not. Unglamorous, and it is the layer that gets audited.

Boundary five: evidence, because someone will ask

At some point the question stops being “does it work” and becomes “who saw what, and when”. If you cannot answer that from stored records, the platform is not defensible regardless of how good the answers are.

Turn on CloudTrail data events for the document buckets. They are off by default, they are billed separately from management events, and they are the only way to see individual object-level reads.
Log the retrieval, not just the generation. Store which chunks came back and which filter was applied. When somebody asks whether the wall held on a specific query, this record is the answer and there is no reconstructing it later.
Carry the end user’s identity through to the audit record. A Lambda execution role in the logs tells you the platform did something. It does not tell you who asked.
Alarm on the filter, not just on errors. A retrieval that ran without a matter filter should page someone, even though it returned HTTP 200 and a perfectly good answer.

That last one is the closest thing to a single takeaway in this post. The dangerous events in this architecture are successful ones.

Troubleshooting the things that will go wrong

Retrieval returns nothing after you add a filter

Nearly always a metadata problem rather than a query problem. Either the sidecar metadata file was missing at ingestion, so the chunks carry no attribute to match, or the attribute was added after the chunks were indexed. Check whether the attribute exists on a known chunk before you touch the query. If it does not, you need a re-sync, not a different filter expression.

Textract returns text but the layout is scrambled

Common on two-column pleadings and on documents with headers and footers on every page. Raw text detection reads in a reading order that is not always yours. Use the layout and table features rather than plain detection, and hold onto the geometry so you can reassemble columns yourself if the default order is wrong for that document class.

The model cites a document that does not exist

Usually the citation is being generated rather than passed through. If document names are being written by the model instead of read from the retrieval response, it will invent plausible ones. Build citations from the retrieval result metadata in your application code and never from the generated text. Bedrock Guardrails helps with contextual grounding, but the structural fix is not asking the model to produce the reference in the first place.

Ingestion is slow and nobody knows where

Instrument per stage before you optimise anything. In most of these pipelines the time is in Textract for large scanned bundles, and the fix is parallelism at the document level rather than tuning the extraction itself. Step Functions with a distributed map over documents gets you there without a queue you have to babysit.

Costs climb faster than volume

Look at re-processing first. A pipeline that re-extracts a document every time anything downstream changes will quietly multiply your Textract bill against a static corpus. Key the extraction cache on the object’s content hash, not its path, and make re-ingestion an explicit action. The billing mechanisms differ per service, so read the current pricing pages rather than trusting a number from a blog post.

Common mistakes

Treating the system prompt as an access control. It is a formatting instruction that happens to look like a rule.
Taking the tenant identifier from the request body. If the client can name the matter, the client can name someone else’s.
Designing the metadata schema after the first ingestion run. Retrofitting a filterable attribute means re-indexing the whole corpus.
Assuming a deleted S3 object is gone from the platform. The chunks, the extraction output and the invocation logs all outlive it.
Setting compliance-mode retention without testing in governance mode. There is no support ticket that fixes a seven-year mistake.
Enabling model invocation logging without treating the log as privileged. You have just made a second copy of the documents in a less protected place.
Building VPC endpoints and leaving the default policy on them. A private path with no policy is still a path.

Best practices worth the effort

One retrieval helper, server-side filter injection, and a review rule that no other code path calls the retrieve API directly.
Metadata schema agreed and frozen before the first production ingestion. Include the fields you might filter on later, even if unused today.
Page-level provenance on every chunk, surfaced in the interface. It builds trust faster than any accuracy improvement.
A confidence threshold chosen per document class, with a human review path for everything under it.
Deletion implemented as a verified workflow across every store, ending in an assertion that the content is actually unretrievable.
Object Lock legal holds for matters under litigation hold, tested in governance mode before anything runs in compliance mode.
Customer-managed KMS keys across S3, the vector store and the logs, so key access is a control you can actually revoke.
An alarm on unfiltered retrievals, because the failure mode is a success response.

Frequently asked questions

Is a legal document intelligence platform on AWS safe for privileged material?

The services support it. Bedrock’s stated position is that inputs and outputs are not shared with model providers or used to train base models, PrivateLink keeps traffic off the public internet, and KMS covers encryption at rest. What determines safety is your own configuration: tenant isolation, log handling and deletion behaviour. The platform is exactly as confidential as its weakest copy of the text.

Should I use Bedrock Knowledge Bases or build retrieval myself?

Start with Knowledge Bases. It handles chunking, embedding, sync and metadata filtering, and those are weeks of work with no differentiation in them. Build your own when you need retrieval behaviour it does not expose, such as unusual re-ranking or per-tenant chunking strategies. Going custom on day one usually means reimplementing the managed service badly.

How do I stop one client’s documents surfacing in another client’s answers?

Tag every chunk with a tenant identifier at ingestion, and inject the matching filter server-side on every retrieval from the authenticated caller’s identity. Never accept the identifier from the client. If the contract requires per-client encryption keys, use a separate collection per client instead, accepting the extra operational load.

Does deleting a document from S3 remove it from the search index?

No. The chunks stay in the vector store until an ingestion job runs and reconciles the data source. Until then the document is deleted and still fully searchable, which is the worst combination available. Always finish a deletion by querying the index and confirming nothing comes back.

What is the difference between an S3 legal hold and a retention period?

A retention period runs until a fixed date and comes in governance mode, which privileged users can override, or compliance mode, which nobody can. A legal hold has no end date and stays until someone explicitly removes it. They are independent, they can both be active on the same object version, and while either is active the object cannot be deleted or overwritten.

Do I need human review, or is the model accurate enough?

Accuracy is the wrong frame. The question is what happens to a wrong answer downstream. Where output feeds a decision with consequences, you want a confidence threshold and a review queue, and Amazon Augmented AI with a private workforce gives you that without building the review tooling yourself. Where output is a search aid a person will verify anyway, review is friction you do not need.

Can I run this in a single AWS account?

Technically yes, and for a pilot it is reasonable. It gets uncomfortable once you have production documents and a development environment in the same place, because an IAM mistake has nowhere to stop. Separate accounts under Organizations also gives you the policy layer, including the AI services opt-out policy, which only exists at the organisation level.

The one thing to take away

A legal document intelligence platform on AWS is not hard to build. Textract reads, Bedrock reasons, OpenSearch retrieves, and the tutorial version works on the first afternoon.

What is hard is that its failures are silent and well-formed. The cross-matter citation, the deleted document that still answers questions, the privileged passage sitting in a log bucket, the retrieval that ran without a filter and returned a beautiful paragraph. None of these throw an error. All of them are the kind of thing that ends a client relationship.

So build the boundaries first and the features second, and make sure every one of them is something you can prove from a stored record rather than something you believe about the code. If you can only take one habit from this: alarm on the successful requests that should not have been possible.

Need a second pair of eyes on your document pipeline?

I work on AWS document processing and retrieval systems where the confidentiality requirements are real. Typical things I get called in for:

Reviewing tenant isolation in an existing RAG setup and finding the paths where the filter can be bypassed.
Designing the metadata and chunking schema before ingestion, so filtering works without a re-index later.
Building Textract pipelines with confidence thresholds, human review routing and page-level provenance.
Implementing verified deletion across S3, derived artefacts, the vector index and logs, with a proof step at the end.
Setting up retention and legal hold with S3 Object Lock, including the governance-mode rehearsal before compliance mode.
Locking down the network and logging path: VPC endpoints with real policies, KMS key separation, and audit records that name the actual user.

If you have an architecture diagram, a Step Functions definition or a retrieval request you are unsure about, send it over and I will tell you what I would change and why.

Work with me on Upwork