<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Amazon Textract | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/amazon-textract/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/amazon-textract/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Thu, 15 Oct 2026 06:06:48 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>Amazon Textract | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/amazon-textract/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>The Join Is the Hard Part: Combining Structured and Unstructured Data on AWS</title>
		<link>https://john-nessime.com/blog/technical-guides/combining-structured-unstructured-data-aws/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 15 Oct 2026 06:06:48 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Athena]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon S3 Vectors]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[Apache Iceberg]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Glue]]></category>
		<category><![CDATA[Bedrock Data Automation]]></category>
		<category><![CDATA[Bedrock Knowledge Bases]]></category>
		<category><![CDATA[Bitemporal Modeling]]></category>
		<category><![CDATA[Confidence Scoring]]></category>
		<category><![CDATA[Data Catalog]]></category>
		<category><![CDATA[Data Governance]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[Data Lake]]></category>
		<category><![CDATA[Data Lineage]]></category>
		<category><![CDATA[Data Quality]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Lake Formation]]></category>
		<category><![CDATA[Lakehouse]]></category>
		<category><![CDATA[Metadata Filtering]]></category>
		<category><![CDATA[Provenance]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[S3 Tables]]></category>
		<category><![CDATA[Schema Design]]></category>
		<category><![CDATA[Table Compaction]]></category>
		<category><![CDATA[Vector Database]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=500</guid>

					<description><![CDATA[<p>S3 holds the documents, Iceberg holds the rows, and everything works right up until a number disagrees with its source PDF. Combining structured and unstructured data on AWS is an identity problem, not a storage one. Here are the five failure families that cause it, why each stays invisible until it isn't, and the ingestion order that avoids most of them.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/combining-structured-unstructured-data-aws/">The Join Is the Hard Part: Combining Structured and Unstructured Data on AWS</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The ticket said &#8220;the contract value in the dashboard is wrong.&#8221; It wasn&#8217;t. The number in the warehouse table was right for the version of the PDF that had been extracted eleven months earlier. Someone had uploaded an amended contract to the same S3 prefix, the extraction job had picked it up, written a new row, and nothing downstream knew which of the two rows was current. Both were true. Only one was correct.</p>



<p class="wp-block-paragraph">That is what combining structured and unstructured data on AWS actually looks like when it goes wrong. Not a service outage. Not a permissions error. A quiet correctness bug that survives every test you wrote, because every individual component did exactly what it was told.</p>



<p class="wp-block-paragraph">The storage side of this problem is solved. S3 holds the PDFs, Iceberg tables hold the rows, Bedrock Data Automation pulls fields out of the documents, S3 Vectors or OpenSearch holds the embeddings. None of that is hard any more. The hard part is the join, and the join is where nearly all the failures live.</p>



<h2 class="wp-block-heading">What this post covers</h2>



<p class="wp-block-paragraph">Five failure families I keep running into on mixed-data architectures, why each one is invisible until it isn&#8217;t, and a working shape that avoids most of them. Then troubleshooting, mistakes, and the questions people actually ask before they build this.</p>



<p class="wp-block-paragraph">Not a service tour. AWS documentation already covers what Glue, Athena, Lake Formation and Bedrock do. What it covers badly is what happens at the seams between them.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Failure family one: there is no join key, only a filename</h2>



<p class="wp-block-paragraph">Structured data arrives with identity built in. A row has a primary key, a foreign key, a surrogate key from the CDC pipeline. Unstructured data arrives with an object key, which is a path, which is a decision someone made about folder layout six months ago.</p>



<p class="wp-block-paragraph">Object keys are not identity. They are location. The difference matters the moment anyone overwrites an object, moves a prefix, or uploads the same document twice under two names. Your join silently changes meaning and nothing raises an error, because to Athena a string is a string.</p>



<h3 class="wp-block-heading">What to use instead</h3>



<p class="wp-block-paragraph">Mint a document ID at ingestion, before extraction, and carry it everywhere. Store it three places: as S3 object metadata on the source object, as a column on every extraction output row, and as vector metadata on every chunk you embed. That ID is the only thing that ties the three planes together.</p>



<p class="wp-block-paragraph">Version the object too. S3 versioning gives you a version ID per object, and a document ID plus a version ID is a genuinely stable identity for &#8220;this exact bytes-on-disk artefact.&#8221; If you skip versioning on a bucket that feeds extraction, an overwrite destroys the evidence for every downstream row derived from it. You will be asked to reproduce that extraction eventually. Auditors, disputes, model retraining, all three.</p>



<p class="wp-block-paragraph">S3 Metadata is worth knowing about here. It maintains queryable Apache Iceberg tables describing the objects in a general purpose bucket, so you can join object-level facts against your business tables in Athena without writing an inventory crawler. Two limits shape how you use it: metadata tables live in an AWS managed table bucket rather than one you own, and the journal table only records change events that happen after you create the configuration. Enable it on day one or you inherit a permanent blind spot for everything already in the bucket. The inventory table covers the existing objects, which is why you want both.</p>



<h2 class="wp-block-heading">Failure family two: extraction confidence gets flattened into a fact</h2>



<p class="wp-block-paragraph">Bedrock Data Automation, Textract, and any model-based extraction return confidence alongside values. Every pipeline I have reviewed does the same thing with it: reads it once, applies a threshold, writes the value into a typed column, and drops the score.</p>



<p class="wp-block-paragraph">Now a decimal that the model was 62% sure about sits in the same column as one it was 99% sure about, and every dashboard, every SQL aggregate, every model feature treats them identically. The uncertainty didn&#8217;t go away. You just stopped being able to see it.</p>



<p class="wp-block-paragraph">Keep the score as a column. It costs one float per field and it buys three things: the ability to filter low-confidence rows out of a report without rerunning extraction, a review queue that ranks by expected damage rather than arrival order, and a way to measure whether a blueprint change actually improved anything. Add two more columns while you are there, one for the extractor or blueprint identifier and one for the timestamp. When a field goes wrong at scale the first question is always &#8220;which extractor produced these,&#8221; and without that column the answer is a week of archaeology.</p>



<pre class="wp-block-code"><code>-- Upsert extraction output into an Iceberg table, keyed on document
-- identity rather than object key. Athena performs merge-on-read for
-- MERGE INTO, so the write is cheap and the read pays the cost until
-- compaction catches up.

MERGE INTO contracts_extracted AS t
USING extraction_staging AS s
  ON t.document_id = s.document_id
 AND t.source_version_id = s.source_version_id
WHEN MATCHED THEN UPDATE SET
  contract_value       = s.contract_value,
  contract_value_conf  = s.contract_value_conf,
  extractor_id         = s.extractor_id,
  extracted_at         = s.extracted_at
WHEN NOT MATCHED THEN INSERT (
  document_id, source_version_id, contract_value,
  contract_value_conf, extractor_id, extracted_at
) VALUES (
  s.document_id, s.source_version_id, s.contract_value,
  s.contract_value_conf, s.extractor_id, s.extracted_at
);</code></pre>



<p class="wp-block-paragraph">The merge-on-read detail matters more than it looks. Athena writes delete files rather than rewriting data files, so a chatty extraction pipeline that merges every few minutes will quietly degrade query performance until compaction runs. On S3 Tables that maintenance is handled for you. On self-managed Iceberg in a general purpose bucket it is your job, and forgetting it is one of the more common reasons a lakehouse gets slower every week for no visible reason.</p>



<h2 class="wp-block-heading">Failure family three: two catalogs, two permission models</h2>



<p class="wp-block-paragraph">This is the one that bites teams who did everything else right.</p>



<p class="wp-block-paragraph">Your tables are governed by Lake Formation, with grants at table and column level. Your documents are governed by S3 bucket policies and IAM. Your vectors are governed by IAM on the vector bucket or index ARN. Three planes, three models, no shared enforcement point.</p>



<p class="wp-block-paragraph">The result is predictable and unpleasant. An analyst who is correctly denied the <code>salary</code> column in Athena runs the same question against the RAG endpoint and gets the number read straight out of the source PDF, because nobody applied column-level thinking to the chunks. The retrieval layer had no idea it was quoting a restricted field.</p>



<p class="wp-block-paragraph">There is no single AWS switch that fixes this. What works is applying the same classification at ingestion and enforcing it independently in each plane:</p>



<ul class="wp-block-list">

<li>Classify the document once, at ingestion, and write the classification into the sidecar metadata that the knowledge base ingests.</li>


<li>Use metadata filters on retrieval so a caller only ever searches the chunks their tier permits.</li>


<li>Keep restricted documents in a separate index or knowledge base where the classification is coarse and the blast radius matters.</li>


<li>Redact at extraction time for anything you genuinely cannot let leak, rather than relying on a filter you might misconfigure later.</li>

</ul>



<p class="wp-block-paragraph">Bedrock Knowledge Bases reads per-document metadata from a sidecar JSON file that sits beside the source object and carries the same filename with <code>.metadata.json</code> appended. That file is where classification, tenant, document ID and effective dates belong.</p>



<pre class="wp-block-code"><code>{
  "metadataAttributes": {
    "document_id": "doc_01HX9K4RZP",
    "classification": "restricted",
    "tenant_id": "acme",
    "effective_year": 2031
  }
}</code></pre>



<p class="wp-block-paragraph">Filterable metadata is what makes tenant isolation possible on a shared index. It is also the thing most likely to be wrong on the first attempt, which brings us to the next failure family.</p>



<h2 class="wp-block-heading">Failure family four: index decisions you cannot take back</h2>



<p class="wp-block-paragraph">S3 Vectors makes vector storage cheap enough that people stop thinking about it, and then hit a wall that has nothing to do with cost. When you create a vector index you fix three things permanently: the dimension, the distance metric, and the list of non-filterable metadata keys. Changing any of them means creating a new index and re-embedding everything.</p>



<p class="wp-block-paragraph">By default every metadata key you attach is filterable. Filterable metadata is subject to a tighter size budget than the total metadata per vector, so if you push a large chunk of text through as a filterable key, writes start failing with validation errors rather than degrading gracefully. Query a key you declared non-filterable and you get a 400 back, not an empty result set.</p>



<p class="wp-block-paragraph">If you are pointing Bedrock Knowledge Bases at an S3 Vectors index you created yourself, declare the Bedrock-managed text and metadata keys as non-filterable at creation time. They hold chunk text and are large. Leaving them filterable is the single most common reason a knowledge base sync fails partway through with validation exceptions and no obvious cause.</p>



<pre class="wp-block-code"><code># Non-filterable keys are set once, at index creation, and cannot be
# changed afterwards. Keep the list short: the cap is small, and every
# key you spend here is one you can never filter on.

aws s3vectors create-index 
  --vector-bucket-name my-vector-bucket 
  --index-name documents 
  --data-type float32 
  --dimension 1024 
  --distance-metric cosine 
  --metadata-configuration '{"nonFilterableMetadataKeys":["AMAZON_BEDROCK_TEXT","AMAZON_BEDROCK_METADATA"]}'</code></pre>



<p class="wp-block-paragraph">The dimension has to match your embedding model&#8217;s output exactly, and the same model has to be used for ingestion and for query. Swapping embedding models mid-life is not a config change, it is a migration. Plan the index around the model you intend to keep, not the one that is convenient today.</p>



<p class="wp-block-paragraph">Worth being even-handed here: S3 Vectors is the cheap, durable option and it is the right default for large corpora with modest query rates. If you need heavy filtering, hybrid keyword-plus-vector search, or very high query throughput, OpenSearch Serverless or a dedicated store like pgvector on Aurora will serve you better, and AWS supports tiering between them for exactly that reason. Pick on query pattern, not on storage price.</p>



<h2 class="wp-block-heading">Failure family five: reprocessing rewrites history</h2>



<p class="wp-block-paragraph">You improve the extraction blueprint. You rerun it across the corpus. Numbers in last quarter&#8217;s board pack change.</p>



<p class="wp-block-paragraph">This is the failure that ends trust in the platform, and it is entirely avoidable. Extraction output is a derived opinion about a document at a point in time. It is not a fact about the world. Model it that way.</p>



<p class="wp-block-paragraph">Keep two timestamps on every extraction row: when the source document became effective, and when this extraction ran. Reports pin to the first, audits use the second. Now a reprocess adds rows rather than changing answers, and &#8220;why did this number move&#8221; has an answer you can query instead of guess at.</p>



<p class="wp-block-paragraph">Iceberg gives you snapshot isolation and time travel for free on the table side, which is genuinely useful for reconciliation. It does not, on its own, tell you that the value changed because a different extractor ran. Only your columns do that.</p>



<h2 class="wp-block-heading">A shape that works for combining structured and unstructured data on AWS</h2>



<p class="wp-block-paragraph">Nothing exotic. The ordering is what matters, because several of these decisions are hard to reverse.</p>



<ol class="wp-block-list">

<li><strong>Land raw, enable versioning, enable S3 Metadata.</strong> Do this before you ingest anything you care about. Both are cheap to turn on and impossible to backfill properly.</li>


<li><strong>Mint the document ID at landing.</strong> Write it to object metadata and to the sidecar metadata file in the same step. One writer, one source of truth.</li>


<li><strong>Classify before you extract.</strong> Classification drives which extraction path runs, which index the chunks land in, and which Lake Formation grants apply downstream.</li>


<li><strong>Extract to a staging table, not straight to the curated table.</strong> Staging gives you somewhere to run validation before anything touches a table an analyst can query.</li>


<li><strong>Merge into Iceberg on document identity, carrying confidence and provenance columns.</strong> Never merge on object key.</li>


<li><strong>Embed separately, keyed by the same document ID.</strong> The vector plane and the table plane are different systems with different refresh rates. Treat them as such.</li>


<li><strong>Join at query time in Athena.</strong> Retrieval returns document IDs, SQL turns those into rows. Resist the urge to materialise one giant denormalised table containing both.</li>

</ol>



<p class="wp-block-paragraph">Step seven is the one people argue with. The argument for denormalising is query simplicity. The argument against is that you now have a third copy that drifts from both parents, and drift in a table nobody owns is how you get back to the ticket at the top of this post.</p>



<h2 class="wp-block-heading">Troubleshooting</h2>



<h3 class="wp-block-heading">Athena says the principal has no privilege on the resource</h3>



<p class="wp-block-paragraph">This is a Lake Formation grant, not an IAM one. IAM policy alone is not enough for catalog-governed tables. Grant on the catalog and database first, then the table. For S3 Metadata tables and S3 Tables specifically, the grant needs to reach the table bucket catalog and namespace, not just the table.</p>



<h3 class="wp-block-heading">Queries against metadata tables fail with a syntax error</h3>



<p class="wp-block-paragraph">Metadata table namespaces need to be quoted with double quotes or backticks in Athena and Redshift. It looks like a parser bug and it is not.</p>



<h3 class="wp-block-heading">Knowledge base sync fails partway with validation errors</h3>



<p class="wp-block-paragraph">Almost always filterable metadata size. Check whether the Bedrock-managed text keys were declared non-filterable when the index was created. If they were not, you need a new index, because that setting is immutable.</p>



<h3 class="wp-block-heading">Retrieval returns documents the caller should not see</h3>



<p class="wp-block-paragraph">Check that the filter is actually being applied at query time and that the metadata key is filterable. A filter on a non-filterable key returns an error rather than silently failing, so if you are getting results, the filter is probably absent from the request entirely.</p>



<h3 class="wp-block-heading">Athena gets slower week over week on a table that isn&#8217;t growing much</h3>



<p class="wp-block-paragraph">Delete files accumulating from merge-on-read writes, or small file proliferation from frequent micro-batches. Check snapshot and manifest counts before you start tuning queries.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">

<li>Joining on the S3 object key because it was there and it worked in the prototype.</li>


<li>Enabling S3 Metadata after the bucket is full, then wondering why the journal table is missing most of the history.</li>


<li>Applying a confidence threshold and discarding the score.</li>


<li>Creating the vector index before deciding on the embedding model.</li>


<li>Assuming Lake Formation column grants protect the same field when it is quoted out of a PDF by a retrieval endpoint.</li>


<li>Reprocessing in place, with no record of which extractor produced which value.</li>


<li>Turning off S3 versioning on the raw bucket to save on storage, then being asked to reproduce an extraction.</li>

</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">

<li>One ingestion writer that mints identity. Everything downstream reads that ID and never invents its own.</li>


<li>Confidence and provenance columns are not optional. They are the difference between debugging and guessing.</li>


<li>Treat extraction output as append-mostly and bitemporal. Effective time and extraction time, always both.</li>


<li>Decide the embedding model before the index, and budget non-filterable keys deliberately. The cap is small and permanent.</li>


<li>Enforce classification in all three planes independently. There is no single choke point.</li>


<li>Watch table maintenance the way you watch disk. Compaction and snapshot expiry on self-managed Iceberg; a monitoring stack such as Amazon Managed Grafana or Grafana Cloud pointed at query runtime and file counts will show the trend long before an analyst complains.</li>


<li>Keep a small, cheap reprocessing rig. A modest VPS from a provider like Contabo or InterServer is enough to dry-run a blueprint change against a sample corpus before you spend Bedrock invocations on the full set.</li>

</ul>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Do I need a lakehouse to combine structured and unstructured data on AWS?</h3>



<p class="wp-block-paragraph">No. You need a stable join key and a place to put extraction output. Athena over Iceberg tables in S3 is enough for most workloads. The lakehouse framing matters when you have several engines and warehouses to unify, not when you have documents and one analytics database.</p>



<h3 class="wp-block-heading">Should extraction output live in the same table as the structured source data?</h3>



<p class="wp-block-paragraph">Separate tables, joined at query time. They have different refresh cadences, different failure modes and different trust levels. Merging them into one table means a bad extraction run corrupts data that had nothing wrong with it.</p>



<h3 class="wp-block-heading">Is S3 Vectors a replacement for OpenSearch?</h3>



<p class="wp-block-paragraph">For cost-sensitive storage of large vector sets with moderate query volume, yes. For high query throughput, rich filtering or hybrid search, OpenSearch or pgvector remains the stronger fit. Tiering between them is a supported pattern and often the honest answer.</p>



<h3 class="wp-block-heading">How do I keep the vector index in sync with the documents?</h3>



<p class="wp-block-paragraph">Drive both from the same ingestion event. S3 event notifications into a queue, then fan out to extraction and embedding. Reconcile periodically by comparing document IDs present in the table against those present in the index, because &#8220;mostly in sync&#8221; degrades quietly.</p>



<h3 class="wp-block-heading">What breaks first when this architecture is done badly?</h3>



<p class="wp-block-paragraph">Trust. Long before anything errors, someone finds a number that disagrees with the source document and stops using the dashboard. The technical failure is downstream of the social one.</p>



<h3 class="wp-block-heading">Can I use Textract instead of Bedrock Data Automation?</h3>



<p class="wp-block-paragraph">Yes, and for pure OCR and form or table extraction with predictable layouts it is still a good fit. Bedrock Data Automation is aimed at mixed modalities and schema-shaped custom output through blueprints. The architectural advice here does not change either way, because the failure families are about identity and governance, not about which extractor you picked.</p>



<h3 class="wp-block-heading">Does Lake Formation govern my documents as well as my tables?</h3>



<p class="wp-block-paragraph">It governs the catalog and the tables registered in it. Raw objects in a general purpose bucket are governed by S3 and IAM, and vectors by IAM on the vector resources. Assume three enforcement points until you have tested otherwise.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">Combining structured and unstructured data on AWS is not a storage problem and it has not been one for a while. Every service in the path works. What fails is identity: the assumption that a file path is a key, that a confidence score is a fact, and that a rerun is idempotent.</p>



<p class="wp-block-paragraph">Mint an ID at the door, carry it into every plane, keep the score and the provenance, and model extraction output as an opinion with a timestamp. Do that and the rest of the architecture is ordinary engineering. Skip it and you will spend a year explaining numbers that were never wrong, exactly.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help wiring documents and tables together?</h2>



<p class="wp-block-paragraph">This is a large part of what I do. If you are standing one of these up, or unpicking one that has drifted, I can help with:</p>



<ul class="wp-block-list">

<li>Designing document identity and provenance so extraction output survives reprocessing, overwrites and audits.</li>


<li>Building the Glue, Athena and Iceberg side of a mixed-data pipeline, including merge strategy and table maintenance that does not rot.</li>


<li>Setting up Bedrock Data Automation or Textract extraction with confidence handling and a human review path that ranks by risk.</li>


<li>Getting metadata filtering and tenant isolation right on S3 Vectors, OpenSearch or Bedrock Knowledge Bases before the index is locked in.</li>


<li>Reconciling Lake Formation grants against what a retrieval endpoint can actually surface, and closing the gaps.</li>


<li>Reviewing an existing lakehouse for the failure families above and giving you a prioritised list rather than a rewrite.</li>

</ul>



<p class="wp-block-paragraph">Send me a table schema, a sample extraction output, or the query that is returning the wrong number, and I will tell you what I see.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/combining-structured-unstructured-data-aws/">The Join Is the Hard Part: Combining Structured and Unstructured Data on AWS</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Turning External Documents Into Structured Data With Amazon Textract</title>
		<link>https://john-nessime.com/blog/technical-guides/amazon-textract-structured-data/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Tue, 22 Sep 2026 06:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon SNS]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Lambda]]></category>
		<category><![CDATA[Bedrock Data Automation]]></category>
		<category><![CDATA[Boto3]]></category>
		<category><![CDATA[Confidence Scoring]]></category>
		<category><![CDATA[Data Quality]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Idempotency]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[Pipeline Design]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Schema Drift]]></category>
		<category><![CDATA[Serverless]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=390</guid>

					<description><![CDATA[<p>Amazon Textract almost never fails loudly. A working guide to turning external PDFs and scans into structured data: picking the right operation per document family, parsing the block graph, routing on per-field confidence, and catching the limits that silently truncate your records.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/amazon-textract-structured-data/">Turning External Documents Into Structured Data With Amazon Textract</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">A vendor sends over a year of delivery notes. It arrives as one PDF, several hundred pages, scanned on a machine that was clearly overdue for service. Someone asks how long it will take to get the reference numbers into the warehouse table, and the honest answer is that the OCR call is the easy part.</p>



<p class="wp-block-paragraph">Document pipelines are not hard because optical character recognition is inaccurate. Amazon Textract reads text well. They are hard because everything after the API call belongs to you, and because Textract almost never fails loudly. You get a 200 response, a list of blocks, a confidence score, and a record with an empty field where the purchase order number should have been. Nothing alerts. The row lands in the warehouse with a null, and someone in finance notices three weeks later while reconciling.</p>



<p class="wp-block-paragraph">That silent gap is the whole problem. Getting reliable Amazon Textract structured data out of files you did not design has less to do with OCR accuracy than with everything around it: which operation to call for which document family, why the response is a graph rather than a record, how to route on confidence instead of trusting it, and the limits that quietly truncate your data. All of it assumes documents you cannot ask anyone to change.</p>



<h2 class="wp-block-heading">Why external documents break pipelines that passed testing</h2>



<p class="wp-block-paragraph">Internal documents are generated by a template you control. External documents vary along three axes at once, and each one has its own failure mode:</p>



<ul class="wp-block-list">
<li><strong>Container.</strong> One logical document per file, or forty stapled into one PDF, or a single page split across three JPEGs.</li>

<li><strong>Text origin.</strong> Born-digital PDFs with a real text layer, flatbed scans, and photographs taken at an angle in bad light. Same file extension, wildly different accuracy.</li>

<li><strong>Layout stability.</strong> The vendor redesigns their invoice template and moves the PO number from the header to a footer block. Nobody tells you.</li>
</ul>



<p class="wp-block-paragraph">You cannot fix any of that upstream. You can only detect it at ingest and route accordingly. The sections below are organized by the failure family each one produces.</p>



<h2 class="wp-block-heading">Failure family one: the file is out of spec before Textract sees it</h2>



<p class="wp-block-paragraph">Textract accepts JPEG, PNG, PDF, and TIFF. It does not support XFA-based PDFs, and it will not open a password-protected PDF. Images are accepted up to 10,000 pixels on any side. For PDFs, the maximum page dimensions are 40 inches and 9,000 points.</p>



<p class="wp-block-paragraph">Two more constraints catch people out. Text detection covers English, French, German, Italian, Portuguese, and Spanish, and several capabilities are narrower still: handwriting, invoices and receipts, identity documents, and Queries are English only. Vertical text, common in Japanese and Chinese documents, is not supported at all. Rotation is fine, including heavy in-plane rotation. Vertical writing direction is not.</p>



<p class="wp-block-paragraph">Build an ingest gate that checks encryption, page count, byte size, and dimensions before it spends an API call, and give each rejection a reason code a human can act on. Then catch exceptions properly instead of wrapping the call in a bare retry. <code>BadDocumentException</code> means Textract could not read the file, and retrying fails identically. <code>DocumentTooLargeException</code> means route to the asynchronous path or split the file. <code>InvalidS3ObjectException</code> is usually IAM or a bucket-region mismatch, not a document problem. Only <code>InternalServerErrorException</code> deserves a retry with backoff.</p>



<h2 class="wp-block-heading">Failure family two: synchronous versus asynchronous is a correctness gate</h2>



<p class="wp-block-paragraph">This is the one that silently eats data, so it goes early.</p>



<p class="wp-block-paragraph">The synchronous operations, <code>DetectDocumentText</code> and <code>AnalyzeDocument</code>, cap at 10 MB in memory. For PDF and TIFF input they also cap at <strong>one page</strong>. Not one page per call with automatic paging. One page, full stop. Hand a synchronous call a 40-page PDF and you get page one, and a pipeline that reports success.</p>



<p class="wp-block-paragraph">The asynchronous operations are a different shape. PDF and TIFF go up to 500 MB and 3,000 pages, while JPEG and PNG stay at 10 MB. You start a job, Textract works in the background, and you collect results by job ID.</p>



<p class="wp-block-paragraph">A synchronous call looks like this. The document lives in S3, and you name the features you want:</p>



<pre class="wp-block-code"><code>aws textract analyze-document 
  --document '{"S3Object":{"Bucket":"my-doc-bucket","Name":"inbound/note-0042.png"}}' 
  --feature-types '["FORMS","TABLES"]' 
  --region us-east-1</code></pre>



<p class="wp-block-paragraph">The asynchronous version returns a job identifier immediately and publishes completion to an SNS topic. That is what you want in production, because polling a job inside a Lambda function means paying for a function that sleeps:</p>



<pre class="wp-block-code"><code>import boto3

textract = boto3.client("textract")

start = textract.start_document_analysis(
    DocumentLocation={
        "S3Object": {"Bucket": "my-doc-bucket", "Name": "inbound/batch-0042.pdf"}
    },
    FeatureTypes=["FORMS", "TABLES"],
    NotificationChannel={
        "SNSTopicArn": "arn:aws:sns:us-east-1:111122223333:textract-jobs",
        "RoleArn": "arn:aws:iam::111122223333:role/TextractSnsPublishRole",
    },
)

job_id = start["JobId"]</code></pre>



<p class="wp-block-paragraph">Results come back paginated, and forgetting the continuation token is a second, quieter way to lose most of a file:</p>



<pre class="wp-block-code"><code>blocks = []
next_token = None

while True:
    kwargs = {"JobId": job_id}
    if next_token:
        kwargs["NextToken"] = next_token

    page = textract.get_document_analysis(**kwargs)
    blocks.extend(page["Blocks"])

    next_token = page.get("NextToken")
    if not next_token:
        break</code></pre>



<p class="wp-block-paragraph">Check <code>JobStatus</code> before you trust <code>Blocks</code>. A job can come back partially successful, which means some pages processed and some did not. If you treat that as success you will publish a document that is missing its middle.</p>



<h2 class="wp-block-heading">Failure family three: the response is a graph, not a record</h2>



<p class="wp-block-paragraph">Textract does not return fields. It returns a flat list of <code>Block</code> objects, each with an ID and relationships pointing at other IDs. Reassembling those into a row is your code, and it is where most of the real work lives.</p>



<p class="wp-block-paragraph">For form data, the shape is specific. A key-value pair is two blocks of type <code>KEY_VALUE_SET</code>, distinguished by their <code>EntityTypes</code> array containing either <code>KEY</code> or <code>VALUE</code>. The key block carries a relationship of type <code>VALUE</code> holding the ID of its value block, and a relationship of type <code>CHILD</code> holding the IDs of the <code>WORD</code> blocks that spell out the label. The value block carries only <code>CHILD</code> relationships pointing at its own words. So reading one field means three hops.</p>



<pre class="wp-block-code"><code>blocks_by_id = {b["Id"]: b for b in blocks}

def child_text(block):
    parts = []
    for rel in block.get("Relationships", []):
        if rel["Type"] != "CHILD":
            continue
        for cid in rel["Ids"]:
            child = blocks_by_id[cid]
            if child["BlockType"] == "WORD":
                parts.append(child["Text"])
            elif child["BlockType"] == "SELECTION_ELEMENT":
                parts.append(child["SelectionStatus"])
    return " ".join(parts)

fields = {}
for b in blocks:
    if b["BlockType"] != "KEY_VALUE_SET":
        continue
    if "KEY" not in b.get("EntityTypes", []):
        continue

    value_text = ""
    for rel in b.get("Relationships", []):
        if rel["Type"] == "VALUE":
            for vid in rel["Ids"]:
                value_text = child_text(blocks_by_id[vid])

    fields[child_text(b)] = (value_text, b["Confidence"])</code></pre>



<p class="wp-block-paragraph">Two things about that snippet. The <code>SELECTION_ELEMENT</code> branch matters because checkboxes and radio buttons are values too, and their content is a selection status rather than text. And the confidence on the key block is not a separate key confidence: Textract returns the same value for both halves of a pair. If you planned to compare label confidence against value confidence to catch mismatches, that signal does not exist.</p>



<p class="wp-block-paragraph">Tables follow the same pattern with different block types, and cells carry row and column indices plus span values for merged cells. Layout adds another set, including title, header, footer, section header, page number, list, and figure blocks, and it sequences text into human reading order. That last property is why Layout is worth requesting when the output is headed for a language model rather than a database: chunking on reading order beats chunking on raw line order.</p>



<p class="wp-block-paragraph">Before hand-rolling any of this, look at the AWS-maintained parsers. The <code>amazon-textract-response-parser</code> package and the higher-level <code>amazon-textract-textractor</code> library both turn the block graph into objects with sensible properties. Writing your own traversal is a good way to understand the model and a poor way to spend two sprints.</p>



<h2 class="wp-block-heading">Getting Amazon Textract structured data out of the right API</h2>



<p class="wp-block-paragraph">Textract is a family of operations, not one endpoint, and the biggest single accuracy win available is usually picking a narrower one. Features are billed per page and they stack, so requesting <code>TABLES</code> on a corpus with no tables costs money and returns noise.</p>



<ul class="wp-block-list">
<li><strong>DetectDocumentText.</strong> Raw text and geometry. Use it when you only need a search index or a text layer.</li>

<li><strong>AnalyzeDocument with FORMS.</strong> Key-value pairs discovered from labels on the page. Works when the label is physically present and stable.</li>

<li><strong>AnalyzeDocument with TABLES.</strong> Grid reconstruction with row and column indices. The right call for line-item grids and statements.</li>

<li><strong>AnalyzeDocument with QUERIES.</strong> You ask natural language questions and get answers back. This is the fix for layout drift.</li>

<li><strong>AnalyzeDocument with LAYOUT and SIGNATURES.</strong> Reading order and structural elements; signature locations for compliance checks.</li>

<li><strong>AnalyzeExpense.</strong> Purpose-built for invoices and receipts, with normalized field types.</li>

<li><strong>AnalyzeID.</strong> Purpose-built for identity documents.</li>
</ul>



<h3 class="wp-block-heading">Why Queries beats FORMS on documents you don&#8217;t control</h3>



<p class="wp-block-paragraph">FORMS gives you whatever labels the document happens to contain. That means your downstream schema is hostage to a vendor&#8217;s wording. When they change &#8220;P.O. No.&#8221; to &#8220;Order Reference&#8221;, your parser looks for a key that no longer exists and writes a null.</p>



<p class="wp-block-paragraph">Queries inverts this. You supply the question and an alias, and the alias comes back attached to the answer, so your output key is fixed by you rather than by the document:</p>



<pre class="wp-block-code"><code>aws textract analyze-document 
  --document '{"S3Object":{"Bucket":"my-doc-bucket","Name":"inbound/note-0042.png"}}' 
  --feature-types '["QUERIES"]' 
  --queries-config '{"Queries":[
      {"Text":"What is the purchase order number?","Alias":"PO_NUMBER"},
      {"Text":"What is the delivery date?","Alias":"DELIVERY_DATE"}
  ]}' 
  --region us-east-1</code></pre>



<p class="wp-block-paragraph">The alias is the entire point. It is a contract between your schema and the extraction layer that survives a vendor redesign. There are ceilings: 15 queries per page synchronously, 30 per page asynchronously, English only. Phrase questions using words that actually appear on the page, and add positional hints when you know the layout, because the model uses them. If Queries underperforms on your corpus, adapters let you tune the feature against your own examples and pass an adapter ID and version at call time.</p>



<h3 class="wp-block-heading">Invoices are their own problem</h3>



<p class="wp-block-paragraph">For invoices and receipts, reach for AnalyzeExpense first. Its response is shaped around the domain rather than around the page: each <code>ExpenseDocument</code> holds <code>SummaryFields</code> for header-level data and <code>LineItemGroups</code> containing the line items. Fields carry a normalized <code>Type</code> such as <code>VENDOR_NAME</code>, plus a <code>ValueDetection</code> and an optional <code>LabelDetection</code>.</p>



<p class="wp-block-paragraph">That optional label is the thing to notice. AnalyzeExpense can identify a vendor name printed only inside a logo, and pull line items out of a grid with no column headers. FORMS structurally cannot do either, because FORMS needs a key to exist. On receipts that is not a small accuracy delta, it is the difference between a usable field and an empty one.</p>



<h2 class="wp-block-heading">Failure family four: confidence is per field and it is not your risk model</h2>



<p class="wp-block-paragraph">Every extracted element carries a confidence score. Two mistakes follow from that.</p>



<p class="wp-block-paragraph">The first is averaging into a document-level score. A document where every field is strong except the invoice total is not a mostly-good document. It is a document with one field that must not be trusted, surrounded by noise. Score fields, not documents.</p>



<p class="wp-block-paragraph">The second is treating one threshold as universal. A misread address on a delivery note is an annoyance. A misread bank account number is an incident. Set thresholds per field from the cost of being wrong, then route into three lanes:</p>



<ol class="wp-block-list">
<li><strong>Auto-accept.</strong> Above threshold and passing a format check. Straight into the target table.</li>

<li><strong>Review.</strong> Below threshold, or above it but failing a validation rule. A human sees the value and the cropped region it came from.</li>

<li><strong>Reject.</strong> Field absent entirely, or the document failed the ingest gate. Back to the sender with a reason.</li>
</ol>



<p class="wp-block-paragraph">Validation rules do more work than thresholds here. A total that does not equal the sum of its line items is wrong at any confidence. Check-digit rules on account and tax identifiers catch character-level errors the model was perfectly confident about. Confidence tells you how sure the model is. Validation tells you whether the answer is even possible.</p>



<p class="wp-block-paragraph">Textract can hand low-confidence results to Amazon Augmented AI through a human loop configuration on the API call, which saves building a review queue. Whether that beats a small internal tool depends on how much domain context your reviewers need on screen. Either way, capture the corrections. Reviewer edits are the only labeled data you get for free, and they are how you calibrate thresholds against your own corpus instead of guessing.</p>



<h2 class="wp-block-heading">Failure family five: reprocessing you didn&#8217;t budget for</h2>



<p class="wp-block-paragraph">Textract bills per page, per feature. Your parser, meanwhile, will change repeatedly in the first months of a pipeline&#8217;s life, because that is what parsers do.</p>



<p class="wp-block-paragraph">So separate extraction from parsing, permanently. Write the raw response JSON to S3 keyed by a hash of the document bytes plus the operation and feature set requested, then parse from that stored artifact. Reprocessing becomes free, deterministic, and diffable: you can replay a parser change across the whole back catalogue and compare outputs before promoting it. Without that cache, every parser bug fix is a bill.</p>



<pre class="wp-block-code"><code>import hashlib

digest = hashlib.sha256(doc_bytes).hexdigest()
features = "-".join(sorted(["FORMS", "TABLES"]))
key = f"textract-raw/{digest}/analyze-document/{features}.json"</code></pre>



<p class="wp-block-paragraph">Hashing content rather than filename also deduplicates the same scan arriving twice under different names, which happens constantly with email-driven intake. For the asynchronous operations, a client request token gives idempotency at the job level; reusing a token with different parameters raises a mismatch error rather than quietly starting a second job.</p>



<p class="wp-block-paragraph">One caveat: source documents often contain personal data, so a cache is a retention decision. Set S3 lifecycle rules on both the originals and the raw JSON, and if working copies land on a workstation or a build box, delete them properly. Tools like O&amp;O SafeErase exist for exactly that gap between &#8220;moved to trash&#8221; and &#8220;actually gone&#8221;.</p>



<h2 class="wp-block-heading">Where Textract stops being the right tool</h2>



<p class="wp-block-paragraph">Being fair about this matters, because AWS now offers overlapping paths to the same outcome. Amazon Bedrock Data Automation is a managed document processing service that handles classification and extraction behind one API, priced per document rather than per page and per feature. AWS positions it as the default starting point for new builds, and for mixed document types with no dominant format that recommendation is sound. You write far less orchestration.</p>



<p class="wp-block-paragraph">Textract keeps the advantage where the corpus is high volume and standardized, where geometry and per-field confidence are needed for auditability, where regional availability constrains you, or where you want extraction to be deterministic and separately testable. Bounding boxes and confidence scores are not a nice-to-have in a regulated workflow, they are the evidence trail.</p>



<p class="wp-block-paragraph">The hybrid is common and sensible: Textract for extraction, a Bedrock model for reasoning over the extracted text. Just do not send a multimodal model a full-page image because OCR felt inconvenient. It costs considerably more per page and gives you no geometry.</p>



<h2 class="wp-block-heading">Troubleshooting</h2>



<ul class="wp-block-list">
<li><strong>Only the first page came through.</strong> A synchronous call on a multi-page PDF or TIFF. Move to the asynchronous operations, and assert the page count Textract reports against the count you expected.</li>

<li><strong>Output truncates partway through a long document.</strong> Missing continuation token handling on the get operation. Loop until the token is absent.</li>

<li><strong>Fields visible on the page never appear in output.</strong> The label is implied rather than printed, or sits inside a logo. Switch that field to Queries, or to AnalyzeExpense if it is an invoice.</li>

<li><strong>Quality collapsed for one sender only.</strong> Template change. Diff the current layout against a stored sample, which is the argument for keeping raw responses.</li>

<li><strong>Throttling during a backfill.</strong> Transactions per second and concurrent job counts are per-account, per-region quotas. Queue submissions rather than fanning out, and request an increase before the migration.</li>

<li><strong>Table cells land in the wrong columns.</strong> Usually a skewed scan. Deskew and raise contrast before submitting, because image preparation is cheaper than post-processing repairs.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Testing on three clean sample documents supplied by the vendor rather than a random sample of real intake.</li>

<li>Requesting every feature type on every page because it is simpler than routing, then being surprised by the bill.</li>

<li>Using FORMS for invoices when AnalyzeExpense exists and understands unlabeled fields.</li>

<li>Treating a confidence score as an accuracy guarantee instead of pairing it with validation rules.</li>

<li>Discarding the raw JSON after parsing, which turns every parser fix into a re-extraction charge.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Classify before you extract. Route each document to the narrowest API that covers it.</li>

<li>Use Queries with explicit aliases for fields your schema depends on, so vendor wording changes cannot break your column names.</li>

<li>Store raw responses immutably, keyed by content hash, and parse from storage.</li>

<li>Set per-field confidence thresholds from the cost of being wrong, and pair every threshold with a format or arithmetic validation.</li>

<li>Instrument it like any other pipeline: extraction latency, straight-through rate, review queue depth, per-field null rate. A slow rise in nulls on one field is the earliest signal of a template change. Prometheus with Grafana, or Grafana Cloud if you would rather not run it yourself, covers this well.</li>

<li>Keep documents and pipeline in the same region, use VPC endpoints for sensitive data, and apply lifecycle rules to originals and extracted output alike.</li>

<li>Keep a golden set of real documents with known correct answers, and run it on every parser change.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Can Amazon Textract process multi-page PDFs?</h3>



<p class="wp-block-paragraph">Yes, but only through the asynchronous operations. Synchronous calls accept a single page for PDF and TIFF input. Asynchronous processing handles PDF and TIFF up to 500 MB and 3,000 pages.</p>



<h3 class="wp-block-heading">What is the difference between FORMS and Queries?</h3>



<p class="wp-block-paragraph">FORMS returns whatever key-value pairs it finds from labels printed on the page, so your output keys follow the document. Queries lets you ask for values by natural language question and attach your own alias to each answer, so output keys stay fixed when the layout changes. For a schema you must keep stable, Queries is the safer contract.</p>



<h3 class="wp-block-heading">How much does Amazon Textract cost?</h3>



<p class="wp-block-paragraph">Billing is per page and varies by operation and feature, and requesting several features on one page adds their costs together. Check the current rate card for your region and multiply by pages, not documents. The practical lever is routing: send each document to the narrowest operation that answers your question, and cache raw responses so parser iteration is free.</p>



<h3 class="wp-block-heading">Does Textract handle handwriting?</h3>



<p class="wp-block-paragraph">It detects handwritten text, in English only. Treat handwritten fields as a separate confidence tier with lower auto-accept thresholds, because the variance across writers is much wider than for printed text.</p>



<h3 class="wp-block-heading">Which languages does Amazon Textract support?</h3>



<p class="wp-block-paragraph">Text detection covers English, French, German, Italian, Portuguese, and Spanish. Handwriting, invoices and receipts, identity documents, and Queries are English only. Vertically written text is not supported.</p>



<h3 class="wp-block-heading">Should I use Textract or Bedrock Data Automation?</h3>



<p class="wp-block-paragraph">Bedrock Data Automation is the lower-effort default for mixed document types, with classification and extraction behind a single managed API. Textract fits better at high volume on standardized documents, where per-field confidence and geometry matter for audit, or where extraction needs to be deterministic and independently testable. Combining them is normal.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Conclusion</h2>



<p class="wp-block-paragraph">If one thing survives this post, make it this: Amazon Textract structured data extraction fails quietly, not loudly. The API returns success while your record is missing its most important field, because a synchronous call read one page of forty, or because a vendor renamed a label, or because a value came back at low confidence and nothing was watching.</p>



<p class="wp-block-paragraph">So build for the silence. Gate at ingest, use asynchronous operations by default, pick the narrowest API per document family, pin your schema with query aliases, store the raw response, and route on per-field confidence backed by validation rules that know what a possible answer looks like. Do that and the OCR really does become the easy part.</p>



<h2 class="wp-block-heading">Need help turning documents into a pipeline you can trust?</h2>



<p class="wp-block-paragraph">Most of the difficulty in document extraction shows up after the proof of concept, once real intake replaces the sample files. That is the part I work on:</p>



<ul class="wp-block-list">
<li>Designing the ingest gate and classification step so each document family reaches the right Textract operation instead of one call doing everything.</li>

<li>Building the asynchronous path properly: S3 events, SNS completion, paginated retrieval, dead-letter handling, idempotent reprocessing.</li>

<li>Replacing brittle FORMS parsing with query aliases or AnalyzeExpense, and validating the change against a golden set of your own documents.</li>

<li>Setting per-field confidence thresholds and validation rules, then wiring a review queue whose corrections feed back into tuning.</li>

<li>Adding observability that catches template drift early: per-field null rates, straight-through rate, and review queue depth on a dashboard.</li>

<li>Getting the cost curve under control through routing, raw-response caching, and lifecycle rules on stored documents and JSON.</li>
</ul>



<p class="wp-block-paragraph">If you have a pipeline misbehaving, send me a sample document with the fields you need and the raw Textract JSON it produced. That is usually enough to see where it is going wrong.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/amazon-textract-structured-data/">Turning External Documents Into Structured Data With Amazon Textract</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>CV Parsing at Scale on AWS: Textract, Bedrock, and the Bias You Have to Document</title>
		<link>https://john-nessime.com/blog/cloud-computing/cv-parsing-aws-textract-bedrock-bias/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sun, 30 Aug 2026 09:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Compliance]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Adverse Impact]]></category>
		<category><![CDATA[Algorithmic Bias]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[Bedrock Guardrails]]></category>
		<category><![CDATA[Confidence Scoring]]></category>
		<category><![CDATA[EU AI Act]]></category>
		<category><![CDATA[HR Technology]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Local Law 144]]></category>
		<category><![CDATA[Model Invocation Logging]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[PII Redaction]]></category>
		<category><![CDATA[Resume Parsing]]></category>
		<category><![CDATA[Structured Outputs]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=540</guid>

					<description><![CDATA[<p>A CV parsing pipeline stops being a parser the moment a score sorts a list. Where Textract and Bedrock actually fit, why redacting names does nothing for bias, and what to document before anyone asks you to explain a ranking.</p>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/cv-parsing-aws-textract-bedrock-bias/">CV Parsing at Scale on AWS: Textract, Bedrock, and the Bias You Have to Document</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The ticket said &#8220;parser is working fine.&#8221; It was. Extraction confidence was high, the dead letter queue was empty, and recruiting liked the shortlists. Then someone in legal asked why candidate 4,412 ranked below candidate 118.</p>



<p class="wp-block-paragraph">Nobody could answer. Not because anything was broken, but because the system had quietly stopped being a parser somewhere between the S3 upload and the sorted list on a recruiter&#8217;s screen. It had become a selection procedure, and nothing in it was designed to explain itself.</p>



<p class="wp-block-paragraph">That is the failure mode worth leading with, because it never pages you. Your alarms stay green while the thing you built lands in a category of regulated system you never intended to build.</p>



<p class="wp-block-paragraph">This post covers CV parsing on AWS as it actually goes: what Amazon Textract gives you and what it doesn&#8217;t, where Bedrock belongs and where it becomes a liability, the bias problem you have to document rather than solve, and the failures that only surface at volume.</p>



<h2 class="wp-block-heading">When CV parsing on AWS becomes a hiring decision</h2>



<p class="wp-block-paragraph">Most of these pipelines start honestly. Someone wants structured JSON out of a PDF pile so the ATS populates fields instead of a coordinator retyping them. Extraction is a data problem with clean success criteria.</p>



<p class="wp-block-paragraph">Then a product manager asks for a match score. It seems small. You already have the skills list and the job description, so you ask a model how well they line up and emit a number. The API contract barely changes.</p>



<p class="wp-block-paragraph">What changed is everything. The moment a recruiter sorts by that number and works down from the top, the pipeline substantially assists a hiring decision. Under New York City&#8217;s Local Law 144 that is an automated employment decision tool, and the obligations attach to the employer using it, not the vendor who wrote the code. Under the EU AI Act, Annex III point 4 lists systems intended to analyse and filter job applications and evaluate candidates as high-risk by classification, not by how confident you are in your prompt.</p>



<p class="wp-block-paragraph">The EU date is worth stating plainly because it moved. Annex III obligations were originally due to apply from 2 August 2026; the Digital Omnibus package pushed stand-alone Annex III systems out to 2 December 2027. That is a reprieve on paperwork, not on design. Systems procured now will still be running then, and the separate prohibition on workplace emotion-recognition AI, plus existing GDPR duties around automated decisions, were never on that timetable.</p>



<p class="wp-block-paragraph">So the first architectural decision isn&#8217;t about services. It&#8217;s about where extraction stops and judgement begins, and whether that boundary is visible in the code or buried in a prompt.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Getting text out: what Textract actually does</h2>



<p class="wp-block-paragraph">Textract is good at CVs in a narrow way. It reads printed and handwritten text, detects tables and form-style key-value pairs, and with the Layout feature type it identifies structural elements such as titles, paragraphs, lists, headers and footers. For a two-column CV with a skills sidebar, layout awareness is the difference between usable text and word salad.</p>



<h3 class="wp-block-heading">Sync versus async is not a performance choice</h3>



<p class="wp-block-paragraph"><code>AnalyzeDocument</code> is synchronous and takes a single image or a document passed as bytes or an S3 object. <code>StartDocumentAnalysis</code> is the asynchronous entry point for multipage documents; it publishes completion status to an SNS topic you drain from SQS before calling <code>GetDocumentAnalysis</code> with the returned job ID.</p>



<p class="wp-block-paragraph">People pick sync because it&#8217;s easier to reason about, then discover half of real CVs are three pages. Go async from the start. The SNS-to-SQS hop isn&#8217;t overhead, it&#8217;s the only thing that gives you a retry surface when a job fails partway through a batch.</p>



<h3 class="wp-block-heading">Queries beat walking the block graph by hand</h3>



<p class="wp-block-paragraph">The Queries feature lets you ask natural-language questions of a document without knowing where the data sits on the page. Pass <code>QUERIES</code> in <code>FeatureTypes</code>, supply a <code>QueriesConfig</code>, then read the <code>QUERY_RESULT</code> blocks out of the response.</p>



<pre class="wp-block-code"><code>aws textract start-document-analysis 
  --document '{"S3Object":{"Bucket":"bucket","Name":"document"}}' 
  --feature-types '["QUERIES"]' 
  --queries-config '{"Queries":[{"Text":"Question"}]}'</code></pre>



<p class="wp-block-paragraph">Each query can carry an alias so answers map back to your schema without positional guessing. In Python, walk the returned blocks and pick the ones whose <code>BlockType</code> is <code>QUERY_RESULT</code>, each carrying answer text and a confidence score.</p>



<pre class="wp-block-code"><code>response = client.analyze_document(
    Document={'S3Object': {'Bucket': bucket, 'Name': document}},
    FeatureTypes=["TABLES", "FORMS", "QUERIES"],
    QueriesConfig={'Queries': [{'Text': question}]}
)

for block in response['Blocks']:
    if block["BlockType"] == "QUERY_RESULT":
        print(block["Text"])</code></pre>



<p class="wp-block-paragraph">Use Queries for facts that exist verbatim on the page: employer name, job title, dates, certification number. Not for anything requiring inference. &#8220;What is this person&#8217;s seniority level?&#8221; is a judgement, and pushing it through an extraction API hides that judgement inside something that looks like OCR.</p>



<h2 class="wp-block-heading">Where Bedrock belongs, and where it stops</h2>



<p class="wp-block-paragraph">Bedrock earns its place in one job: turning messy extracted text into a consistent schema. CVs are the worst kind of semi-structured document because the structure is a fashion trend. One person lists dates as &#8220;Mar 2019 to present,&#8221; another renders them as a bar chart. Normalising that is exactly the fuzzy transformation a language model handles well, and one you can verify.</p>



<ol class="wp-block-list">
<li>Constrain output to a schema and reject anything that fails validation. A job title you didn&#8217;t ask for, or an employment date absent from the source text, is a failed record rather than a creative flourish.</li>



<li>Turn on model invocation logging before you process a single real CV. It&#8217;s disabled by default, and it&#8217;s the only native record of what you sent and what came back.</li>
</ol>



<p class="wp-block-paragraph">That second point isn&#8217;t a nice-to-have. When someone asks a year from now why a candidate scored the way they did, the invocation log is your answer or you don&#8217;t have one.</p>



<pre class="wp-block-code"><code>aws bedrock put-model-invocation-logging-configuration 
  --logging-config '{
    "s3Config": {"bucketName": "my-bedrock-logs", "keyPrefix": "cv-pipeline/"},
    "textDataDeliveryEnabled": true
  }'</code></pre>



<p class="wp-block-paragraph">Logs go to CloudWatch Logs, S3, or both, and only destinations in the same account and Region are supported. Apply lifecycle rules and encryption up front, because these logs contain full CVs and they outlive the hiring round that produced them.</p>



<p class="wp-block-paragraph">Bedrock Guardrails fits here through sensitive information filters, which detect PII in prompts and responses and either block the payload or mask individual entities with identifier tags. There&#8217;s a detect mode that reports what a guardrail would have caught without acting, which is the right way to tune against real documents first. Don&#8217;t oversell it to yourself: masking is not anonymisation, and a filter firing on <code>NAME</code> and <code>EMAIL</code> does very little about bias, for reasons that are the whole point of the next section.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The bias problem you must document, not solve</h2>



<p class="wp-block-paragraph">Every engineer&#8217;s instinct is to strip the protected attributes and declare it handled. Remove the name, the photo, the address, and the model can&#8217;t discriminate on what it can&#8217;t see. It doesn&#8217;t work, and understanding why separates a pipeline you can defend from one you can&#8217;t.</p>



<h3 class="wp-block-heading">Proxies survive redaction</h3>



<p class="wp-block-paragraph">A CV is dense with correlated signal. University name tracks geography and socioeconomics. Graduation year is an age proxy that survives every name redaction you can write. Employment gaps correlate with caregiving. Postcode correlates with almost everything. Add professional association membership, the language a qualification is written in, whether military service appears, the phrasing conventions of a non-native English speaker.</p>



<p class="wp-block-paragraph">None of those are protected attributes. All of them carry information about protected attributes, and a model optimising for similarity to your existing successful hires will use them. Redaction removes the label, not the signal, and it makes disparity harder to detect because you deleted the field you would have measured against.</p>



<blockquote class="wp-block-quote is-layout-flow wp-block-quote-is-layout-flow">
<p class="wp-block-paragraph">You can&#8217;t prove a screening pipeline is unbiased. You can measure its outcomes, record what you measured, and act when the numbers move.</p>
</blockquote>



<h3 class="wp-block-heading">What the four-fifths rule actually asks for</h3>



<p class="wp-block-paragraph">The Uniform Guidelines on Employee Selection Procedures set out the rule of thumb federal enforcement agencies apply: a selection rate for any race, sex or ethnic group falling below four-fifths of the rate for the highest group is generally treated as evidence of adverse impact. It sits at 29 CFR 1607.4(D).</p>



<ul class="wp-block-list">
<li><strong>It&#8217;s a rule of thumb, not a safe harbour.</strong> The same guidelines note smaller differences can still constitute adverse impact where they&#8217;re significant in statistical and practical terms, and courts have declined to treat clearing 0.8 as proof nothing is wrong.</li>



<li><strong>It applies to the selection procedure.</strong> That means your parser, your scorer and the recruiter&#8217;s sort order taken together. Auditing the model in isolation measures the wrong thing.</li>
</ul>



<p class="wp-block-paragraph">The computation is trivial, which is why there&#8217;s no excuse for skipping it.</p>



<pre class="wp-block-code"><code>SELECT
  demographic_group,
  COUNT(*) AS applicants,
  SUM(CASE WHEN advanced THEN 1 ELSE 0 END) AS selected,
  1.0 * SUM(CASE WHEN advanced THEN 1 ELSE 0 END) / COUNT(*) AS selection_rate
FROM screening_outcomes
WHERE requisition_id = :req
GROUP BY demographic_group;</code></pre>



<p class="wp-block-paragraph">Divide each selection rate by the largest and you have the impact ratio. Anything under 0.8 is a signal to stop and look, not a number to explain away.</p>



<h3 class="wp-block-heading">Where the demographic data comes from</h3>



<p class="wp-block-paragraph">This is the genuinely hard part and where most teams stall. You need demographic data to measure disparity, and you must not feed it into the screening decision. The pattern that works is a strict split: self-reported responses collected separately at application time, stored in a different bucket under a different KMS key, joined to outcomes only in an analysis account the scoring pipeline has no path to.</p>



<p class="wp-block-paragraph">NYC&#8217;s regime allows historical or test data, and requires the audit be performed by someone not involved in using or developing the tool. That independence requirement is easy to miss when you&#8217;re the consultant who built the thing. You can build the measurement harness. You can&#8217;t be the independent auditor of your own pipeline.</p>



<h3 class="wp-block-heading">Document the pipeline, not the model</h3>



<p class="wp-block-paragraph">The artifact that saves you is boring and written down. At minimum: which model and version produced scores in each date range, the scoring logic in force, every field that reached the scorer, every field withheld and why, where human review sits and what a reviewer can override, and the impact ratios from each measurement run.</p>



<p class="wp-block-paragraph">Keep it in version control beside the infrastructure code. A change to the prompt is a change to the selection procedure and should be reviewable as one.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Problems that only appear at volume</h2>



<ul class="wp-block-list">
<li><strong>Throttling arrives as data loss.</strong> Without retries using exponential backoff and jitter, a throttled call becomes a missing candidate rather than an error anyone sees. Assert on record counts at every stage boundary.</li>



<li><strong>Duplicates are the norm.</strong> The same person applies to three roles with two slightly different files. Deduplicate on content hash before you spend on extraction, and keep an idempotency key so a retried job doesn&#8217;t create a second candidate record.</li>



<li><strong>Cost is dominated by re-runs.</strong> Extraction is the expensive step and it&#8217;s deterministic on the same input. Store the raw Textract response in S3 keyed by document hash. Reprocessing after a schema change should cost storage, not inference.</li>



<li><strong>Retention outlives the requisition.</strong> Extracted text, invocation logs and intermediate JSON all contain personal data and all quietly accumulate. Set lifecycle rules at build time, not after the first data subject request.</li>



<li><strong>Scanned CVs are a different pipeline.</strong> Photographed documents behave nothing like exported PDFs. Route them separately with lower confidence thresholds and higher review rates.</li>
</ul>



<p class="wp-block-paragraph">On cost visibility, native tooling shows the shape of spend but is slow to attribute it. If this runs alongside other AI workloads, a dedicated platform such as Vantage or CloudZero makes per-pipeline attribution less painful than slicing Cost Explorer by tag every month. And if the upload front door is public, putting Cloudflare in front of it deals with bot traffic before it becomes Textract spend.</p>



<h2 class="wp-block-heading">Troubleshooting</h2>



<ul class="wp-block-list">
<li><strong>High confidence, wrong fields.</strong> Almost always multi-column layouts read in the wrong order. Add the Layout feature type and check reading order in the block graph against what a human sees.</li>



<li><strong>Async jobs complete but results never arrive.</strong> Check the SNS topic policy allows publishing to your queue and that the role passed to Textract can publish. This fails silently and looks like a stuck pipeline.</li>



<li><strong>The model invents employment history.</strong> Your prompt is asking it to summarise rather than extract. Constrain the schema, require every value trace to source text, and fail validation instead of accepting a plausible answer.</li>



<li><strong>Scores shift after a model update.</strong> Pin the model identifier rather than tracking a moving alias, and re-run your measurement set against a held-out corpus before promoting any change.</li>



<li><strong>Guardrails block legitimate CVs.</strong> Sensitive information filters are probabilistic and context-dependent. Run detect mode against a real corpus and read the traces before switching to block.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating a match score as a data field rather than a decision, and skipping the governance that follows.</li>



<li>Redacting names and considering the bias question closed.</li>



<li>Enabling invocation logging after the pilot, leaving the first thousand decisions with no record.</li>



<li>Building human review that only sees the shortlist, which reviews the output of the bias rather than the bias.</li>



<li>Assuming a US-headquartered employer is out of scope. Both regimes reach based on where the candidate is, not where the company is.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Draw a hard line between extraction and scoring: separate services, separate IAM roles, separate logs.</li>



<li>Enable model invocation logging before the first real document, with encryption and lifecycle rules already applied.</li>



<li>Cache raw extraction output keyed by document hash so reprocessing never re-bills inference.</li>



<li>Collect demographic data separately, key it differently, and join it to outcomes only in an isolated analysis account.</li>



<li>Run impact ratios on a schedule rather than on request, and alert on movement instead of waiting for an audit.</li>



<li>Version prompts and scoring logic in the same repository as the infrastructure, reviewed as changes to a selection procedure.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Is Textract or Bedrock the right tool for CV parsing?</h3>



<p class="wp-block-paragraph">Both, for different halves. Textract turns pixels into text with layout awareness and confidence scores. Bedrock turns that text into a consistent schema. Using a language model for the OCR step gives up per-field confidence, which is exactly the signal you need to decide what goes to human review.</p>



<h3 class="wp-block-heading">Does removing names and photos make a resume screening pipeline compliant?</h3>



<p class="wp-block-paragraph">No. Redaction handles direct identifiers while leaving proxies such as graduation year, institution, postcode and employment gaps intact. It also removes the fields you&#8217;d need to measure disparity, so it can make a pipeline harder to audit while feeling safer.</p>



<h3 class="wp-block-heading">Do these rules apply if my company isn&#8217;t in New York or the EU?</h3>



<p class="wp-block-paragraph">Possibly. Local Law 144 turns on whether the candidate resides in New York City, including for remote roles, rather than where the employer sits. The EU AI Act has similar extraterritorial reach. Check with counsel, not with an architecture diagram.</p>



<h3 class="wp-block-heading">Can I run the bias audit myself?</h3>



<p class="wp-block-paragraph">You can build the measurement pipeline and run it continuously, and you should. The formal NYC bias audit has to be performed by an independent party not involved in using or developing the tool, which rules out the team that built it.</p>



<h3 class="wp-block-heading">How do I keep extraction costs predictable at scale?</h3>



<p class="wp-block-paragraph">Deduplicate before extraction, cache raw responses by content hash, and choose feature types deliberately. Requesting tables, forms and queries on every document when you only need queries multiplies cost for nothing. Both services bill per unit of work, so the lever is doing less work rather than finding a cheaper tier.</p>



<h3 class="wp-block-heading">Should the pipeline auto-reject candidates?</h3>



<p class="wp-block-paragraph">Treat that as a question for counsel, not engineering. GDPR restricts solely automated decisions with significant effects on individuals, and the practical answer in most designs is a meaningful human review before any rejection. Build the human step in early; retrofitting it is painful.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">CV parsing on AWS is two systems wearing one name. The extraction half is an engineering problem with clean success criteria, and Textract plus Bedrock will carry you a long way given retries, caching and schema validation. The scoring half is a selection procedure, governed by rules that don&#8217;t care what you called the service.</p>



<p class="wp-block-paragraph">You won&#8217;t eliminate bias from a system trained on hiring history. What you can do is know where the boundary sits, log what crossed it, measure outcomes on a schedule, and be able to answer the question about candidate 4,412 when it arrives. Build the documentation while you build the pipeline. Reconstructing it later from CloudWatch and memory is a much worse job.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Work with me on document pipelines and AI governance</h2>



<p class="wp-block-paragraph">Most of the work here isn&#8217;t the extraction. It&#8217;s the boundary drawing, the evidence trail and the parts that only break at volume. Things I help with:</p>



<ul class="wp-block-list">
<li>Designing Textract and Bedrock pipelines with an auditable split between extraction and any scoring or ranking step.</li>



<li>Building the measurement harness that computes selection rates and impact ratios on a schedule, with demographic data isolated from the scoring path.</li>



<li>Setting up model invocation logging, retention, encryption and access controls so the evidence trail exists before you need it.</li>



<li>Fixing throughput and cost problems: throttling, deduplication, extraction caching and reprocessing that doesn&#8217;t re-bill inference.</li>



<li>Adding human-in-the-loop review that routes on confidence and score bands, including sampled rejections rather than shortlists only.</li>



<li>Writing the technical documentation an independent auditor or your legal team will actually ask for.</li>
</ul>



<p class="wp-block-paragraph">Send me a sample Textract response, a scoring prompt or an architecture sketch and I&#8217;ll tell you where the boundary is currently blurred.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/cv-parsing-aws-textract-bedrock-bias/">CV Parsing at Scale on AWS: Textract, Bedrock, and the Bias You Have to Document</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Legal Document Intelligence on AWS: The Five Boundaries That Have to Hold</title>
		<link>https://john-nessime.com/blog/cloud-computing/legal-document-intelligence-aws/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Tue, 25 Aug 2026 18:00:00 +0000</pubDate>
				<category><![CDATA[Case Studies]]></category>
		<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Web Security]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[Amazon Comprehend]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[Audit Logging]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS KMS]]></category>
		<category><![CDATA[AWS Organizations]]></category>
		<category><![CDATA[Bedrock Guardrails]]></category>
		<category><![CDATA[Bedrock Knowledge Bases]]></category>
		<category><![CDATA[Cloud Security]]></category>
		<category><![CDATA[Compliance]]></category>
		<category><![CDATA[Data Residency]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Encryption]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[Litigation Hold]]></category>
		<category><![CDATA[Metadata Filtering]]></category>
		<category><![CDATA[Multi-Tenant]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[OpenSearch Serverless]]></category>
		<category><![CDATA[PrivateLink]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[S3 Object Lock]]></category>
		<category><![CDATA[Step Functions]]></category>
		<category><![CDATA[Tenant Isolation]]></category>
		<category><![CDATA[Vector Database]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=272</guid>

					<description><![CDATA[<p>A confident answer with a citation from a matter the reader was walled off from. Nothing crashed, nothing alerted. Building legal document intelligence on AWS with Textract, Bedrock and OpenSearch is the easy half; the hard half is tenant isolation, verified deletion, S3 Object Lock holds, what leaves the account, and an audit trail that names the actual user.</p>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/legal-document-intelligence-aws/">Legal Document Intelligence on AWS: The Five Boundaries That Have to Hold</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The demo was going well until someone asked where the third citation came from.</p>



<p class="wp-block-paragraph">The answer on screen was good. Fluent, specific, correctly hedged, three sources listed underneath it. Then a partner in the room asked a simple question: which matter is source three from? Nobody could answer it in the room, and when we went and looked, it was from a matter that half the people watching the demo were formally walled off from.</p>



<p class="wp-block-paragraph">Nothing had crashed. No alarm fired. The retrieval layer had done exactly what a similarity search does, which is return the nearest vectors, and the nearest vectors did not care about the ethical wall. That is the thing about building a legal document intelligence platform on AWS: the failures that matter almost never look like failures. They look like a confident answer with a citation attached.</p>



<p class="wp-block-paragraph">This post is about the parts of that build that are hard, organised around the five boundaries that actually have to hold. The extraction pipeline is the easy half. I will cover it, but quickly, because the AWS documentation is good and the failure modes are visible. The other four boundaries fail quietly, and that is where the engineering goes.</p>



<h2 class="wp-block-heading">The shape of the thing</h2>



<p class="wp-block-paragraph">Before the boundaries, the skeleton. A legal document intelligence platform on AWS almost always ends up looking like this, whether you plan it or arrive at it:</p>



<ul class="wp-block-list">
<li>Documents land in Amazon S3, one prefix per matter, versioning on.</li>



<li>An S3 event triggers AWS Lambda, which starts an asynchronous Amazon Textract job.</li>



<li>Textract output lands back in S3 as JSON. Something normalises it into text plus page and bounding-box coordinates.</li>



<li>Amazon Comprehend or a foundation model classifies the document and pulls entities: parties, dates, governing law, clause types.</li>



<li>Chunks get embedded and written to a vector store, usually Amazon OpenSearch Serverless behind Amazon Bedrock Knowledge Bases.</li>



<li>An application layer, typically API Gateway plus Lambda, takes a question, retrieves, and calls a model on Amazon Bedrock.</li>
</ul>



<p class="wp-block-paragraph">AWS Step Functions is worth reaching for once you have more than three stages, because Textract jobs are long-running and Lambda timeouts are not a retry strategy. That is the whole architecture. You can stand it up in a fortnight. Then you spend six months on everything below.</p>



<h2 class="wp-block-heading">Boundary one: isolation, and why metadata filtering is not optional</h2>



<p class="wp-block-paragraph">This is the one from the demo, and it is the one that ends engagements.</p>



<p class="wp-block-paragraph">A vector index has no native concept of a matter, a client, or an ethical wall. If you pool every document into one index and rely on the prompt to keep things separate, you have built a system where a well-phrased question can pull privileged material across a wall. Prompt instructions are not an access control. They are a suggestion that usually works.</p>



<p class="wp-block-paragraph">There are two shapes that do work, and the choice between them is a real trade-off rather than a best practice.</p>



<h3 class="wp-block-heading">Pooled index with server-side filtering</h3>



<p class="wp-block-paragraph">Every chunk carries a <code>matter_id</code> and a <code>client_id</code> as metadata. Every retrieval call attaches a filter. With an S3 data source in Bedrock Knowledge Bases, the metadata comes from a sidecar JSON file that sits next to the document and carries a <code>metadataAttributes</code> object.</p>



<pre class="wp-block-code"><code>{
  "metadataAttributes": {
    "matter_id": "M-4417",
    "client_id": "C-108",
    "doc_type": "engagement_letter",
    "privileged": true
  }
}</code></pre>



<p class="wp-block-paragraph">The retrieval call then narrows the search before the model ever sees a chunk:</p>



<pre class="wp-block-code"><code>"retrievalConfiguration": {
  "vectorSearchConfiguration": {
    "filter": {
      "equals": { "key": "matter_id", "value": "M-4417" }
    }
  }
}</code></pre>



<p class="wp-block-paragraph">Two rules make this safe, and both are the kind of thing that gets skipped under deadline. First, the filter value is derived on the server from the authenticated caller&#8217;s claims, never accepted from the client. If the browser can send you a <code>matter_id</code>, the browser can send you a different one. Second, the filter is applied by code that no feature request can bypass. A helper that builds every retrieval request, and a code review rule that no other path may call the retrieve API directly.</p>



<p class="wp-block-paragraph">The trap here is ordering. You cannot filter on an attribute the index does not have. Add <code>matter_id</code> after ingestion and every chunk already in the index is invisible to that filter, which means it either returns for everyone or for no one, depending on how your filter is written. Design the metadata schema before the first ingestion run, not after the first demo.</p>



<h3 class="wp-block-heading">Separate collection per client</h3>



<p class="wp-block-paragraph">The heavier option. A separate OpenSearch Serverless collection per client, which buys you a separate AWS KMS key per client, separate index settings, and a failure mode where a bug in the filtering logic cannot reach across clients at all.</p>



<p class="wp-block-paragraph">It costs you sprawl. Every collection is a resource to provision, monitor, patch policy on, and eventually delete. Ingestion jobs run per data source per knowledge base, so freshness guarantees fragment. For a firm with twelve institutional clients this is fine. For a platform onboarding a hundred small clients it becomes the main operational burden of the product.</p>



<p class="wp-block-paragraph">My default is pooled with strict server-side filtering, and per-client collections only where the client&#8217;s own contract demands a dedicated encryption key. If you cannot say out loud which of those two you are running, you are running neither properly.</p>



<h2 class="wp-block-heading">Boundary two: extraction, and the confidence score everyone ignores</h2>



<p class="wp-block-paragraph">Textract returns a confidence score between 0 and 100 with each prediction. AWS is explicit in its own best-practice guidance that applications sensitive to detection errors should enforce a minimum threshold and route anything below it for human scrutiny, and that the right threshold depends entirely on how the output gets used.</p>



<p class="wp-block-paragraph">Legal documents sit at the harsh end of that. A misread date on an archival scan is a curiosity. A misread date on a limitation period is a problem with a name on it.</p>



<p class="wp-block-paragraph">Practical notes from this layer:</p>



<ul class="wp-block-list">
<li>Use the asynchronous operations for anything multipage. <code>StartDocumentAnalysis</code> and <code>GetDocumentAnalysis</code> handle long PDFs and TIFFs; the synchronous <code>AnalyzeDocument</code> path is for single pages and will fight you on a 400-page bundle.</li>



<li>Textract&#8217;s Queries feature earns its place on structured instruments. Instead of extracting everything and grepping, you ask the document a direct question and get a scoped answer with its own confidence.</li>



<li>Keep the page number and bounding box for every chunk. When a lawyer asks where a sentence came from, &#8220;page 14, second column&#8221; is an answer. &#8220;It&#8217;s in the corpus somewhere&#8221; is not, and the platform loses credibility the first time you say it.</li>



<li>Amazon Augmented AI wires low-confidence predictions into a human review queue with a private workforce, so the reviewers are your people rather than an anonymous pool. For privileged material that distinction is the entire point.</li>
</ul>



<p class="wp-block-paragraph">What I would skip early: building a custom entity recognition model. Comprehend supports custom entity recognition and it is genuinely useful, but you need labelled data you do not have yet, and a foundation model with a decent prompt gets you far enough to find out which entities the users actually care about. Train the custom model once the requirement has stopped moving.</p>



<h2 class="wp-block-heading">Boundary three: retention, where deletion is a distributed problem</h2>



<p class="wp-block-paragraph">Here is the failure that will not show up in any test suite you write.</p>



<p class="wp-block-paragraph">A client asks for a document to be removed. Someone deletes the S3 object. The ticket closes. The document is still fully searchable, because its chunks are sitting in the vector index and will stay there until the data source is re-synced. Meanwhile the Textract JSON output is in a second bucket, the extracted text may be in a database, and if model invocation logging is on, whole passages of the document are sitting in CloudWatch Logs or another S3 bucket entirely.</p>



<p class="wp-block-paragraph">One delete, at least four places holding a copy. Write the deletion path as a real workflow with an assertion at the end, not as a single API call:</p>



<ol class="wp-block-list">
<li>Delete the source object and, if versioning is on, its versions.</li>



<li>Delete the derived artefacts: Textract JSON, normalised text, any thumbnails or page images.</li>



<li>Trigger an ingestion job so the knowledge base drops the orphaned chunks.</li>



<li>Confirm removal by querying the index for the document identifier and expecting nothing back.</li>



<li>Deal with the logs, which means either a retention policy short enough to make the problem expire or a deliberate decision, written down, that logs are out of scope.</li>
</ol>



<p class="wp-block-paragraph">Step four is the one people leave out, and it is the only step that actually proves anything.</p>



<h3 class="wp-block-heading">The opposite problem: things that must not be deleted</h3>



<p class="wp-block-paragraph">Legal work has the reverse requirement too. When a matter goes into litigation hold, the documents need to survive an administrator with delete permissions and a bad afternoon.</p>



<p class="wp-block-paragraph">S3 Object Lock is the mechanism, and it has two independent controls that people routinely conflate. A retention period protects an object version until a fixed date, in either governance mode, which privileged users can override, or compliance mode, which nobody can override and where the period cannot be shortened. A legal hold has no date at all. It stays until someone with <code>s3:PutObjectLegalHold</code> explicitly removes it. The two can be active at once, and while either is active the object version cannot be deleted or overwritten.</p>



<pre class="wp-block-code"><code># Place an indefinite hold on a specific object version
aws s3api put-object-legal-hold 
  --bucket matters-archive 
  --key M-4417/exhibit-c.pdf 
  --version-id 3sL7f2Qz9pXvB1kR 
  --legal-hold Status=ON</code></pre>



<p class="wp-block-paragraph">Two things bite here. Object Lock requires versioning and turns it on automatically, and once enabled on a bucket you cannot turn it off or suspend versioning again. And compliance mode is genuinely permanent: if you set a seven-year retention when you meant seven days, that is the answer, for everyone, including the account root. Test in governance mode first. The bypass path exists and is deliberately awkward, requiring both the <code>s3:BypassGovernanceRetention</code> permission and an explicit <code>x-amz-bypass-governance-retention:true</code> header on the request, which is exactly the level of friction you want on that operation.</p>



<p class="wp-block-paragraph">Design the hold model before you design the deletion model, because holds win. A deletion request that collides with an active hold is a legal question, not an engineering one, and the platform&#8217;s job is to surface the collision clearly rather than resolve it silently.</p>



<h2 class="wp-block-heading">Boundary four: disclosure, or what leaves the account</h2>



<p class="wp-block-paragraph">This is the boundary the client&#8217;s general counsel will ask about, usually in writing, usually before signature. Three separate controls, and they are not interchangeable.</p>



<h3 class="wp-block-heading">The AI services opt-out policy</h3>



<p class="wp-block-paragraph">AWS Organizations has a governance policy type that opts your accounts out of having content processed by certain AI services stored and used for service improvement. The list includes Amazon Textract and Amazon Comprehend, which are precisely the two doing the reading in this architecture. The policy type has to be enabled at the organisation root before you can attach anything:</p>



<pre class="wp-block-code"><code>aws organizations enable-policy-type 
  --root-id r-example 
  --policy-type AISERVICES_OPT_OUT_POLICY</code></pre>



<p class="wp-block-paragraph">Read the AWS documentation on this one carefully rather than taking my summary as gospel, because there is a caveat in it that matters: the services may still need to store your data operationally even when you have opted out of it being used for improvement. Opting out is not the same as the data never existing outside your account. Say that plainly in the client conversation. It is a much better position than being asked about it later.</p>



<h3 class="wp-block-heading">Bedrock&#8217;s data position, and the log that undoes it</h3>



<p class="wp-block-paragraph">Amazon Bedrock&#8217;s published position is that inputs and outputs are not shared with third-party model providers and are not used to train the base models, and that fine-tuning operates on a private copy. That is a strong starting point for privileged content and it is the main reason Bedrock rather than a direct provider API shows up in these builds.</p>



<p class="wp-block-paragraph">Then there is model invocation logging. It captures full prompt and response payloads to CloudWatch Logs or S3, and you want it on, because without it you cannot reconstruct what the system told someone. But understand what you have just built: a log that contains privileged document text, at a per-region setting, in a destination that probably has looser access controls than the document store it came from. Encrypt it with the same KMS key discipline, restrict it harder than you think you need to, and set a retention period on purpose.</p>



<p class="wp-block-paragraph">Worth separating in your head: CloudTrail records that an API call happened and who made it. Model invocation logging records what was in it. You need both, for different questions, and only one of them contains client confidences.</p>



<h3 class="wp-block-heading">Network path</h3>



<p class="wp-block-paragraph">Interface VPC endpoints via AWS PrivateLink keep traffic to Bedrock, Textract and the rest on the AWS network rather than out through an internet gateway or NAT. Add a gateway endpoint for S3 while you are there.</p>



<p class="wp-block-paragraph">Endpoint policies are the part that gets forgotten. An endpoint without a policy is a private path to the whole service, including into accounts that are not yours. Scope it to the operations and resources you actually use, and pair it with an S3 bucket policy that rejects requests not arriving through your endpoint. The first protects what leaves; the second protects what can be reached.</p>



<h3 class="wp-block-heading">The parts that are not on the architecture diagram</h3>



<p class="wp-block-paragraph">Two of these have caught people out badly, and neither is an AWS control.</p>



<p class="wp-block-paragraph">The first is the development environment. You do not want real client documents on a laptop or in a scratch account, so build the parsing and chunking code against synthetic documents on a small separate box. A cheap VPS from a provider like Contabo or InterServer is fine for this, and the separation is worth more than the convenience of iterating in the production account.</p>



<p class="wp-block-paragraph">The second is the reviewers. If your human review loop involves contractors working remotely, their network path and their disks are part of your boundary whether or not you drew them. A managed VPN such as NordVPN or Surfshark handles the network side, and when a matter closes and local copies have to go, a dedicated erasure tool from something like O&amp;O Software does what dragging a folder to the bin does not. Unglamorous, and it is the layer that gets audited.</p>



<h2 class="wp-block-heading">Boundary five: evidence, because someone will ask</h2>



<p class="wp-block-paragraph">At some point the question stops being &#8220;does it work&#8221; and becomes &#8220;who saw what, and when&#8221;. If you cannot answer that from stored records, the platform is not defensible regardless of how good the answers are.</p>



<ul class="wp-block-list">
<li>Turn on CloudTrail data events for the document buckets. They are off by default, they are billed separately from management events, and they are the only way to see individual object-level reads.</li>



<li>Log the retrieval, not just the generation. Store which chunks came back and which filter was applied. When somebody asks whether the wall held on a specific query, this record is the answer and there is no reconstructing it later.</li>



<li>Carry the end user&#8217;s identity through to the audit record. A Lambda execution role in the logs tells you the platform did something. It does not tell you who asked.</li>



<li>Alarm on the filter, not just on errors. A retrieval that ran without a matter filter should page someone, even though it returned HTTP 200 and a perfectly good answer.</li>
</ul>



<p class="wp-block-paragraph">That last one is the closest thing to a single takeaway in this post. The dangerous events in this architecture are successful ones.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting the things that will go wrong</h2>



<h3 class="wp-block-heading">Retrieval returns nothing after you add a filter</h3>



<p class="wp-block-paragraph">Nearly always a metadata problem rather than a query problem. Either the sidecar metadata file was missing at ingestion, so the chunks carry no attribute to match, or the attribute was added after the chunks were indexed. Check whether the attribute exists on a known chunk before you touch the query. If it does not, you need a re-sync, not a different filter expression.</p>



<h3 class="wp-block-heading">Textract returns text but the layout is scrambled</h3>



<p class="wp-block-paragraph">Common on two-column pleadings and on documents with headers and footers on every page. Raw text detection reads in a reading order that is not always yours. Use the layout and table features rather than plain detection, and hold onto the geometry so you can reassemble columns yourself if the default order is wrong for that document class.</p>



<h3 class="wp-block-heading">The model cites a document that does not exist</h3>



<p class="wp-block-paragraph">Usually the citation is being generated rather than passed through. If document names are being written by the model instead of read from the retrieval response, it will invent plausible ones. Build citations from the retrieval result metadata in your application code and never from the generated text. Bedrock Guardrails helps with contextual grounding, but the structural fix is not asking the model to produce the reference in the first place.</p>



<h3 class="wp-block-heading">Ingestion is slow and nobody knows where</h3>



<p class="wp-block-paragraph">Instrument per stage before you optimise anything. In most of these pipelines the time is in Textract for large scanned bundles, and the fix is parallelism at the document level rather than tuning the extraction itself. Step Functions with a distributed map over documents gets you there without a queue you have to babysit.</p>



<h3 class="wp-block-heading">Costs climb faster than volume</h3>



<p class="wp-block-paragraph">Look at re-processing first. A pipeline that re-extracts a document every time anything downstream changes will quietly multiply your Textract bill against a static corpus. Key the extraction cache on the object&#8217;s content hash, not its path, and make re-ingestion an explicit action. The billing mechanisms differ per service, so read the current pricing pages rather than trusting a number from a blog post.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li><strong>Treating the system prompt as an access control.</strong> It is a formatting instruction that happens to look like a rule.</li>



<li><strong>Taking the tenant identifier from the request body.</strong> If the client can name the matter, the client can name someone else&#8217;s.</li>



<li><strong>Designing the metadata schema after the first ingestion run.</strong> Retrofitting a filterable attribute means re-indexing the whole corpus.</li>



<li><strong>Assuming a deleted S3 object is gone from the platform.</strong> The chunks, the extraction output and the invocation logs all outlive it.</li>



<li><strong>Setting compliance-mode retention without testing in governance mode.</strong> There is no support ticket that fixes a seven-year mistake.</li>



<li><strong>Enabling model invocation logging without treating the log as privileged.</strong> You have just made a second copy of the documents in a less protected place.</li>



<li><strong>Building VPC endpoints and leaving the default policy on them.</strong> A private path with no policy is still a path.</li>
</ul>



<h2 class="wp-block-heading">Best practices worth the effort</h2>



<ul class="wp-block-list">
<li>One retrieval helper, server-side filter injection, and a review rule that no other code path calls the retrieve API directly.</li>



<li>Metadata schema agreed and frozen before the first production ingestion. Include the fields you might filter on later, even if unused today.</li>



<li>Page-level provenance on every chunk, surfaced in the interface. It builds trust faster than any accuracy improvement.</li>



<li>A confidence threshold chosen per document class, with a human review path for everything under it.</li>



<li>Deletion implemented as a verified workflow across every store, ending in an assertion that the content is actually unretrievable.</li>



<li>Object Lock legal holds for matters under litigation hold, tested in governance mode before anything runs in compliance mode.</li>



<li>Customer-managed KMS keys across S3, the vector store and the logs, so key access is a control you can actually revoke.</li>



<li>An alarm on unfiltered retrievals, because the failure mode is a success response.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Is a legal document intelligence platform on AWS safe for privileged material?</h3>



<p class="wp-block-paragraph">The services support it. Bedrock&#8217;s stated position is that inputs and outputs are not shared with model providers or used to train base models, PrivateLink keeps traffic off the public internet, and KMS covers encryption at rest. What determines safety is your own configuration: tenant isolation, log handling and deletion behaviour. The platform is exactly as confidential as its weakest copy of the text.</p>



<h3 class="wp-block-heading">Should I use Bedrock Knowledge Bases or build retrieval myself?</h3>



<p class="wp-block-paragraph">Start with Knowledge Bases. It handles chunking, embedding, sync and metadata filtering, and those are weeks of work with no differentiation in them. Build your own when you need retrieval behaviour it does not expose, such as unusual re-ranking or per-tenant chunking strategies. Going custom on day one usually means reimplementing the managed service badly.</p>



<h3 class="wp-block-heading">How do I stop one client&#8217;s documents surfacing in another client&#8217;s answers?</h3>



<p class="wp-block-paragraph">Tag every chunk with a tenant identifier at ingestion, and inject the matching filter server-side on every retrieval from the authenticated caller&#8217;s identity. Never accept the identifier from the client. If the contract requires per-client encryption keys, use a separate collection per client instead, accepting the extra operational load.</p>



<h3 class="wp-block-heading">Does deleting a document from S3 remove it from the search index?</h3>



<p class="wp-block-paragraph">No. The chunks stay in the vector store until an ingestion job runs and reconciles the data source. Until then the document is deleted and still fully searchable, which is the worst combination available. Always finish a deletion by querying the index and confirming nothing comes back.</p>



<h3 class="wp-block-heading">What is the difference between an S3 legal hold and a retention period?</h3>



<p class="wp-block-paragraph">A retention period runs until a fixed date and comes in governance mode, which privileged users can override, or compliance mode, which nobody can. A legal hold has no end date and stays until someone explicitly removes it. They are independent, they can both be active on the same object version, and while either is active the object cannot be deleted or overwritten.</p>



<h3 class="wp-block-heading">Do I need human review, or is the model accurate enough?</h3>



<p class="wp-block-paragraph">Accuracy is the wrong frame. The question is what happens to a wrong answer downstream. Where output feeds a decision with consequences, you want a confidence threshold and a review queue, and Amazon Augmented AI with a private workforce gives you that without building the review tooling yourself. Where output is a search aid a person will verify anyway, review is friction you do not need.</p>



<h3 class="wp-block-heading">Can I run this in a single AWS account?</h3>



<p class="wp-block-paragraph">Technically yes, and for a pilot it is reasonable. It gets uncomfortable once you have production documents and a development environment in the same place, because an IAM mistake has nowhere to stop. Separate accounts under Organizations also gives you the policy layer, including the AI services opt-out policy, which only exists at the organisation level.</p>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">A legal document intelligence platform on AWS is not hard to build. Textract reads, Bedrock reasons, OpenSearch retrieves, and the tutorial version works on the first afternoon.</p>



<p class="wp-block-paragraph">What is hard is that its failures are silent and well-formed. The cross-matter citation, the deleted document that still answers questions, the privileged passage sitting in a log bucket, the retrieval that ran without a filter and returned a beautiful paragraph. None of these throw an error. All of them are the kind of thing that ends a client relationship.</p>



<p class="wp-block-paragraph">So build the boundaries first and the features second, and make sure every one of them is something you can prove from a stored record rather than something you believe about the code. If you can only take one habit from this: alarm on the successful requests that should not have been possible.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a second pair of eyes on your document pipeline?</h2>



<p class="wp-block-paragraph">I work on AWS document processing and retrieval systems where the confidentiality requirements are real. Typical things I get called in for:</p>



<ul class="wp-block-list">
<li>Reviewing tenant isolation in an existing RAG setup and finding the paths where the filter can be bypassed.</li>



<li>Designing the metadata and chunking schema before ingestion, so filtering works without a re-index later.</li>



<li>Building Textract pipelines with confidence thresholds, human review routing and page-level provenance.</li>



<li>Implementing verified deletion across S3, derived artefacts, the vector index and logs, with a proof step at the end.</li>



<li>Setting up retention and legal hold with S3 Object Lock, including the governance-mode rehearsal before compliance mode.</li>



<li>Locking down the network and logging path: VPC endpoints with real policies, KMS key separation, and audit records that name the actual user.</li>
</ul>



<p class="wp-block-paragraph">If you have an architecture diagram, a Step Functions definition or a retrieval request you are unsure about, send it over and I will tell you what I would change and why.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/legal-document-intelligence-aws/">Legal Document Intelligence on AWS: The Five Boundaries That Have to Hold</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Amazon Textract Data Extraction: What Breaks on Real Contracts and Reports</title>
		<link>https://john-nessime.com/blog/devops/amazon-textract-data-extraction/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 20 Aug 2026 18:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon SNS]]></category>
		<category><![CDATA[Amazon SQS]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Lambda]]></category>
		<category><![CDATA[Batch Processing]]></category>
		<category><![CDATA[Boto3]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[Data Quality]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Idempotency]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[Pipeline Design]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Serverless]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=270</guid>

					<description><![CDATA[<p>Textract rarely fails loudly. It returns a plausible result that is quietly incomplete: a truncated result set, a tick box read as an empty string, a clause split across a page break. A practitioner's guide to the failure modes that actually bite when you point Amazon Textract at contracts, technical reports and correspondence, plus how to choose between sync and async, which feature types are worth paying for, and where Textract stops being the right tool.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/amazon-textract-data-extraction/">Amazon Textract Data Extraction: What Breaks on Real Contracts and Reports</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The pipeline passed every test. Three sample agreements, all the right fields, clean JSON out the other side. Then it ran against the real archive and somebody in legal noticed that every contract longer than about forty pages was missing its termination clause. No errors. No failed jobs. The dashboard was green the whole time.</p>



<p class="wp-block-paragraph">The bug was a missing loop. The job had finished, the results were there, and the code had read the first chunk of them and stopped.</p>



<p class="wp-block-paragraph">That is the shape of most problems with Amazon Textract data extraction on real documents. The service rarely fails loudly. It returns a plausible-looking result that is quietly incomplete, and you find out weeks later when somebody asks a question the data cannot answer. This post covers the failures that actually bite on contracts, technical reports and correspondence: sync versus async, reading the block graph without losing fields, which feature types are worth paying for, and where Textract stops being the right tool.</p>



<h2 class="wp-block-heading">Why Amazon Textract data extraction is not just OCR</h2>



<p class="wp-block-paragraph">Plain OCR gives you words and their positions. Textract gives you a graph. Every response is a flat array of <code>Block</code> objects, and each block carries an ID plus relationships to other block IDs. A page relates to its lines, a line to its words, a key to its value, a table to its cells.</p>



<p class="wp-block-paragraph">That structure is the whole point, and it is where the pain lives. Nothing is nested for you. To read one form field you find the <code>KEY_VALUE_SET</code> block with entity type <code>KEY</code>, follow its <code>VALUE</code> relationship to another block, then follow that block&#8217;s <code>CHILD</code> relationships to the words. Three hops, and getting a hop wrong returns an empty string rather than an exception.</p>



<p class="wp-block-paragraph">Contracts and reports make this harder than invoices do. An invoice has a total. A master services agreement has a liability cap buried in a numbered sub-clause that spans a page break.</p>



<h2 class="wp-block-heading">Synchronous or asynchronous: the choice is made for you</h2>



<p class="wp-block-paragraph">People reach for <code>AnalyzeDocument</code> first because it returns results in the same call and is easy to test in a notebook. Then they hit the quotas, which AWS documents as hard limits you cannot raise. Synchronous operations cap JPEG, PNG, PDF and TIFF at 10 MB in memory, and cap PDF and TIFF at <strong>one page</strong>. Asynchronous operations keep JPEG and PNG at 10 MB but take PDF and TIFF up to 500 MB and 3,000 pages.</p>



<p class="wp-block-paragraph">One page. That single constraint decides your architecture. Any real contract goes through <code>StartDocumentAnalysis</code> and <code>GetDocumentAnalysis</code>, which means it goes through S3, which means you need somewhere to stage documents and a way to learn when the job is done.</p>



<p class="wp-block-paragraph">Three other constraints from the same quota page are worth knowing before you promise anything to a client:</p>



<ul class="wp-block-list">
<li>Text detection covers English, French, German, Italian, Portuguese and Spanish. Query detection is English only.</li>

<li>Vertical text is not supported. Rotation is fine, including odd in-plane angles, but vertically written scripts are not.</li>

<li>Password-protected PDFs are rejected and XFA-based PDFs are unsupported. Both turn up in legal archives more often than you would expect.</li>
</ul>



<p class="wp-block-paragraph">Pass a <code>ClientRequestToken</code> on submission. It is an idempotency token: the same token returns the same <code>JobId</code> instead of starting a duplicate job. If your submitter is a Lambda function behind an S3 event, and S3 events can be delivered more than once, that is the difference between paying once and paying twice.</p>



<pre class="wp-block-code"><code>import boto3

textract = boto3.client("textract")

job = textract.start_document_analysis(
    DocumentLocation={
        "S3Object": {"Bucket": "contracts-intake", "Name": "msa/acme-2.pdf"}
    },
    FeatureTypes=["FORMS", "TABLES"],
    ClientRequestToken="msa-acme-2-v1",
    NotificationChannel={
        "SNSTopicArn": "arn:aws:sns:eu-west-1:111122223333:textract-done",
        "RoleArn": "arn:aws:iam::111122223333:role/TextractSnsPublish",
    },
)</code></pre>



<p class="wp-block-paragraph">The notification channel is optional but you want it. Polling in a loop inside a Lambda function burns billed duration doing nothing, and Lambda&#8217;s execution ceiling will cut you off on long documents anyway. Publish to SNS, fan out to SQS, let a second function do the reading, and you get a natural place to hang a dead letter queue.</p>



<h2 class="wp-block-heading">The result set you never finished reading</h2>



<p class="wp-block-paragraph">This is the one that cost the termination clauses. <code>GetDocumentAnalysis</code> returns results in pages. When there are more blocks than fit in one response, the response carries a <code>NextToken</code> and you have to call again with it. If you do not, you get the beginning of the document and nothing tells you so. <code>JobStatus</code> still reports success, because the job did succeed. Your code just stopped reading.</p>



<p class="wp-block-paragraph">Short test documents fit in a single response. That is exactly why this survives testing and dies in production.</p>



<pre class="wp-block-code"><code>def fetch_all_blocks(textract, job_id):
    blocks, next_token = [], None

    while True:
        kwargs = {"JobId": job_id}
        if next_token:
            kwargs["NextToken"] = next_token

        response = textract.get_document_analysis(**kwargs)

        if response["JobStatus"] == "FAILED":
            raise RuntimeError(response.get("StatusMessage", "job failed"))

        blocks.extend(response["Blocks"])
        next_token = response.get("NextToken")

        if not next_token:
            return blocks</code></pre>



<p class="wp-block-paragraph">Then make the failure detectable. Treat the highest page number present in the blocks as a checksum against the page count of the source file. Submit a 60-page PDF, get blocks that stop at page 12, and something is wrong no matter what the job status says.</p>



<h2 class="wp-block-heading">Choosing feature types without paying for all of them</h2>



<p class="wp-block-paragraph"><code>AnalyzeDocument</code> and <code>StartDocumentAnalysis</code> take a <code>FeatureTypes</code> list: <code>FORMS</code>, <code>TABLES</code>, <code>QUERIES</code>, <code>SIGNATURES</code> and <code>LAYOUT</code>. This is not cosmetic. Textract bills per page and the per-page rate depends on which features you enabled, so turning all five on because you might need them later multiplies your bill across the whole archive. Rates change and vary by region, so check the current pricing page rather than trusting a number in a blog post. The mechanism does not change: each feature adds to the per-page cost, and you pay it on every page you submit, including blank separator sheets.</p>



<ul class="wp-block-list">
<li><strong>FORMS</strong> finds key-value pairs where the document has an explicit label. Good for cover sheets, signature blocks, report headers. Useless for prose clauses with no label.</li>

<li><strong>TABLES</strong> reconstructs rows, columns and cells, and also identifies table titles, footers and merged cells. This is what you want for rate cards and appendix tables.</li>

<li><strong>QUERIES</strong> lets you ask natural-language questions and get answers back under an alias you chose. This is the feature that makes contracts tractable.</li>

<li><strong>SIGNATURES</strong> returns the location and confidence of handwritten signatures, electronic signatures and initials. It tells you something is signed at a coordinate, not whose signature it is.</li>

<li><strong>LAYOUT</strong> groups text into titles, headers, footers, section headers, paragraphs and lists in reading order. Skippable on single-column documents. On two-column reports it is the difference between coherent text and interleaved nonsense.</li>
</ul>



<p class="wp-block-paragraph">A practical pattern: run a cheap text-detection pass to classify each document, then apply the expensive feature set only to the pages that matter. A 200-page appendix of scanned site photographs does not need FORMS and TABLES.</p>



<h2 class="wp-block-heading">Reading the block graph without losing fields</h2>



<p class="wp-block-paragraph">Here is the failure that produces silent empty strings. A key-value pair&#8217;s value is not always text. If the value is a tick box, the value block&#8217;s children include a <code>SELECTION_ELEMENT</code> block, and that block has no text at all. It has a <code>SelectionStatus</code> of <code>SELECTED</code> or <code>NOT_SELECTED</code>. Concatenate only <code>WORD</code> children and a ticked box comes back as an empty string, which looks exactly like a field nobody filled in.</p>



<pre class="wp-block-code"><code>def block_text(block, block_map):
    parts = []
    for rel in block.get("Relationships", []):
        if rel["Type"] != "CHILD":
            continue
        for child_id in rel["Ids"]:
            child = block_map[child_id]
            if child["BlockType"] == "WORD":
                parts.append(child["Text"])
            elif child["BlockType"] == "SELECTION_ELEMENT":
                parts.append(child["SelectionStatus"])
    return " ".join(parts)</code></pre>



<p class="wp-block-paragraph">Build <code>block_map</code> once as a dictionary keyed on block ID before you traverse anything. Scanning the block list linearly for each lookup turns a three-hop traversal into an accidental quadratic, which you will notice the first time a 3,000-page bundle goes through.</p>



<h2 class="wp-block-heading">Queries: asking for clauses that have no label</h2>



<p class="wp-block-paragraph">FORMS works when the document says &#8220;Effective Date:&#8221; next to the date. Contracts frequently do not. The governing law sits in a paragraph of prose halfway down page nine. Queries handles that. You pass questions in plain English, each with an <code>Alias</code> you choose, and the response pairs <code>QUERY</code> blocks with <code>QUERY_RESULT</code> blocks through an <code>ANSWER</code> relationship. The alias is what makes this usable downstream: you match on <code>governing_law</code>, not on the exact wording of the question, so you can reword questions without breaking your schema.</p>



<pre class="wp-block-code"><code>QueriesConfig = {
    "Queries": [
        {"Text": "What is the governing law?",
         "Alias": "governing_law", "Pages": ["*"]},
        {"Text": "What is the termination notice period?",
         "Alias": "termination_notice", "Pages": ["9-*"]},
    ]
}</code></pre>



<p class="wp-block-paragraph">The limits matter. AWS documents a maximum of 15 queries per page for synchronous operations and 30 per page for asynchronous ones. A fifty-field schema cannot be asked in one pass. You either split it across multiple calls, which multiplies per-page spend, or you narrow <code>Pages</code> so each query runs only where the answer plausibly lives. Signature blocks are at the end, definitions near the front. Running every query against every page is the most common way people accidentally triple their bill.</p>



<p class="wp-block-paragraph">Queries are pre-trained on a spread of business documents including paystubs, bank statements, loan applications and mortgage notes. Your niche contract template was not in that set. Expect good results on common concepts and mediocre ones on house-specific terminology.</p>



<h2 class="wp-block-heading">Documents that fight back</h2>



<h3 class="wp-block-heading">Clauses that span a page break</h3>



<p class="wp-block-paragraph">Textract analyses pages. A clause starting at the bottom of page 11 and finishing at the top of page 12 is two disconnected fragments as far as the block graph is concerned. Queries will often return the fragment on one page and miss the qualifier on the other, which is the worst outcome available because the answer looks complete. The fix is not clever: reassemble full text in reading order using LAYOUT, then run clause-level logic over that rather than over per-page answers. Use Queries to <em>locate</em> a clause and the reassembled text to <em>read</em> it.</p>



<h3 class="wp-block-heading">Two-column reports and email threads</h3>



<p class="wp-block-paragraph">Without LAYOUT, a two-column technical report reads as alternating lines from both columns. Everything downstream inherits that corruption and it passes a spot check, because individual lines look fine. LAYOUT sequences elements in reading order and returns block types for titles, headers, footers and section headers, which is also what you need to chunk a report by section.</p>



<p class="wp-block-paragraph">Printed email chains are quoted text inside quoted text, newest message at the top, the same signature block repeated five times. Textract extracts all of it faithfully, which is the problem. Splitting a thread into individual messages is a text-processing job you do afterwards, not something a feature type solves.</p>



<h3 class="wp-block-heading">PDFs that were never scanned</h3>



<p class="wp-block-paragraph">This is the cheapest win available and almost everybody misses it. A large share of contract archives are born-digital PDFs exported from a word processor, and they already contain a text layer. Running OCR over them pays a service to guess at text you could have read directly. Put a triage step in front of the pipeline; <code>pdftotext</code> from poppler-utils pulls the embedded layer if there is one:</p>



<pre class="wp-block-code"><code>pdftotext -layout contract.pdf - | head -c 2000</code></pre>



<p class="wp-block-paragraph">Substantial text back means born-digital: route it down a cheaper path and reserve Textract for genuinely scanned material. Triage across a large archive is an embarrassingly parallel batch job that sits better on a plain VPS from somewhere like Contabo or InterServer than on per-invocation serverless billing, since you are CPU-bound for minutes at a time rather than reacting to events. One caveat: born-digital does not mean clean. Some exporters produce a text layer with broken word spacing or mangled ligatures, so sample before you trust it.</p>



<h2 class="wp-block-heading">Confidence scores you can actually act on</h2>



<p class="wp-block-paragraph">Every block carries a confidence score and the instinct is to threshold on it. That is usually wrong. Confidence tells you how sure the model is about <em>which characters are on the page</em>, not whether the field is semantically right. Textract can read a date with total certainty and hand you the wrong one, because it picked up the printing date in the footer instead of the effective date in the recitals. High confidence, completely wrong answer.</p>



<p class="wp-block-paragraph">It is also per block. A key-value pair has a score on the key, another on the value and separate ones on each word, so there is no single number to gate on. Validate the extracted value instead. Does the date parse and fall in a plausible range? Does the total equal the sum of the line items? Does the party name appear in the signature block as well as the preamble? Route to human review on failed validation and treat low confidence as one input among several.</p>



<h2 class="wp-block-heading">When Custom Queries adapters are worth the effort</h2>



<p class="wp-block-paragraph">If pre-trained Queries keeps getting your house document type wrong, train an adapter. Create it with <code>CreateAdapter</code>, annotate sample documents, train a version with <code>CreateAdapterVersion</code>, then reference it at inference time.</p>



<pre class="wp-block-code"><code>AdaptersConfig = {
    "Adapters": [
        {"AdapterId": ADAPTER_ID, "Version": "1", "Pages": ["1-5"]},
        {"AdapterId": ADAPTER_ID, "Version": "1", "Pages": ["6-*"]},
    ]
}</code></pre>



<p class="wp-block-paragraph">That <code>Pages</code> string takes digits, hyphens and an asterisk with no blank spaces, a page can only have one adapter applied to it, and an asterisk meaning all pages must be the only element in the list.</p>



<p class="wp-block-paragraph">Two things to weigh. AWS sets a minimum of five samples per query level for training or testing and says plainly that more is better, so treat five as a demo floor rather than a number that survives layout variation. And successful trainings per month are capped per account, so the tight annotate-train-evaluate loop you want is not available. Adapters suit one high-volume, house-specific document type with a stable schema. They are a poor fit for a long tail of one-off templates, which is what most legal archives actually are.</p>



<h2 class="wp-block-heading">Where Textract stops and a language model starts</h2>



<p class="wp-block-paragraph">This is a real decision now, so both sides deserve a hearing. Textract&#8217;s case is strong on standardised, high-volume documents. It is deterministic in a way generative models are not, it returns bounding-box geometry so you can point at exactly where a value came from, it gives per-element confidence, and it bills predictably per page. When an auditor asks why a field has the value it does, geometry beats a model&#8217;s reasoning.</p>



<p class="wp-block-paragraph">The generative side, whether that is Amazon Bedrock Data Automation as a managed document processing service or a model called directly through Amazon Bedrock, wins where the task needs reasoning rather than pattern matching. &#8220;Does this agreement contain an assignment restriction, and if so summarise it&#8221; is a comprehension question, and Textract has no answer for it. Generative approaches also cope better with layouts that were in nobody&#8217;s training set.</p>



<p class="wp-block-paragraph">Most production pipelines use both, which is what AWS&#8217;s own reference architectures push: Textract for faithful extraction with geometry and confidence, then a model over the extracted text for classification and comprehension. Choosing from scratch, ask whether your documents are standardised and high-volume. If yes, Textract plus a thin post-processing layer is cheaper and more auditable. If they are varied and volume is modest, a managed generative service gets you working faster. Google Document AI and Azure AI Document Intelligence cover similar ground if you are not committed to AWS.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting</h2>



<ul class="wp-block-list">
<li><strong>Extraction stops partway through long documents.</strong> You are not following <code>NextToken</code>. Compare the highest page number in your blocks against the source page count.</li>

<li><strong>Fields come back empty on forms you know were filled in.</strong> The value is a <code>SELECTION_ELEMENT</code> and you are only reading <code>WORD</code> children.</li>

<li><strong>The document is rejected as too large.</strong> You are on the synchronous API. Multi-page PDFs must go through <code>StartDocumentAnalysis</code>, and the 10 MB cap is on in-memory size, not file size on disk.</li>

<li><strong>Textract cannot read the document at all.</strong> Check for password protection and XFA-based forms. Both are documented as unsupported.</li>

<li><strong>The service cannot access the S3 object.</strong> Usually the execution role missing <code>s3:GetObject</code>, or a bucket in a different region from the Textract endpoint you called.</li>

<li><strong>Throttling under batch load.</strong> Transactions-per-second quotas are per account per region. Smooth spiky traffic through a queue and retry with exponential backoff and jitter rather than raising concurrency.</li>

<li><strong>Garbled text on reports that look fine to you.</strong> Multi-column layout without LAYOUT enabled, or a source scan below roughly 150 DPI. Re-scanning beats any amount of post-processing.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Testing only on short documents, which hides the pagination bug entirely.</li>

<li>Enabling every feature type on every page because it is easier than deciding.</li>

<li>Running OCR over born-digital PDFs that already carry a text layer.</li>

<li>Treating a confidence score as a correctness score.</li>

<li>Scanning the block array linearly instead of building an ID map.</li>

<li>Polling for job completion inside a Lambda function instead of using SNS.</li>

<li>Storing only extracted values and discarding the raw JSON, so reprocessing means paying again.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Persist the full Textract JSON to S3 keyed on a hash of the source document. Reprocessing then costs storage, not extraction.</li>

<li>Classify first, extract second. A cheap text-detection pass tells you which expensive feature set each document needs.</li>

<li>Validate on the value, not the score. Parse dates, check ranges, cross-reference names against other parts of the document.</li>

<li>Use aliases on every query and treat them as your schema contract.</li>

<li>Set a per-job page-count assertion and alert on mismatches. That is your canary for silent truncation.</li>

<li>Keep source documents in an encrypted bucket with a lifecycle policy. Contracts and correspondence should not accumulate indefinitely by accident.</li>

<li>Track cost per document rather than per API call. Feature stacking makes the per-call figure meaningless.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Can Amazon Textract handle multi-page PDF contracts?</h3>



<p class="wp-block-paragraph">Yes, through the asynchronous API only. Synchronous operations cap PDF and TIFF at one page. Asynchronous operations accept PDF and TIFF up to 500 MB and 3,000 pages, staged through S3.</p>



<h3 class="wp-block-heading">Why is my Textract output missing the end of the document?</h3>



<p class="wp-block-paragraph">Almost certainly because you read the first response from <code>GetDocumentAnalysis</code> and stopped. Results are paginated with a <code>NextToken</code> and the job status still reports success, so nothing signals the truncation. Loop until <code>NextToken</code> is absent.</p>



<h3 class="wp-block-heading">How many Textract Queries can I ask per page?</h3>



<p class="wp-block-paragraph">AWS documents 15 per page for synchronous operations and 30 per page for asynchronous ones. Larger schemas need multiple passes, which costs more, so narrowing each query&#8217;s page range is worth the effort.</p>



<h3 class="wp-block-heading">Does Textract work on handwriting and signatures?</h3>



<p class="wp-block-paragraph">It detects handwriting, and the SIGNATURES feature returns the location and confidence of handwritten signatures, electronic signatures and initials. It does not verify identity: it tells you something was signed and where, not by whom.</p>



<h3 class="wp-block-heading">What languages does Amazon Textract support?</h3>



<p class="wp-block-paragraph">Text detection covers English, French, German, Italian, Portuguese and Spanish. Queries detection is English only, vertically written text is unsupported, and Textract does not return the detected language in its output.</p>



<h3 class="wp-block-heading">Should I use Textract or a generative model?</h3>



<p class="wp-block-paragraph">Textract when documents are standardised, volume is high, and you need auditable geometry and per-field confidence. A generative service when layouts vary widely, volume is modest, or the task needs comprehension rather than extraction. Most production pipelines use both.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">Amazon Textract data extraction fails quietly far more often than it fails loudly. A truncated result set, a tick box read as an empty string, a clause split across a page break, a confidently extracted wrong date: none of these throw an exception and none show up on a dashboard.</p>



<p class="wp-block-paragraph">So build the assertions that make silence detectable. Compare expected page counts against extracted ones, validate values rather than trusting scores, and keep the raw JSON so you can prove what the service actually returned. Textract is genuinely good at reading documents. Your job is noticing when it did not read all of them.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need a document extraction pipeline that does not lose fields?</h2>



<p class="wp-block-paragraph">I build and fix document processing pipelines on AWS. Typical work looks like this:</p>



<ul class="wp-block-list">
<li>Auditing an existing Textract pipeline for silent truncation, dropped selection elements and mis-scoped queries</li>

<li>Designing the async architecture end to end: S3 intake, SNS and SQS fan-out, Lambda or container workers, dead letter handling and retries</li>

<li>Cutting per-page cost by classifying first, triaging born-digital PDFs out of the OCR path and narrowing feature types per page range</li>

<li>Turning block-graph JSON into a clean schema in CSV, a relational database or a data lake, with validation rules and a review queue for exceptions</li>

<li>Building the hybrid path where Textract handles extraction and a model on Amazon Bedrock handles classification and comprehension</li>

<li>Adding the observability that makes quiet failures loud: page-count assertions, per-document cost tracking, confidence distribution monitoring</li>
</ul>



<p class="wp-block-paragraph">If you have a redacted sample document, a chunk of Textract JSON or an extraction coming back half empty, send it over and I will tell you what is going wrong with it.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/amazon-textract-data-extraction/">Amazon Textract Data Extraction: What Breaks on Real Contracts and Reports</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Building an Insurance Claims Processing Pipeline on AWS That Fails Loudly</title>
		<link>https://john-nessime.com/blog/cloud-computing/insurance-claims-processing-pipeline-aws/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 20 Aug 2026 09:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Insurance Technology]]></category>
		<category><![CDATA[Workflow Automation]]></category>
		<category><![CDATA[Amazon SNS]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[Bedrock Data Automation]]></category>
		<category><![CDATA[Claims Automation]]></category>
		<category><![CDATA[Confidence Scoring]]></category>
		<category><![CDATA[Data Validation]]></category>
		<category><![CDATA[Dead Letter Queue]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Event-Driven Architecture]]></category>
		<category><![CDATA[HIPAA]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Idempotency]]></category>
		<category><![CDATA[Insurance Claims]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Serverless]]></category>
		<category><![CDATA[Step Functions]]></category>
		<category><![CDATA[Straight-Through Processing]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=464</guid>

					<description><![CDATA[<p>Claims pipelines rarely crash. They succeed, emit clean JSON, and hand a wrong number to a payment system. Six failure families in an insurance claims processing pipeline on AWS, with the Textract, Bedrock Data Automation and Step Functions details that decide whether a bad extraction is visible or silent.</p>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/insurance-claims-processing-pipeline-aws/">Building an Insurance Claims Processing Pipeline on AWS That Fails Loudly</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The worst ticket on a claims pipeline is never the one that says the pipeline is down. A stuck queue is loud. It pages somebody, somebody restarts something, and it gets fixed before lunch. The bad ticket arrives three weeks later from finance: a run of claims was auto-approved at amounts nobody can reconcile, and every single execution in the Step Functions console is green.</p>



<p class="wp-block-paragraph">That is the failure mode that defines this problem. An insurance claims processing pipeline on AWS almost never falls over in the way you designed it to fall over. It succeeds. It emits well-formed JSON. It hands a number to a payment system, and the number is wrong, and nothing in the pipeline had any reason to think otherwise.</p>



<p class="wp-block-paragraph">This post is organized by failure family rather than by service. I&#8217;ll walk through the six ways these pipelines go quietly wrong, what each one costs, and what the fix actually looks like in Textract, Bedrock, Step Functions and S3. There&#8217;s a troubleshooting section, the mistakes I see repeated, and an FAQ at the end.</p>



<h2 class="wp-block-heading">The shape most claims pipelines end up with</h2>



<p class="wp-block-paragraph">Before the failure families make sense, the skeleton. Almost every serverless claims pipeline lands on roughly the same set of stages, whatever the vendor deck calls them:</p>



<ol class="wp-block-list">
<li><strong>Intake.</strong> A document lands in S3 from a portal upload, an SFTP drop, or a mail scanning vendor. An S3 event or EventBridge rule starts an execution.</li>

<li><strong>Classification.</strong> Work out what the packet actually contains. A first notice of loss, a CMS-style claim form, a police report, an itemized bill, forty pages of photographs.</li>

<li><strong>Extraction.</strong> Pull the fields you need. Amazon Textract for OCR, forms and tables, or Amazon Bedrock Data Automation with a blueprint that names the fields directly.</li>

<li><strong>Validation.</strong> Check the extracted values against business rules, policy data, and each other.</li>

<li><strong>Routing.</strong> Straight-through processing, human review, or rejection with a reason.</li>

<li><strong>Persistence and audit.</strong> The claim record, the extraction artifacts, and enough evidence to explain a decision months later.</li>
</ol>



<p class="wp-block-paragraph">Nothing controversial there. AWS publishes an open-source GenAI IDP Accelerator that implements exactly this shape, with a Bedrock Data Automation mode and a Textract-plus-foundation-model pipeline mode, and it&#8217;s a reasonable place to start reading. The interesting part is not the boxes. It&#8217;s what happens between them.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Failure family one: the field that was never there</h2>



<p class="wp-block-paragraph">This is the one that pays out the wrong number, and it is worth more attention than everything else in this post combined.</p>



<p class="wp-block-paragraph">Every extraction service gives you confidence scores. So the obvious design is a gate: if every field scores above some threshold, approve automatically; if anything falls below, send it to a human. That gate is sound reasoning applied to the wrong population.</p>



<p class="wp-block-paragraph">A confidence score only exists for a value that came back. When the extractor doesn&#8217;t find a field at all, there is no low score to catch, because there&#8217;s nothing to score. The gate iterates over four returned fields, finds all four above threshold, and reports a clean pass. The fifth field, the one that determines coordination of benefits or the deductible offset, is simply absent from the response. Downstream code treats absent as zero, or as null, or as &#8220;not applicable,&#8221; and the claim goes through.</p>



<p class="wp-block-paragraph">The fix is structural, not statistical. Validate <em>presence against a schema</em> before you validate confidence, and treat the two as separate gates with separate outcomes:</p>



<pre class="wp-block-code"><code># Two gates, not one. Presence first, then confidence.
# 'extracted' is the flattened field map from Textract Queries
# or a Bedrock Data Automation blueprint result.

REQUIRED = {
    "claim_number",
    "date_of_service",
    "billed_amount",
    "member_id",
    "secondary_payer_indicator",
}

def gate(extracted, scores, threshold=0.95):
    missing = REQUIRED - set(extracted)
    if missing:
        # Never silently default. This is a routing decision.
        return "HUMAN_REVIEW", {"reason": "missing_fields",
                                "fields": sorted(missing)}

    weak = [f for f in REQUIRED if scores.get(f, 0.0) &lt; threshold]
    if weak:
        return "HUMAN_REVIEW", {"reason": "low_confidence",
                                "fields": sorted(weak)}

    return "STRAIGHT_THROUGH", {}
</code></pre>



<p class="wp-block-paragraph">Two details matter here. The set difference is computed against a declared schema, not against whatever keys happen to be in the response, so an absent field becomes a first-class routing reason. And <code>scores.get(f, 0.0)</code> defaults to zero rather than to a passing value, so a field that arrives without a score fails closed.</p>



<p class="wp-block-paragraph">If you&#8217;re on Textract Queries, there&#8217;s a second reason to be explicit: Queries let you attach an alias to each question, which means your schema keys are yours rather than whatever label happened to be printed on the form. That&#8217;s the difference between &#8220;the field is missing&#8221; and &#8220;the field moved and we didn&#8217;t notice.&#8221;</p>



<h2 class="wp-block-heading">Failure family two: confidence scores that answer a different question</h2>



<p class="wp-block-paragraph">Assume you&#8217;ve fixed presence. The next trap is what the confidence number is measuring.</p>



<p class="wp-block-paragraph">OCR confidence is a statement about characters. It says the model is highly sure those pixels read <code>1,240.00</code>. It is not a statement that <code>1,240.00</code> is the billed amount rather than the allowed amount from the box directly above it, or the prior balance from a remittance summary that happened to be stapled into the same packet. Read it as a legibility score, because that&#8217;s closer to what it is.</p>



<p class="wp-block-paragraph">Bedrock Data Automation narrows this gap: blueprints define fields semantically, confidence scores come with bounding boxes, and the visual grounding lets you point at the region a value came from. Textract Queries narrow it too, by asking a question rather than harvesting a label. Neither eliminates the problem, because a high-confidence read of the wrong region still scores high.</p>



<p class="wp-block-paragraph">What actually catches this is cross-field invariants. They cost almost nothing and they fail for reasons a human can read:</p>



<ul class="wp-block-list">
<li><strong>Arithmetic.</strong> Line items sum to the claimed total. If they don&#8217;t, one of the two is wrong and you don&#8217;t yet know which.</li>

<li><strong>Temporal.</strong> Date of service falls inside the policy period and before the date of submission. A service date after the submission date is a parsing error nine times out of ten.</li>

<li><strong>Referential.</strong> The member or policy identifier resolves against your system of record. An identifier that matches the format but not a real record is a strong signal you read the wrong box.</li>

<li><strong>Range.</strong> Amounts within a plausible band for the claim type. Not a fraud model, just a tripwire for a decimal point that moved.</li>

<li><strong>Page provenance.</strong> Fields that must come from the same page or the same document within the packet. Bounding box data makes this checkable rather than assumed.</li>
</ul>



<p class="wp-block-paragraph">An invariant failure is more useful than a low score, because it names a contradiction. &#8220;Line items sum to 1,180 but the claimed total reads 1,240&#8221; is something a reviewer resolves in seconds. &#8220;Confidence 0.91&#8221; is something a reviewer stares at.</p>



<h2 class="wp-block-heading">Failure family three: the claim that stops halfway</h2>



<p class="wp-block-paragraph">Claims documents are multi-page packets, so you&#8217;ll be using Textract&#8217;s asynchronous operations. That means jobs, notifications, and a whole class of orchestration bugs that only show up under load or after a weekend.</p>



<p class="wp-block-paragraph">The asynchronous pattern is: call <code>StartDocumentAnalysis</code>, get a <code>JobId</code> back, and let Textract publish completion to an SNS topic you nominate. A request looks like this:</p>



<pre class="wp-block-code"><code>{
  "DocumentLocation": {
    "S3Object": { "Bucket": "claims-intake", "Name": "packets/abc123.pdf" }
  },
  "FeatureTypes": ["FORMS", "TABLES"],
  "ClientRequestToken": "abc123-v1",
  "JobTag": "fnol-packet",
  "NotificationChannel": {
    "SNSTopicArn": "arn:aws:sns:REGION:ACCOUNT:textract-complete",
    "RoleArn": "arn:aws:iam::ACCOUNT:role/TextractPublishRole"
  },
  "OutputConfig": {
    "S3Bucket": "claims-extraction",
    "S3Prefix": "raw/"
  },
  "KMSKeyId": "alias/claims-cmk"
}
</code></pre>



<p class="wp-block-paragraph">Three of those parameters are doing load-bearing work that is easy to skip.</p>



<p class="wp-block-paragraph"><code>ClientRequestToken</code> is the idempotency token. Reuse the same token and you get the same <code>JobId</code> back instead of a second job. Derive it from the document, not from the invocation, and a Lambda retry or a duplicated S3 event stops turning into a duplicate charge and a duplicate claim record.</p>



<p class="wp-block-paragraph"><code>OutputConfig</code> writes results into a bucket you control. Without it, results stay internal to Textract and the only way to read them is the <code>Get</code> operations, which have their own throttling limits. Under concurrency those limits become the bottleneck: you end up polling more jobs than you&#8217;re allowed to poll, backing off, and watching end-to-end latency climb for reasons that have nothing to do with the documents. Writing to S3 sidesteps the whole path.</p>



<p class="wp-block-paragraph"><code>JobTag</code> shows up in the completion notification. In a mixed pipeline where the same topic carries first notice of loss packets, itemized bills and ID documents, that tag is what lets the notification handler route without a lookup.</p>



<p class="wp-block-paragraph">One expiry to plan around: a Textract <code>JobId</code> is only valid for seven days. If your retry story is &#8220;requeue it and someone will look on Monday,&#8221; a bad weekend turns recoverable failures into full reprocessing. Persist the S3 output location, not the job identifier.</p>



<h3 class="wp-block-heading">Callbacks that never come back</h3>



<p class="wp-block-paragraph">Human review means pausing a workflow for hours or days, which in Step Functions means the callback pattern. You append <code>.waitForTaskToken</code> to the resource ARN, pass <code>$$.Task.Token</code> into the payload, and the execution parks until something calls <code>SendTaskSuccess</code> or <code>SendTaskFailure</code> with that token.</p>



<p class="wp-block-paragraph">The trap is that a callback task with no timeout waits until the execution itself hits its quota, and Standard workflow executions can run for up to a year. A reviewer who leaves, a review UI that drops the token, a queue consumer that crashes after reading the message and before writing it to the review table: all of these produce an execution that is neither failed nor finished. It just sits there. Nobody alerts on it because nothing broke.</p>



<pre class="wp-block-code"><code>"AwaitAdjusterDecision": {
  "Type": "Task",
  "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
  "Parameters": {
    "FunctionName": "enqueue-review-task",
    "Payload": {
      "claimId.$": "$.claimId",
      "taskToken.$": "$$.Task.Token"
    }
  },
  "TimeoutSeconds": 259200,
  "HeartbeatSeconds": 3600,
  "Catch": [{
    "ErrorEquals": ["States.Timeout"],
    "Next": "EscalateStaleReview"
  }],
  "Next": "ApplyDecision"
}
</code></pre>



<p class="wp-block-paragraph"><code>TimeoutSeconds</code> is the maximum total lifetime of the task regardless of heartbeats. <code>HeartbeatSeconds</code> is the maximum gap between <code>SendTaskHeartbeat</code> calls, so a review app that periodically confirms the item is still in someone&#8217;s queue will fail fast when that app dies, rather than at the outer limit. AWS&#8217;s own guidance is to set the heartbeat below the task timeout for exactly this reason: a heartbeat failure tells you the worker died, a timeout tells you the work took too long, and those are different incidents. Catch <code>States.Timeout</code> and route to a real state. An unhandled timeout is just a differently-shaped silence.</p>



<p class="wp-block-paragraph">One constraint worth knowing before you design around it: the callback pattern requires Standard workflows. Express workflows support request-response integrations only, so no <code>.waitForTaskToken</code> and no <code>.sync</code>. If you split your pipeline into a fast Express path and a Standard review path, the boundary between them is where the token has to live.</p>



<h2 class="wp-block-heading">Failure family four: the human review service you can no longer sign up for</h2>



<p class="wp-block-paragraph">This one catches people copying a reference architecture, and it&#8217;s the reason to read publication dates on IDP blog posts.</p>



<p class="wp-block-paragraph">Amazon Augmented AI, known as A2I, was the managed answer to human-in-the-loop review. It plugged directly into Textract&#8217;s <code>AnalyzeDocument</code>, watched confidence conditions, and spun up review tasks with a worker UI for you. It appears in a great many architecture diagrams for claims and lending workflows.</p>



<p class="wp-block-paragraph">Per the AWS documentation, SageMaker A2I is no longer open to new customers. Existing customers can keep using it, and AWS continues security and availability work, but no new features are planned. If you&#8217;re standing up a new account today, that diagram does not deploy.</p>



<p class="wp-block-paragraph">Be fair about what that costs you, because A2I genuinely removed real work: the task assignment logic, the worker UI, the private workforce plumbing through Cognito, result consolidation. Rebuilding it means owning all of that. What you get back is that the review queue becomes yours, which in practice means you can put claim-specific context on the screen instead of a generic key-value editor. For adjusters that difference is not cosmetic.</p>



<p class="wp-block-paragraph">A minimal replacement is not exotic:</p>



<ul class="wp-block-list">
<li>A DynamoDB table of review items, each holding the claim identifier, the extracted values, the bounding boxes, and the Step Functions task token.</li>

<li>A small web app for reviewers that renders the page image with the boxes overlaid, so a reviewer confirms placement rather than retyping values.</li>

<li>An API that writes the corrected values and calls <code>SendTaskSuccess</code> with the stored token.</li>

<li>Authentication in front of it. Amazon Cognito if you want to stay inside AWS, or an identity-aware proxy such as Cloudflare Access if your reviewers are external adjusters you&#8217;d rather not create AWS identities for.</li>

<li>A sweeper that finds review items older than your heartbeat window and escalates them.</li>
</ul>



<p class="wp-block-paragraph">If the review app is a small internal tool with no data residency requirement of its own, it doesn&#8217;t have to live in the same account or even the same provider. A modest VPS from a host like Contabo or InterServer running behind a Cloudflare Tunnel is a legitimate answer for a reviewer console that talks to AWS over scoped API credentials. Just be honest about what crosses that boundary, which brings us to the next family.</p>



<h2 class="wp-block-heading">Failure family five: claim data in places nobody decided to put it</h2>



<p class="wp-block-paragraph">Claims documents carry protected health information, financial identifiers, and often photographs of people and property. The pipeline you drew has three or four places that data lives. The pipeline you deployed has a dozen.</p>



<p class="wp-block-paragraph">The ones that get missed:</p>



<ul class="wp-block-list">
<li><strong>Lambda logs.</strong> One <code>print</code> of an event payload during a debugging session, and CloudWatch Logs is now a claims repository with a different retention policy and a different access model.</li>

<li><strong>Dead letter queues.</strong> A DLQ holds the full failed message. If that message carries extracted values, your DLQ is regulated data, and it is usually the least governed thing in the account.</li>

<li><strong>Step Functions execution history.</strong> State input and output are visible in the console and the history API. Passing extracted fields between states puts them there.</li>

<li><strong>Intermediate extraction output.</strong> The bucket you pointed <code>OutputConfig</code> at holds raw OCR of the whole packet, often with a lifecycle policy nobody wrote.</li>

<li><strong>Model invocation logging.</strong> Bedrock can log inputs and outputs to S3 or CloudWatch. Useful for debugging, and another copy of everything.</li>
</ul>



<p class="wp-block-paragraph">The pattern that keeps this manageable is passing pointers, not payloads. States carry an S3 key and a claim identifier; the values themselves stay in one encrypted bucket with one lifecycle policy and one access policy. It makes debugging marginally more annoying and it makes the data map fit on a page.</p>



<p class="wp-block-paragraph">On regulated workloads, check the current AWS HIPAA-eligible services list and your executed BAA for every service in the path, in the specific region you&#8217;re deploying to. Eligibility is per service and it changes. Textract has long been used for claims workflows on that basis, and Bedrock is listed as HIPAA eligible, but &#8220;I read a blog post&#8221; is not a control. Pull the list yourself before PHI touches anything.</p>



<p class="wp-block-paragraph">Two smaller things worth deciding early. Reviewers working from home should reach the console over something better than the open internet, whether that&#8217;s a corporate tunnel, a business VPN account from a provider like NordVPN or Surfshark, or an identity-aware proxy. And if reviewers ever download claim documents locally, agree what happens to those files afterward, because a deleted file is not an erased file. Tools such as O&amp;O SafeErase exist for exactly that gap on Windows endpoints.</p>



<h2 class="wp-block-heading">Failure family six: paying twice for the same page</h2>



<p class="wp-block-paragraph">Document AI services bill per page. That single fact reshapes how you think about retries, because in most pipelines a retry is free and here it isn&#8217;t.</p>



<p class="wp-block-paragraph">The expensive patterns are all shaped the same way. A poison document fails a downstream parser, gets requeued, and is re-extracted on every attempt. A batch job re-runs over an entire prefix instead of a delta. A misconfigured S3 event delivers twice. An operator reprocesses a day&#8217;s intake to fix a mapping bug in the transform stage, when the extraction stage was fine all along.</p>



<p class="wp-block-paragraph">The structural fix is separating extraction from interpretation. Extract once, write the raw result to S3 keyed by a content hash of the document, and let every downstream stage read from that. When the mapping bug shows up, you re-run interpretation over stored output and pay nothing. Combined with <code>ClientRequestToken</code>, most accidental double-charges disappear.</p>



<p class="wp-block-paragraph">Also route the packet before you extract it. Forty pages of accident photographs do not need forms and tables analysis. Classification is cheaper than extraction, and page-level routing is often the single largest lever on the bill.</p>



<p class="wp-block-paragraph">For attributing that spend, cost allocation tags on the buckets and functions give you the AWS-native view, and platforms like Vantage or CloudZero are worth a look if you need per-claim or per-client unit costs rather than per-service totals. Whatever you use, the metric that matters is cost per claim processed, split by straight-through versus reviewed. Those two numbers tell you whether the automation is earning its keep.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting an insurance claims processing pipeline on AWS</h2>



<p class="wp-block-paragraph">Symptoms you&#8217;ll actually see, and where to look first.</p>



<ul class="wp-block-list">
<li><strong>Executions succeed but downstream amounts are wrong.</strong> Check whether required fields are present, not just confident. Diff the schema against the response keys for a sample of recent claims. This is failure family one until proven otherwise.</li>

<li><strong>Executions stuck in Running for days.</strong> A callback task with no timeout. List running executions ordered by start time and look for the state name of your review task.</li>

<li><strong>Throttling on the extraction stage under load.</strong> If you&#8217;re polling <code>Get</code> operations, move to <code>OutputConfig</code> and SNS notification and stop polling. If you&#8217;re already there, check the start-operation limits rather than assuming the whole service is slow.</li>

<li><strong>The same claim appearing twice.</strong> Look for a missing or per-invocation <code>ClientRequestToken</code>, and check whether your S3 event handler is idempotent. Delivery is at-least-once.</li>

<li><strong>Extraction quality dropped for one document type.</strong> Usually the form changed, not the model. Compare bounding boxes for the affected field against an older sample. If the box moved, that&#8217;s a layout change, and query aliases or a blueprint update is the fix.</li>

<li><strong>Review queue growing faster than reviewers clear it.</strong> Break the routing reasons apart. If most items are low confidence on one field, that&#8217;s an extraction problem wearing a staffing problem&#8217;s clothes.</li>

<li><strong>SNS notification arrives, handler can&#8217;t find the results.</strong> Confirm the notification role has permission to publish and the handler is reading the S3 prefix rather than calling <code>Get</code> with an expired job identifier.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Gating only on confidence, so a missing field is indistinguishable from a clean extraction.</li>

<li>Defaulting absent values to zero or null in the transform layer instead of raising a routing decision.</li>

<li>Copying an architecture diagram that includes A2I into a new AWS account.</li>

<li>Callback tasks with no <code>TimeoutSeconds</code> and no heartbeat.</li>

<li>Passing extracted claim values through Step Functions state rather than passing an S3 pointer.</li>

<li>Treating a single global confidence threshold as adequate for every field. A name and a dollar amount do not carry the same downstream risk.</li>

<li>Running forms and tables analysis over every page of a packet including the photographs.</li>

<li>No metric for straight-through rate, so nobody notices when it quietly drops.</li>
</ul>



<h2 class="wp-block-heading">Best practices worth the effort</h2>



<ul class="wp-block-list">
<li><strong>Declare the schema, then validate presence, then confidence, then invariants.</strong> Four gates, four distinct rejection reasons, four things a reviewer can act on.</li>

<li><strong>Set per-field thresholds by consequence.</strong> Get the payable amount wrong and money moves. Get a street suffix wrong and a letter is slightly odd.</li>

<li><strong>Keep a labeled regression set.</strong> A few dozen real packets with known-correct values, run on every blueprint or query change. Bedrock Data Automation can use ground-truth examples to refine blueprint instructions, which only works if you maintain the ground truth.</li>

<li><strong>Store bounding boxes alongside values.</strong> They make review faster and they turn &#8220;quality dropped&#8221; from a guess into a comparison.</li>

<li><strong>Alarm on rates, not just errors.</strong> Straight-through rate, review-queue age, and cost per claim. A pipeline that stops approving anything is broken even though nothing threw.</li>

<li><strong>Make every stage idempotent on a content hash.</strong> Reprocessing is normal. It should be safe and cheap.</li>

<li><strong>Instrument the pipeline like a pipeline.</strong> CloudWatch covers the AWS surface; if you&#8217;re consolidating with on-premises claims systems, a platform like Grafana Cloud gives you one place to correlate both sides.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Should I use Amazon Textract or Bedrock Data Automation for claims extraction?</h3>



<p class="wp-block-paragraph">Textract is the sharper tool when your documents are standardized forms and you want deterministic OCR with forms, tables and targeted queries. Bedrock Data Automation is stronger on mixed packets, because it splits along logical document boundaries, classifies each part, and applies a blueprint per document type, with confidence scores and visual grounding on the output. Claims intake is usually mixed packets, which tilts toward Data Automation, but the honest answer is to run both against a sample of your real documents. The evaluation costs a day and it decides your architecture.</p>



<h3 class="wp-block-heading">What replaces Amazon A2I for human review?</h3>



<p class="wp-block-paragraph">For new AWS accounts, a custom review path: a queue or table of review items, a reviewer UI, and the Step Functions callback pattern to resume the workflow. It&#8217;s more code than A2I but not a large amount, and it gives you a review screen designed around claims rather than around generic key-value pairs. Existing A2I customers can continue as they are, though building on a service with no planned features is a decision to make deliberately rather than by default.</p>



<h3 class="wp-block-heading">What straight-through processing rate should I expect?</h3>



<p class="wp-block-paragraph">Anyone quoting you a number without seeing your documents is guessing. It depends almost entirely on document quality and how many fields you require. What&#8217;s reliable is the method: measure your current rate, split failures by reason, and fix the largest reason. Requiring one rarely-present field can dominate everything else, and that&#8217;s a policy decision as much as an engineering one.</p>



<h3 class="wp-block-heading">How do I keep an insurance claims processing pipeline on AWS HIPAA-aligned?</h3>



<p class="wp-block-paragraph">Start from the AWS HIPAA-eligible services list and an executed BAA, and confirm eligibility for each service in your specific region. Then do the unglamorous work: customer-managed KMS keys, no PHI in logs or state payloads, scoped IAM roles per stage, VPC endpoints where the service supports them, retention policies on every bucket and queue including dead letter queues, and CloudTrail configured so you can answer who accessed which claim. Eligibility is permission to build; the controls are yours.</p>



<h3 class="wp-block-heading">Can I run the whole pipeline with Step Functions Express workflows?</h3>



<p class="wp-block-paragraph">Not the part that waits for a human. Express workflows support request-response integrations only, so the callback pattern requires Standard. A common split is Express for the high-volume deterministic stages and Standard for anything holding a task token, with the two connected by an event or a queue.</p>



<h3 class="wp-block-heading">How do I stop duplicate claims from duplicate events?</h3>



<p class="wp-block-paragraph">Treat every trigger as at-least-once. Derive an idempotency key from the document itself, usually a content hash plus a version marker, pass it as <code>ClientRequestToken</code> to the extraction call, and use it as the conditional write key when you create the claim record. Then a duplicate event is a no-op rather than a second claim.</p>



<h3 class="wp-block-heading">Is it worth starting from the AWS GenAI IDP Accelerator?</h3>



<p class="wp-block-paragraph">As a reference for structure and as a way to get a working pipeline in front of stakeholders quickly, yes. As a production system you inherit wholesale, be careful: you&#8217;re adopting someone else&#8217;s opinions about classification, review and storage, and you&#8217;ll be reading that code anyway the first time something behaves oddly. Read it, borrow the patterns, own what you deploy.</p>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">An insurance claims processing pipeline on AWS is not hard to build. Textract, Bedrock Data Automation, Step Functions and S3 will get you a working pipeline in a couple of weeks. What&#8217;s hard is making it fail in ways you can see.</p>



<p class="wp-block-paragraph">Every expensive failure in this space shares one shape: the pipeline had no opinion about what it did not receive. A field that didn&#8217;t come back scored nothing, a callback that never fired errored nothing, a duplicate event failed nothing. If you take one design rule from this, take that one. Declare what a complete claim looks like, check for its absence explicitly, and route anything incomplete to a human with a reason attached.</p>



<p class="wp-block-paragraph">Green executions are not evidence. Reconciled numbers are.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help with your claims pipeline?</h2>



<p class="wp-block-paragraph">I work with teams building document-heavy workflows on AWS, and claims pipelines are one of the places where a small amount of design care prevents a large amount of reconciliation work. Things I can help with:</p>



<ul class="wp-block-list">
<li>Reviewing an existing extraction pipeline for silent-failure paths, particularly missing-field handling and default values in the transform layer</li>

<li>Designing the routing logic: schema gates, per-field thresholds, cross-field invariants, and the escalation rules that sit behind them</li>

<li>Building a human review path with the Step Functions callback pattern, including timeouts, heartbeats and a sweeper for stale tasks</li>

<li>Running a structured evaluation of Amazon Textract against Bedrock Data Automation on your actual documents, with a labeled regression set you keep afterward</li>

<li>Tracing where claim data actually lands across logs, queues, execution history and intermediate buckets, then shrinking that footprint</li>

<li>Instrumenting straight-through rate, review-queue age and cost per claim so regressions surface before finance finds them</li>
</ul>



<p class="wp-block-paragraph">If you&#8217;d like a second opinion, send me a state machine definition, a sample extraction response with the values redacted, or the routing code that decides what goes to review. That&#8217;s usually enough to spot the gap.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/insurance-claims-processing-pipeline-aws/">Building an Insurance Claims Processing Pipeline on AWS That Fails Loudly</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Extracting Clauses, Parties, Dates and Obligations with Amazon Bedrock</title>
		<link>https://john-nessime.com/blog/technical-guides/extract-contract-data-amazon-bedrock/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Tue, 18 Aug 2026 11:17:11 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Lambda]]></category>
		<category><![CDATA[Bedrock Data Automation]]></category>
		<category><![CDATA[Confidence Scoring]]></category>
		<category><![CDATA[Contract Analysis]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[Schema Design]]></category>
		<category><![CDATA[Step Functions]]></category>
		<category><![CDATA[Structured Outputs]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=392</guid>

					<description><![CDATA[<p>Valid JSON is not correct data. A practical guide to extracting clauses, parties, dates and obligations from contracts with Amazon Bedrock: the two build paths, schema design, the three failure families that produce confident wrong answers, and the validation layer that catches them.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/extract-contract-data-amazon-bedrock/">Extracting Clauses, Parties, Dates and Obligations with Amazon Bedrock</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The extraction job finishes clean. Every contract in the batch produces valid JSON, every field is populated, the schema validator passes, and the results load into the obligation tracker without a single error. Three weeks later someone in legal asks why the tracker says the renewal notice window is 30 days when the contract says 90. That is the moment you learn the real problem with contract data extraction: the failure mode is not malformed output. It is well-formed output that is wrong.</p>



<p class="wp-block-paragraph">This post is a practical guide to how I&#8217;d extract contract data with Amazon Bedrock: the clauses, parties, dates, and obligations that feed contract lifecycle tools, obligation trackers, and compliance dashboards. It covers the two build paths Bedrock gives you, how to design the extraction schema, the three failure families that produce confident wrong answers, and the validation layer that catches them before they reach a system someone trusts.</p>



<h2 class="wp-block-heading">Two ways to extract contract data with Amazon Bedrock</h2>



<p class="wp-block-paragraph">Bedrock offers two genuinely different routes to the same destination, and picking the wrong one costs you either flexibility or months of pipeline plumbing you didn&#8217;t need to build.</p>



<h3 class="wp-block-heading">Path one: call a model directly</h3>



<p class="wp-block-paragraph">You send the contract to a foundation model, typically Anthropic Claude, through the Converse or InvokeModel API, with a prompt and a schema describing the fields you want back. Converse accepts PDF documents as a content block, so contract bytes go straight from S3 without a separate OCR step. One detail that catches people: full visual understanding of a PDF through Converse with Claude requires citations to be enabled. Without them the API falls back to plain text extraction, and anything living in a table or a scanned signature page silently disappears from the model&#8217;s view.</p>



<p class="wp-block-paragraph">The piece that changed this path from fragile to dependable is structured outputs. Instead of asking the model nicely for JSON and writing defensive parsing code for the day it prepends an apology, you attach a JSON Schema to the request and Bedrock enforces it during token generation through constrained decoding. The response cannot fail to parse. One mechanic worth knowing: the first request with a new schema triggers a grammar compilation step, and compiled grammars are cached for a limited time, so schema churn has a latency cost.</p>



<h3 class="wp-block-heading">Path two: Bedrock Data Automation</h3>



<p class="wp-block-paragraph">Bedrock Data Automation (BDA) is the managed intelligent document processing service. You define a blueprint, which is a list of fields with types and natural language instructions for how to find and normalize each one, attach it to a project, and invoke the async processing API against files in S3. BDA returns two things a raw model call does not give you out of the box: a confidence score per field and visual grounding, meaning bounding boxes pointing back to where on the page each value came from. A project can hold multiple document blueprints, so a mixed intake of NDAs, master service agreements, and amendments routes each document to the right extraction logic automatically.</p>



<p class="wp-block-paragraph">My honest read on the choice: BDA wins when you want the boring parts managed and you need per-field confidence and grounding for a human review loop, which for contracts you almost always do. The direct model path wins when you need reasoning the blueprint format can&#8217;t express, like interpreting a clause against the definitions section. Plenty of real pipelines use both: BDA for the structured pass, a direct Claude call for the clauses that need interpretation. Amazon Textract still earns its place for high-volume standardized forms, but contracts are exactly the variable, long-tail documents layout-based extraction was never great at.</p>



<h2 class="wp-block-heading">Design the schema before you touch the API</h2>



<p class="wp-block-paragraph">The schema is not a formality. With structured outputs, the field names, descriptions, and ordering you write become context the model generates against, so the schema is effectively part of the prompt. A vague schema produces structurally valid garbage: JSON that parses perfectly and means nothing.</p>



<p class="wp-block-paragraph">Three design rules that pay for themselves. First, one concept per field: &#8220;renewal terms&#8221; as a single string is unusable downstream; split it into whether renewal exists, whether it is automatic, the duration, and the notice period to prevent it. Second, write field descriptions like instructions to a junior reviewer, including what the field is not. Third, make every field nullable and tell the model to return null when the contract is silent. A model forced to fill a required field will fill it with something.</p>



<p class="wp-block-paragraph">Here is the shape I&#8217;d start with for a general commercial agreement, trimmed to the core:</p>



<pre class="wp-block-code"><code>{
  "type": "object",
  "properties": {
    "parties": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "legal_name": { "type": ["string", "null"],
            "description": "Full registered entity name from the preamble or signature block, not the defined shorthand" },
          "defined_term": { "type": ["string", "null"],
            "description": "The shorthand assigned in the preamble, e.g. Supplier, Client" },
          "role": { "type": ["string", "null"] }
        }
      }
    },
    "effective_date": { "type": ["string", "null"],
      "description": "Date the agreement takes effect. Null if only a signature date exists." },
    "governing_law": { "type": ["string", "null"] },
    "obligations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "obligor": { "type": ["string", "null"] },
          "description": { "type": ["string", "null"] },
          "deadline_text": { "type": ["string", "null"],
            "description": "The deadline exactly as written, e.g. within 30 days of the Effective Date" },
          "source_clause": { "type": ["string", "null"],
            "description": "Section number where this obligation appears" }
        }
      }
    }
  }
}</code></pre>



<p class="wp-block-paragraph">Notice <code>deadline_text</code> keeps the deadline as written instead of asking the model to compute a calendar date. That is deliberate, and the reason is the second failure family below.</p>



<h2 class="wp-block-heading">Failure family one: parties that aren&#8217;t who they say they are</h2>



<p class="wp-block-paragraph">Contracts name their parties once, in the preamble, then never again. From that point on everything refers to &#8220;the Supplier&#8221; or &#8220;the Receiving Party.&#8221; A naive extraction returns the defined term as the party name, which is useless for matching against a vendor master or a CRM. Worse, the signature block sometimes names a different legal entity than the preamble, usually an affiliate signing on behalf of a subsidiary, and the model will happily pick whichever it saw last.</p>



<p class="wp-block-paragraph">The fix is structural, not clever prompting. Extract the legal name and the defined term as separate fields, as in the schema above, with instructions to take legal names from the preamble and cross-check the signature block. Then validate outside the model: if the two entities differ, flag the document for review rather than picking one. That mismatch is occasionally a genuine drafting error in the contract itself, which makes it exactly the kind of thing you want surfaced, not smoothed over.</p>



<h2 class="wp-block-heading">Failure family two: dates that mean different things</h2>



<p class="wp-block-paragraph">A contract carries several dates that look interchangeable and are not: execution, effective, commencement, expiry, and the deadlines derived from notice periods. The extraction failure that costs money is almost never a misread date string. It is a correctly read date assigned to the wrong concept, or a relative deadline flattened into a wrong absolute one.</p>



<p class="wp-block-paragraph">Relative dates are the invisible one. &#8220;Either party may terminate by written notice no later than ninety days before the end of the then-current term&#8221; contains no date at all. If your schema demands a date type, the model computes one, and its arithmetic rests on assumptions about the term start that may be wrong, especially when an amendment has reset the term. That computed date looks exactly as trustworthy in your tracker as a real one. This is why I extract deadline language verbatim and compute calendar dates in code, where the arithmetic is testable and the inputs are auditable.</p>



<p class="wp-block-paragraph">Cheap validation catches most of the rest. The effective date must not fall after the expiry. A notice period must be a plausible number of days. If the agreement auto-renews, a renewal duration should exist. Fields that fail these checks get flagged. None of this needs a model; it is twenty lines of code standing between extraction and the systems people trust.</p>



<h2 class="wp-block-heading">Failure family three: obligations, cross-references, and the amendment problem</h2>



<p class="wp-block-paragraph">Obligations are the hardest target because they are relational. &#8220;The Supplier shall deliver the reports described in Schedule 3 within the period set out in Section 4.2, except as provided in Section 9&#8221; is one obligation spread across three locations. Chunk the contract carelessly and the model sees the reference with no Section 4.2 in sight. It will either drop the deadline or invent a plausible one, and the second outcome is far worse because nothing downstream looks broken.</p>



<p class="wp-block-paragraph">Contracts resist chunking more than almost any other document type, because defined terms and cross-references make every section depend on distant ones. The practical answers, in order of preference:</p>



<ol class="wp-block-list">
<li>Don&#8217;t chunk. Modern model context windows fit most commercial agreements whole, and whole-document extraction is the single biggest accuracy win available. Reach for chunking only when a document genuinely doesn&#8217;t fit.</li>



<li>If you must split, prepend the definitions section and the table of contents to every chunk, and split on section boundaries rather than fixed token counts so no clause straddles a cut.</li>



<li>Capture the source clause reference for every obligation, as in the schema above, so a reviewer can jump straight to the language instead of re-reading the contract.</li>
</ol>



<p class="wp-block-paragraph">Then there are amendments. An amendment is a short document that quietly rewrites the base agreement, and if you extract each file independently your dataset will assert the original terms with full confidence long after they stopped being true. Treat the contract family, not the file, as the unit of extraction: process the base agreement, process each amendment, and resolve them in order so later documents override earlier ones. BDA&#8217;s multi-blueprint routing helps here, since amendments have a recognizably different shape and can carry their own blueprint focused on what changed.</p>



<h2 class="wp-block-heading">The validation layer that makes it trustworthy</h2>



<p class="wp-block-paragraph">Schema compliance tells you the output is well-formed. It says nothing about whether it is true. The trust comes from three checks layered after extraction.</p>



<ul class="wp-block-list">
<li><strong>Cross-field consistency.</strong> Dates in order, notice periods plausible, every obligation carrying an obligor that matches an extracted party. Pure code, no model.</li>



<li><strong>Grounding.</strong> Every extracted value should point back to a location in the document. BDA gives you this natively through bounding boxes; on the direct model path, requiring a source clause reference per field is the lightweight version. A value that cannot be traced to the page is a value you cannot audit.</li>



<li><strong>Confidence-based routing.</strong> Use BDA&#8217;s per-field confidence scores to route low-confidence extractions to human review and let high-confidence ones flow through. The goal is not eliminating review; it is spending reviewer minutes only where the model is unsure. Party names and governing law typically extract very reliably, while obligations from unusual templates are where automation honestly ends and review begins.</li>
</ul>



<p class="wp-block-paragraph">Operationally this looks like S3 event notifications triggering a Step Functions workflow: extract, validate, then branch to auto-accept or a review queue. Instrument it from day one. Track the auto-accept rate, per-field confidence distributions, and validation failure counts; a slow drift in confidence is usually the first sign a new contract template has entered the intake. I ship these to Grafana Cloud with the rest of the pipeline metrics, and since direct model calls bill by tokens while BDA bills per unit of content processed, a cost dashboard in something like Vantage or CloudZero shows quickly whether whole-document extraction is worth its token count for your mix. It usually is.</p>



<h2 class="wp-block-heading">Troubleshooting</h2>



<p class="wp-block-paragraph"><strong>Fields come back null for content that is clearly in the document.</strong> On the Converse path with PDFs, check whether you are getting text-only extraction; content in tables, stamps, or scanned pages needs the visual analysis mode. For scans, verify image quality before blaming the model.</p>



<p class="wp-block-paragraph"><strong>The same contract extracts differently on reruns.</strong> Set temperature to zero for extraction work, then look for instructions the model can resolve more than one way. Ambiguity in a schema description is the usual culprit.</p>



<p class="wp-block-paragraph"><strong>First requests with a new schema are slow.</strong> That is grammar compilation for structured outputs. Stabilize your schemas and warm them before batch runs rather than generating schemas dynamically per request.</p>



<p class="wp-block-paragraph"><strong>BDA accuracy is mediocre on your templates.</strong> Blueprint instructions are tunable, and BDA can refine them automatically from a small set of example documents with known correct values. Feed it your hardest real examples, not your cleanest ones.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating schema-valid JSON as verified data. Structure and truth are separate properties, and the second one needs its own checks.</li>



<li>Extracting the defined term as the party name and matching it against vendor records.</li>



<li>Letting the model compute calendar dates from relative deadline language instead of extracting the language and computing in code.</li>



<li>Chunking by token count through a document whose clauses reference each other across fifty pages.</li>



<li>Processing amendments as standalone documents and silently keeping stale base-agreement terms.</li>



<li>Making every schema field required, which converts &#8220;the contract is silent&#8221; into a fabricated answer.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list">
<li>Extract whole documents whenever the context window allows. Every chunking strategy is a workaround, not a feature.</li>



<li>Keep a small labeled evaluation set of your own contracts and score every prompt or blueprint change against it before shipping.</li>



<li>Require a source reference for every extracted value, whether that is a BDA bounding box or a clause number from the model.</li>



<li>Route on confidence and validation results, and measure your auto-accept rate as a first-class metric.</li>



<li>Version your schemas and store the schema version alongside every extraction, so you can tell which records predate a fix.</li>



<li>Keep humans in the loop for high-stakes fields. The realistic goal is automating the bulk and focusing legal review, not replacing it.</li>
</ul>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Should I use Bedrock Data Automation or call Claude directly for contract extraction?</h3>



<p class="wp-block-paragraph">BDA if you want managed parsing, per-field confidence scores, and visual grounding for a review workflow. Direct model calls if you need interpretive reasoning over clauses or conversational access to the document. Many production pipelines combine both.</p>



<h3 class="wp-block-heading">Do I still need Amazon Textract?</h3>



<p class="wp-block-paragraph">Usually not as the primary extractor for contracts, since layout-based extraction struggles with variable legal drafting. It remains a good fit for high-volume standardized forms, and some pipelines route documents between Textract and generative extraction by type.</p>



<h3 class="wp-block-heading">How accurate is contract data extraction in practice?</h3>



<p class="wp-block-paragraph">It varies sharply by field type. Parties, governing law, and explicit dates extract reliably. Obligations, renewal mechanics, and anything expressed through cross-references are meaningfully harder, which is why confidence routing and human review exist. Distrust any vendor quoting one accuracy number for &#8220;contracts.&#8221;</p>



<h3 class="wp-block-heading">How do I extract obligations specifically, not just metadata?</h3>



<p class="wp-block-paragraph">Model each obligation as its own object with obligor, description, deadline language, and source clause. Extract against the whole document so cross-references resolve, and validate that every obligor matches an extracted party.</p>



<h3 class="wp-block-heading">Can Bedrock handle scanned contracts?</h3>



<p class="wp-block-paragraph">Yes. BDA processes scanned documents as part of its parsing stage, and Claude&#8217;s visual document understanding reads scanned pages when invoked with visual analysis. Scan quality still matters; poor scans degrade every approach.</p>



<h3 class="wp-block-heading">Is it safe to put contracts through Bedrock?</h3>



<p class="wp-block-paragraph">Bedrock runs within your AWS account boundary, and AWS states that customer content is not used to train the underlying models. For sensitive agreements the usual controls apply: VPC endpoints for private connectivity, KMS encryption on the buckets, and IAM scoped to the pipeline roles. Confirm the specifics against your own compliance requirements.</p>



<h2 class="wp-block-heading">Conclusion</h2>



<p class="wp-block-paragraph">If you remember one thing, make it this: when you extract contract data with Amazon Bedrock, valid output is the starting line, not the finish. Structured outputs and BDA blueprints have genuinely solved the formatting problem, which means the remaining risk is concentrated in meaning: the wrong entity, the wrong date concept, the obligation whose deadline lives two sections away. Design the schema around those failures, validate outside the model, ground every value to the page, and route on confidence. That is the difference between a demo and a pipeline a legal team will actually trust.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help building a contract extraction pipeline on AWS?</h2>



<p class="wp-block-paragraph">I design and build document intelligence pipelines on AWS as a freelance DevOps engineer. If you&#8217;re extracting data from contracts or other legal documents, here&#8217;s what I can help with:</p>



<ul class="wp-block-list">
<li>Designing extraction schemas and BDA blueprints for your actual contract types, including amendments and multi-document families</li>



<li>Building the full serverless pipeline: S3 intake, Step Functions orchestration, extraction, validation, and review routing</li>



<li>Adding the validation and grounding layer that catches confident wrong answers before they reach your CLM or tracker</li>



<li>Setting up evaluation sets and accuracy measurement so prompt and blueprint changes are tested, not guessed</li>



<li>Locking down the security side: VPC endpoints, KMS encryption, and IAM scoping for sensitive documents</li>



<li>Cost tuning across model choice, whole-document versus chunked extraction, and BDA versus direct model calls</li>
</ul>



<p class="wp-block-paragraph">Send me a sample contract (redacted is fine), your target field list, or the output of a pipeline that isn&#8217;t behaving, and I&#8217;ll tell you what I&#8217;d do with it.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/extract-contract-data-amazon-bedrock/">Extracting Clauses, Parties, Dates and Obligations with Amazon Bedrock</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Building an AI Construction Claims Platform on AWS That Holds Up Under Scrutiny</title>
		<link>https://john-nessime.com/blog/case-studies/ai-construction-claims-platform-aws/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Mon, 03 Aug 2026 14:05:46 +0000</pubDate>
				<category><![CDATA[Case Studies]]></category>
		<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Athena]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon S3 Vectors]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Glue]]></category>
		<category><![CDATA[Bedrock Guardrails]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Embeddings]]></category>
		<category><![CDATA[Generative AI]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[Metadata Filtering]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[Serverless]]></category>
		<category><![CDATA[SQL]]></category>
		<category><![CDATA[Vector Database]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=130</guid>

					<description><![CDATA[<p>Semantic search finds the most persuasive document, not the earliest one. Here is how to architect an AI construction claims and dispute intelligence platform on AWS so retrieval respects the contractual clock, schedule data stays out of the vector index, every answer resolves to a page, and privileged material never shares a retrieval path with project records.</p>
<p>The post <a href="https://john-nessime.com/blog/case-studies/ai-construction-claims-platform-aws/">Building an AI Construction Claims Platform on AWS That Holds Up Under Scrutiny</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Someone hands you a shared drive and asks a question that sounds trivial: &#8220;Did we give notice of the delay event inside the contractual period, or didn&#8217;t we?&#8221;</p>



<p class="wp-block-paragraph">The answer is in there. It is one email, or one line in a site diary, sitting among forty thousand other files. Nobody can read forty thousand files, so the instinct is to point a language model at the pile and ask it. That instinct is right. The naive implementation of it is where the money goes.</p>



<p class="wp-block-paragraph">Here is the failure mode that bites hardest, and it is invisible until an expert challenges you on it. You build retrieval over the document set, ask about notice of delay, and the system confidently returns a letter that discusses the delay event in great detail. It is a good letter. It is also dated eleven months after the event, written by the claims consultant during preparation of the claim itself. It scored highest precisely because it was written to argue the point. The contemporaneous notice, the thing you actually needed, was four badly typed lines in a routine progress email that mentioned the word &#8220;delay&#8221; once.</p>



<p class="wp-block-paragraph">Semantic similarity has no concept of a deadline. That single gap is the difference between an <strong>AI construction claims platform</strong> that shortens a disclosure exercise and one that quietly manufactures a wrong answer with a citation attached to it.</p>



<p class="wp-block-paragraph">This post covers how to build that platform on AWS: how to lay out ingestion, how to make retrieval respect the contractual clock, why schedule data must never go anywhere near your vector index, how to keep privileged material out of the same retrieval path as project records, and which AWS building blocks are actually the current ones now that several of the obvious candidates have been moved to maintenance mode.</p>



<h2 class="wp-block-heading">What a claims platform actually has to answer</h2>



<p class="wp-block-paragraph">Before any architecture, be honest about the question shapes. They are not all the same problem and they do not all get solved by retrieval.</p>



<ol class="wp-block-list"><li><strong>Chronology.</strong> What happened, in what order, and on what date was it recorded? This is a retrieval and metadata problem.</li><li><strong>Entitlement.</strong> Which clause applies, and what did it require the parties to do? This is retrieval over the contract plus careful prompting.</li><li><strong>Causation.</strong> Which event moved the critical path, and by how much? This is schedule data and date arithmetic. It is not a language problem at all.</li><li><strong>Quantum.</strong> What did the disruption cost? This is cost and resource data, joined to the events above.</li></ol>



<p class="wp-block-paragraph">Treat all four as &#8220;ask the documents&#8221; and you will get fluent nonsense on two of them. The architecture below splits them deliberately.</p>



<h2 class="wp-block-heading">Failure one: retrieval that finds the best match instead of the first one</h2>



<p class="wp-block-paragraph">Two corpora live in every dispute bundle and they look identical to an embedding model.</p>



<ul class="wp-block-list"><li><strong>Contemporaneous records.</strong> Site diaries, progress emails, minutes, early warnings, RFIs, instructions. Written while the project was running, by people with no idea a dispute was coming.</li><li><strong>Claim-era material.</strong> Narratives, expert reports, without-prejudice correspondence, internal analysis. Written afterwards, specifically to be persuasive about the same events.</li></ul>



<p class="wp-block-paragraph">Claim-era material wins on cosine similarity almost every time, because it is denser in exactly the terms you searched for. If your retriever cannot distinguish them, every answer is contaminated by the argument you were trying to test.</p>



<p class="wp-block-paragraph">The fix is metadata, applied at ingestion, and it is cheap to get right and expensive to retrofit. Amazon Bedrock Knowledge Bases reads a sidecar file that sits next to each document in S3, named with the full original filename plus <code>.metadata.json</code>. So <code>letter-0421.pdf</code> gets <code>letter-0421.pdf.metadata.json</code>. The naming convention is the only link between them; there is no separate registration step.</p>



<pre class="wp-block-code"><code>{
  "metadataAttributes": {
    "doc_date": 20240314,
    "corpus": "contemporaneous",
    "doc_type": "site_correspondence",
    "matter_id": "matter-0007",
    "date_source": "email_header",
    "privileged": false
  }
}</code></pre>



<p class="wp-block-paragraph">Look closely at <code>doc_date</code>. It is an integer, not a string, and that is not a style choice. Bedrock Knowledge Bases metadata attributes support STRING, NUMBER, BOOLEAN and STRING_LIST. The range comparison operators, the ones you need to express &#8220;on or before the notice deadline&#8221;, only apply to NUMBER. Store the date as <code>"2024-03-14"</code> and your filter will not throw an error. It will just quietly match nothing, or match everything, depending on how you wrote it. You will find out weeks later when someone asks why a document they can see in the bundle never appears in results.</p>



<p class="wp-block-paragraph">With the date as a sortable integer, a query filter can express the contractual window directly.</p>



<pre class="wp-block-code"><code>{
  "andAll": [
    { "equals":              { "key": "corpus",   "value": "contemporaneous" } },
    { "equals":              { "key": "matter_id","value": "matter-0007" } },
    { "greaterThanOrEquals": { "key": "doc_date", "value": 20240301 } },
    { "lessThanOrEquals":    { "key": "doc_date", "value": 20240329 } }
  ]
}</code></pre>



<p class="wp-block-paragraph">That is the whole trick. You are no longer asking &#8220;what is the most relevant document about this delay&#8221;. You are asking &#8220;what did the parties actually write during the window in which the contract required them to write it&#8221;. Those are different questions and only one of them is worth anything in a dispute.</p>



<h3 class="wp-block-heading">Where the date comes from matters more than the date</h3>



<p class="wp-block-paragraph">Do not use the S3 object timestamp. It records when someone copied a folder, usually years after the fact and identical across ten thousand files. Derive the date from the document itself: the <code>Date:</code> header on an email, the printed date on a letter, the period covered by a diary entry.</p>



<p class="wp-block-paragraph">Sometimes you cannot, because the scanned undated fax exists in every project archive. Record that honestly with a <code>date_source</code> attribute rather than guessing, and treat unknown-date documents as a separate review pile. An extension of time argument built on an inferred date is an argument you will lose.</p>



<h2 class="wp-block-heading">Failure two: treating the programme like a document</h2>



<p class="wp-block-paragraph">This one is worse, because the output looks right.</p>



<p class="wp-block-paragraph">Oracle Primavera P6 exports XER and PMXML files. Asta Powerproject and Microsoft Project have their own formats. XER in particular is a plain text dump of relational tables, so it goes through a text pipeline without complaint. Chunk it, embed it, and you now have vectors representing fragments of a table of activity codes with no relationships attached.</p>



<p class="wp-block-paragraph">Ask that index how much float activity A1200 had at the March data date and you will get a number. It will be well formatted and it will be invented. Total float is the product of a forward and backward pass across the whole logic network under a specific calendar. It cannot be recovered from a retrieved fragment, and a language model asked to produce it will produce something plausible instead of admitting that.</p>



<p class="wp-block-paragraph">Schedule data goes into a structured store, and the model queries it rather than reasoning about it.</p>



<ol class="wp-block-list"><li>Parse each programme file into tables. <code>PyP6Xer</code> handles XER from Python; MPXJ is a Java library that reads XER, PMXML, Asta Powerproject and MSPDI among others, which matters when the bundle contains four scheduling tools.</li><li>Load activities, logic links, calendars, resource assignments and WBS into Amazon Aurora PostgreSQL for interactive work, or into S3 with AWS Glue and Amazon Athena when you have hundreds of updates and want columnar scans.</li><li>Stamp every row with the <em>data date</em> of the update it came from. This is the single most important column in the whole platform. Without it you have a pile of schedules; with it you have a time series of the project&#8217;s own view of itself.</li><li>Run windows analysis, as-planned versus as-built comparison and float erosion in SQL or Python, deterministically, so the same inputs always give the same numbers.</li><li>Expose the results to the model as a tool it can call, or as generated SQL against a defined schema. The model turns a question into a query and narrates the result. It does not do the arithmetic.</li></ol>



<p class="wp-block-paragraph">A rough shape of the query that makes float erosion visible:</p>



<pre class="wp-block-code"><code>SELECT
    a.activity_id,
    a.data_date,
    a.total_float_days,
    a.total_float_days - LAG(a.total_float_days)
        OVER (PARTITION BY a.activity_id ORDER BY a.data_date)
      AS float_change
FROM   schedule_activities a
WHERE  a.project_id = 'PRJ-01'
  AND  a.data_date BETWEEN DATE '2024-01-01' AND DATE '2024-06-30'
ORDER BY a.activity_id, a.data_date;</code></pre>



<p class="wp-block-paragraph">Nothing clever there, and that is the point. Every number is traceable to a row that came from a named XER file. When an opposing expert asks where a figure came from, the answer is a file name and a query, not &#8220;the model said so&#8221;.</p>



<p class="wp-block-paragraph">Be realistic about effort here. Programme parsing and normalisation across inconsistent updates is the hardest part of the build and the part clients always underestimate. Activity IDs get reused, calendars change mid-project, and someone will have re-baselined without telling anyone. Budget for it.</p>



<h2 class="wp-block-heading">Failure three: an answer with no paper trail</h2>



<p class="wp-block-paragraph">In most RAG applications a citation is a nice touch. In dispute work it <em>is</em> the product. An answer that cannot be traced to a page of a disclosed document is not evidence, it is a rumour with good grammar.</p>



<p class="wp-block-paragraph">Design for that from the ingestion layer, not the presentation layer.</p>



<ul class="wp-block-list"><li><strong>Keep page and position.</strong> Amazon Bedrock Data Automation returns confidence scores and bounding box data alongside extracted fields, and Amazon Textract returns geometry per block. Carry both through the pipeline so a citation resolves to a page and a region, not just a file.</li><li><strong>Route low confidence to humans.</strong> Handwritten site diaries and faxed variation orders will produce low-confidence extractions. Those should land in a review queue by default rather than silently entering the index.</li><li><strong>Reject ungrounded answers.</strong> Amazon Bedrock Guardrails includes contextual grounding checks that score whether a response is supported by the retrieved passages. It reduces confident invention. It does not eliminate it, and anyone who tells you otherwise is selling something.</li><li><strong>Keep an immutable evidential copy.</strong> S3 Versioning plus S3 Object Lock on the landing bucket means the file the platform indexed is provably the file that was disclosed.</li></ul>



<p class="wp-block-paragraph">One design rule underpins all of it: the platform shortlists evidence, it does not decide entitlement. Recognised frameworks for this work, the Society of Construction Law Delay and Disruption Protocol and AACE International&#8217;s Recommended Practice 29R-03 on forensic schedule analysis, both assume a named analyst applying a stated method and exercising judgement. A system that outputs &#8220;the contractor is entitled to 42 days&#8221; is not helping. A system that outputs &#8220;here are the eleven contemporaneous documents inside the notice window, here is the float movement across those updates, here is what is missing&#8221; is doing real work.</p>



<h2 class="wp-block-heading">Failure four: one index for privileged and non-privileged material</h2>



<p class="wp-block-paragraph">Dispute bundles contain legal advice, counsel&#8217;s opinions, without-prejudice correspondence and internal settlement analysis. Those must not be retrievable through the same path as project records.</p>



<p class="wp-block-paragraph">The tempting shortcut is a <code>privileged: false</code> metadata filter on every query. Do not rely on that as your boundary. A metadata filter is a query parameter. One missing filter in one code path, one debug endpoint, one caching layer that drops it, and privileged material surfaces in a general search. The blast radius of that mistake is not a bug report.</p>



<p class="wp-block-paragraph">Separate the indexes physically and separate the IAM roles that can reach them. Amazon S3 Vectors makes this practical: you can set a dedicated customer-managed KMS key per vector index, and you get a large number of indexes per vector bucket, so per-matter and per-sensitivity separation does not become an operational burden. Keep the metadata flag as well, because defence in depth is free, but make the identity boundary the one you actually trust.</p>



<p class="wp-block-paragraph">Amazon Macie is worth pointing at the landing bucket to find personal data you did not expect, particularly in HR records and accident reports that get swept into project archives.</p>



<h2 class="wp-block-heading">Choosing the AWS building blocks, including what not to build on</h2>



<p class="wp-block-paragraph">A lot of published architectures for this kind of platform are now pointing at services AWS has stopped developing. Two matter here, and the dates are the point.</p>



<ul class="wp-block-list"><li><strong>Amazon Kendra</strong> entered maintenance mode on 30 June 2026 and stops accepting new customers on 30 July 2026. Existing customers keep support and security fixes but no new capability. AWS directs new enterprise search and RAG work to Amazon Bedrock Knowledge Bases. If a tutorial or a proposal you are reading starts with a Kendra index, it predates that change.</li><li><strong>Amazon Bedrock Agents</strong> moved to maintenance mode in the same round of service availability changes, with Amazon Bedrock AgentCore as the successor for agentic orchestration. Check the current AWS service availability page before you commit an orchestration layer.</li></ul>



<p class="wp-block-paragraph">For the retrieval layer itself, Bedrock Knowledge Bases now comes in two shapes and the choice is a real trade-off rather than a marketing tier.</p>



<h3 class="wp-block-heading">Managed Knowledge Base</h3>



<p class="wp-block-paragraph">AWS manages the vector store, embeddings model, re-ranker and retrieval orchestration as a single primitive, with native connectors for Amazon S3, SharePoint, Confluence, Google Drive, OneDrive and a web crawler, plus automatic parsing strategy selection and a retriever that decomposes multi-step queries. The connectors pull source permissions along with content, which matters when the document set lives in the client&#8217;s SharePoint rather than a bucket you control.</p>



<p class="wp-block-paragraph">Where it wins: you get a working retrieval layer in an afternoon instead of a fortnight, and the parsing tuning that normally eats the first weeks of a build is done for you. For a first matter, or a proof of value before a client commits budget, this is the one I would reach for.</p>



<h3 class="wp-block-heading">Custom Knowledge Base</h3>



<p class="wp-block-paragraph">You bring your own vector store and control chunking, embedding model and index layout.</p>



<p class="wp-block-paragraph">Where it wins: claims work has awkward chunking requirements. A two-page letter split mid-sentence at a page boundary produces a chunk where the notice sentence has lost its date and its addressee. Controlling chunk boundaries around document structure, and controlling which index a document lands in, are both easier when you own the store. Where it doesn&#8217;t: you now own embedding model upgrades, re-indexing, sync failures and capacity, which is real ongoing work for a small team.</p>



<p class="wp-block-paragraph">Start managed, build a retrieval evaluation set of real questions with known correct documents, and only move to custom when that set demonstrates the problem is chunking. Most teams migrate on a hunch and discover the problem was metadata all along.</p>



<h3 class="wp-block-heading">Where the vector storage bill actually comes from</h3>



<p class="wp-block-paragraph">Rates change, so learn the billing mechanism rather than a number. Amazon S3 Vectors charges on three axes: upload volume by logical gigabyte, storage by logical gigabyte, and queries by data processed, where data processed scales with the size of the index being searched. Note that filtering does not reduce the data processed by a query.</p>



<p class="wp-block-paragraph">That shape suits claims work unusually well. A dispute archive is enormous and cold: millions of chunks, queried by a handful of analysts a few hundred times a day, so you pay mostly for storage, which is the cheap axis. Compare that against Amazon OpenSearch Serverless, which prices on provisioned compute units and therefore rewards high query volume against a smaller index, or Aurora PostgreSQL with pgvector when you already need Aurora for the schedule tables and would rather run one system than two.</p>



<p class="wp-block-paragraph">The practical lever is to split indexes per matter. Query cost scales with index size, so one giant index across every dispute you have ever run makes every query more expensive than it needs to be, on top of being a bad idea for confidentiality.</p>



<h2 class="wp-block-heading">A reference pipeline</h2>



<ol class="wp-block-list"><li>Everything lands in S3 under a per-matter prefix, with Versioning and Object Lock enabled on the evidential copy.</li><li>S3 event notifications trigger AWS Step Functions. Use Step Functions rather than a chain of Lambdas so that a failed extraction on page 300 of a 400-page bundle is visible and resumable.</li><li>Classify and split. Scanned bundles arrive as one PDF containing forty separate documents. Splitting them correctly is a prerequisite for dating them correctly.</li><li>Extract text with Amazon Bedrock Data Automation or Amazon Textract, keeping confidence scores and geometry.</li><li>Derive the document date and write the <code>.metadata.json</code> sidecar. Anything undated goes to the review queue.</li><li>Route by type: correspondence to the knowledge base, programme files to the XER parser and the relational store, cost data to its own tables.</li><li>Sync the knowledge base, then run your retrieval evaluation set before anyone uses it. A sync that succeeds is not the same as an index that answers correctly.</li><li>Serve through an API that refuses to return an answer without citations, and log every query with the filters that were applied.</li></ol>



<p class="wp-block-paragraph">Define the whole thing in Terraform or OpenTofu from the start. Matters are per-client and short-lived, and standing one up should be a variable file, not an afternoon in the console. Point Amazon CloudWatch, or Grafana Cloud if you already run Grafana elsewhere, at the Step Functions execution metrics so a silently failing extraction stage does not go unnoticed for a week.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting</h2>



<h3 class="wp-block-heading">Date filters return nothing, and no error</h3>



<p class="wp-block-paragraph">Almost always the date was stored as a string. Range operators need NUMBER. Convert to an integer in <code>YYYYMMDD</code> form and re-sync the affected documents.</p>



<h3 class="wp-block-heading">A document is in the bucket but never appears in results</h3>



<p class="wp-block-paragraph">Check the sidecar filename first. It must be the complete original filename with <code>.metadata.json</code> appended, extension included. <code>report.pdf.metadata.json</code> works; <code>report.metadata.json</code> is a file the ingestion job will happily ignore. After that, check whether a filter in the query path is excluding it.</p>



<h3 class="wp-block-heading">Answers cite the right document but the wrong passage</h3>



<p class="wp-block-paragraph">Chunking split the document somewhere structurally meaningful. Look at the raw chunks for that file. If the notice sentence and its date are in different chunks, no amount of prompt tuning fixes it. That is the signal to take control of chunking.</p>



<h3 class="wp-block-heading">Float figures do not match the client&#8217;s own analysis</h3>



<p class="wp-block-paragraph">Check calendars before you check logic. Different activity calendars, a changed default calendar, or an update where someone applied a progress override will move float without any logic change. Reconcile activity counts between your parsed tables and the source file before trusting anything downstream.</p>



<h3 class="wp-block-heading">Query costs jumped without more usage</h3>



<p class="wp-block-paragraph">An index grew. With storage-side vector search, query cost tracks the size of the index being scanned, so ingesting a large new bundle raises the price of every subsequent query against that index. Split by matter.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list"><li>Using the file&#8217;s storage timestamp as the document date. It records the migration, not the event.</li><li>Indexing claim narratives and contemporaneous records into the same corpus with no way to tell them apart.</li><li>Embedding programme exports because they happen to be text files.</li><li>Treating a metadata filter as a privilege boundary instead of an optimisation.</li><li>Letting the model state entitlement conclusions rather than assembling and citing evidence.</li><li>Building on services that have moved to maintenance mode because the tutorial you followed predates the change.</li><li>Shipping without a retrieval evaluation set, so you have no way to know whether a change made things better or worse.</li><li>One index for every matter, which is both a cost problem and a confidentiality problem.</li></ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list"><li>Make the document date a first-class, numeric, filterable attribute, and record where it came from.</li><li>Keep an immutable evidential copy separate from the working copy the pipeline mutates.</li><li>Separate structured schedule and cost data from unstructured documents, and let the model query the former rather than reason about it.</li><li>Build a retrieval evaluation set from real questions with known correct documents before you tune anything.</li><li>Enforce citations at the API layer, so an uncited answer is impossible rather than discouraged.</li><li>Isolate privileged material by index and by IAM role, with metadata as a second layer.</li><li>Log every query with its filters, so you can reconstruct how any given answer was reached.</li><li>Define infrastructure as code so a new matter is a deployment, not a project.</li></ul>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Can an AI construction claims platform replace a delay expert?</h3>



<p class="wp-block-paragraph">No, and building toward that goal produces something unusable. Established forensic frameworks assume a named analyst applying a stated method whose reasoning can be tested. The platform&#8217;s value is compressing weeks of document review into hours and making the schedule data queryable, so the expert spends their time on judgement rather than searching.</p>



<h3 class="wp-block-heading">Should I use Amazon Kendra for the search layer?</h3>



<p class="wp-block-paragraph">Not for a new build. Kendra entered maintenance mode on 30 June 2026 and closed to new customers on 30 July 2026, with AWS pointing to Bedrock Knowledge Bases for equivalent and more current capability. Existing Kendra deployments continue to be supported, so this is a migration assessment rather than an emergency, but starting there now means starting on a service with no roadmap.</p>



<h3 class="wp-block-heading">How do I stop the model inventing float and delay figures?</h3>



<p class="wp-block-paragraph">Do not give it the chance. Keep schedule data in a relational or columnar store and have the model generate queries against a defined schema, or call a tool that runs a fixed calculation. The arithmetic happens in SQL or Python where it is deterministic and reproducible; the model only turns questions into queries and results into sentences.</p>



<h3 class="wp-block-heading">Which vector store should I choose for a claims archive?</h3>



<p class="wp-block-paragraph">Match the store to your query pattern. Large, cold archives queried by a few analysts favour storage-priced options like Amazon S3 Vectors, where you mostly pay to keep the data. Smaller indexes hit constantly favour compute-priced options like Amazon OpenSearch Serverless. If you already run Aurora PostgreSQL for schedule data, pgvector alongside it is a legitimate way to avoid operating a second system.</p>



<h3 class="wp-block-heading">How do I handle scanned and handwritten site records?</h3>



<p class="wp-block-paragraph">Extract them with confidence scores retained, set a threshold, and route everything below it to human review before indexing. Handwritten diaries are frequently the most probative documents in a delay claim and also the least reliable to read automatically, so the review queue is not an edge case. Plan capacity for it.</p>



<h3 class="wp-block-heading">Where do documents come from if they are not already in S3?</h3>



<p class="wp-block-paragraph">Most project records live in a common data environment such as Procore, Autodesk Construction Cloud, Aconex or a client SharePoint tenancy. Bedrock Managed Knowledge Base has native connectors for SharePoint, Confluence, Google Drive and OneDrive that ingest permissions alongside content. For platforms without a native connector, export to S3 and keep the export manifest as part of the disclosure record.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">An <strong>AI construction claims platform</strong> lives or dies on whether it understands time. Every hard requirement in this build traces back to that: numeric dates so you can filter to a contractual window, a data date on every schedule row so float movement is measurable, a corpus flag so contemporaneous records are not drowned out by material written to argue about them, and citations that resolve to a page so any answer can be checked.</p>



<p class="wp-block-paragraph">Get the temporal metadata right at ingestion and the rest of the architecture is ordinary AWS work. Get it wrong and you have built a very expensive way to retrieve the most persuasive document instead of the true one.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help building this on AWS?</h2>



<p class="wp-block-paragraph">I design and build document and data platforms on AWS, and this kind of system sits squarely in that work. Things I can help with:</p>



<ul class="wp-block-list"><li>Designing the ingestion pipeline: S3 landing zones with Object Lock, Step Functions orchestration, splitting and classifying scanned bundles, and confidence-based routing to human review.</li><li>Getting the temporal metadata model right, including date derivation, sidecar generation and filter design against Amazon Bedrock Knowledge Bases.</li><li>Parsing Primavera P6 XER and PMXML exports into queryable tables in Aurora PostgreSQL or S3 with Glue and Athena, with a data date on every row.</li><li>Choosing and sizing the vector layer across Amazon S3 Vectors, OpenSearch Serverless and pgvector, based on your actual query pattern rather than a benchmark.</li><li>Building index and IAM separation for privileged material, plus KMS key strategy and Macie scanning of landing buckets.</li><li>Setting up retrieval evaluation, citation enforcement, query audit logging and CloudWatch or Grafana dashboards over the pipeline so failures surface early.</li></ul>



<p class="wp-block-paragraph">If you are partway into something like this already, send me a sample metadata sidecar, a Step Functions execution history, or a query that returns the wrong document, and I will tell you what I think is going on.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/case-studies/ai-construction-claims-platform-aws/">Building an AI Construction Claims Platform on AWS That Holds Up Under Scrutiny</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
