That is the failure mode that defines this problem. An insurance claims processing pipeline on AWS almost never falls over in the way you designed it to fall over. It succeeds. It emits well-formed JSON. It hands a number to a payment system, and the number is wrong, and nothing in the pipeline had any reason to think otherwise.

This post is organized by failure family rather than by service. I’ll walk through the six ways these pipelines go quietly wrong, what each one costs, and what the fix actually looks like in Textract, Bedrock, Step Functions and S3. There’s a troubleshooting section, the mistakes I see repeated, and an FAQ at the end.

The shape most claims pipelines end up with

Before the failure families make sense, the skeleton. Almost every serverless claims pipeline lands on roughly the same set of stages, whatever the vendor deck calls them:

1. Intake. A document lands in S3 from a portal upload, an SFTP drop, or a mail scanning vendor. An S3 event or EventBridge rule starts an execution.
2. Classification. Work out what the packet actually contains. A first notice of loss, a CMS-style claim form, a police report, an itemized bill, forty pages of photographs.
3. Extraction. Pull the fields you need. Amazon Textract for OCR, forms and tables, or Amazon Bedrock Data Automation with a blueprint that names the fields directly.
4. Validation. Check the extracted values against business rules, policy data, and each other.
5. Routing. Straight-through processing, human review, or rejection with a reason.
6. Persistence and audit. The claim record, the extraction artifacts, and enough evidence to explain a decision months later.

Nothing controversial there. AWS publishes an open-source GenAI IDP Accelerator that implements exactly this shape, with a Bedrock Data Automation mode and a Textract-plus-foundation-model pipeline mode, and it’s a reasonable place to start reading. The interesting part is not the boxes. It’s what happens between them.

Failure family one: the field that was never there

This is the one that pays out the wrong number, and it is worth more attention than everything else in this post combined.

Every extraction service gives you confidence scores. So the obvious design is a gate: if every field scores above some threshold, approve automatically; if anything falls below, send it to a human. That gate is sound reasoning applied to the wrong population.

A confidence score only exists for a value that came back. When the extractor doesn’t find a field at all, there is no low score to catch, because there’s nothing to score. The gate iterates over four returned fields, finds all four above threshold, and reports a clean pass. The fifth field, the one that determines coordination of benefits or the deductible offset, is simply absent from the response. Downstream code treats absent as zero, or as null, or as “not applicable,” and the claim goes through.

The fix is structural, not statistical. Validate presence against a schema before you validate confidence, and treat the two as separate gates with separate outcomes:

# Two gates, not one. Presence first, then confidence.
# 'extracted' is the flattened field map from Textract Queries
# or a Bedrock Data Automation blueprint result.

REQUIRED = {
    "claim_number",
    "date_of_service",
    "billed_amount",
    "member_id",
    "secondary_payer_indicator",
}

def gate(extracted, scores, threshold=0.95):
    missing = REQUIRED - set(extracted)
    if missing:
        # Never silently default. This is a routing decision.
        return "HUMAN_REVIEW", {"reason": "missing_fields",
                                "fields": sorted(missing)}

    weak = [f for f in REQUIRED if scores.get(f, 0.0) < threshold]
    if weak:
        return "HUMAN_REVIEW", {"reason": "low_confidence",
                                "fields": sorted(weak)}

    return "STRAIGHT_THROUGH", {}

Two details matter here. The set difference is computed against a declared schema, not against whatever keys happen to be in the response, so an absent field becomes a first-class routing reason. And scores.get(f, 0.0) defaults to zero rather than to a passing value, so a field that arrives without a score fails closed.

If you’re on Textract Queries, there’s a second reason to be explicit: Queries let you attach an alias to each question, which means your schema keys are yours rather than whatever label happened to be printed on the form. That’s the difference between “the field is missing” and “the field moved and we didn’t notice.”

Failure family two: confidence scores that answer a different question

Assume you’ve fixed presence. The next trap is what the confidence number is measuring.

OCR confidence is a statement about characters. It says the model is highly sure those pixels read 1,240.00. It is not a statement that 1,240.00 is the billed amount rather than the allowed amount from the box directly above it, or the prior balance from a remittance summary that happened to be stapled into the same packet. Read it as a legibility score, because that’s closer to what it is.

Bedrock Data Automation narrows this gap: blueprints define fields semantically, confidence scores come with bounding boxes, and the visual grounding lets you point at the region a value came from. Textract Queries narrow it too, by asking a question rather than harvesting a label. Neither eliminates the problem, because a high-confidence read of the wrong region still scores high.

What actually catches this is cross-field invariants. They cost almost nothing and they fail for reasons a human can read:

• Arithmetic. Line items sum to the claimed total. If they don’t, one of the two is wrong and you don’t yet know which.
• Temporal. Date of service falls inside the policy period and before the date of submission. A service date after the submission date is a parsing error nine times out of ten.
• Referential. The member or policy identifier resolves against your system of record. An identifier that matches the format but not a real record is a strong signal you read the wrong box.
• Range. Amounts within a plausible band for the claim type. Not a fraud model, just a tripwire for a decimal point that moved.
• Page provenance. Fields that must come from the same page or the same document within the packet. Bounding box data makes this checkable rather than assumed.

An invariant failure is more useful than a low score, because it names a contradiction. “Line items sum to 1,180 but the claimed total reads 1,240” is something a reviewer resolves in seconds. “Confidence 0.91” is something a reviewer stares at.

Failure family three: the claim that stops halfway

Claims documents are multi-page packets, so you’ll be using Textract’s asynchronous operations. That means jobs, notifications, and a whole class of orchestration bugs that only show up under load or after a weekend.

The asynchronous pattern is: call StartDocumentAnalysis, get a JobId back, and let Textract publish completion to an SNS topic you nominate. A request looks like this:

{
  "DocumentLocation": {
    "S3Object": { "Bucket": "claims-intake", "Name": "packets/abc123.pdf" }
  },
  "FeatureTypes": ["FORMS", "TABLES"],
  "ClientRequestToken": "abc123-v1",
  "JobTag": "fnol-packet",
  "NotificationChannel": {
    "SNSTopicArn": "arn:aws:sns:REGION:ACCOUNT:textract-complete",
    "RoleArn": "arn:aws:iam::ACCOUNT:role/TextractPublishRole"
  },
  "OutputConfig": {
    "S3Bucket": "claims-extraction",
    "S3Prefix": "raw/"
  },
  "KMSKeyId": "alias/claims-cmk"
}

Three of those parameters are doing load-bearing work that is easy to skip.

ClientRequestToken is the idempotency token. Reuse the same token and you get the same JobId back instead of a second job. Derive it from the document, not from the invocation, and a Lambda retry or a duplicated S3 event stops turning into a duplicate charge and a duplicate claim record.

OutputConfig writes results into a bucket you control. Without it, results stay internal to Textract and the only way to read them is the Get operations, which have their own throttling limits. Under concurrency those limits become the bottleneck: you end up polling more jobs than you’re allowed to poll, backing off, and watching end-to-end latency climb for reasons that have nothing to do with the documents. Writing to S3 sidesteps the whole path.

JobTag shows up in the completion notification. In a mixed pipeline where the same topic carries first notice of loss packets, itemized bills and ID documents, that tag is what lets the notification handler route without a lookup.

One expiry to plan around: a Textract JobId is only valid for seven days. If your retry story is “requeue it and someone will look on Monday,” a bad weekend turns recoverable failures into full reprocessing. Persist the S3 output location, not the job identifier.

Callbacks that never come back

Human review means pausing a workflow for hours or days, which in Step Functions means the callback pattern. You append .waitForTaskToken to the resource ARN, pass $$.Task.Token into the payload, and the execution parks until something calls SendTaskSuccess or SendTaskFailure with that token.

The trap is that a callback task with no timeout waits until the execution itself hits its quota, and Standard workflow executions can run for up to a year. A reviewer who leaves, a review UI that drops the token, a queue consumer that crashes after reading the message and before writing it to the review table: all of these produce an execution that is neither failed nor finished. It just sits there. Nobody alerts on it because nothing broke.

"AwaitAdjusterDecision": {
  "Type": "Task",
  "Resource": "arn:aws:states:::lambda:invoke.waitForTaskToken",
  "Parameters": {
    "FunctionName": "enqueue-review-task",
    "Payload": {
      "claimId.$": "$.claimId",
      "taskToken.$": "$$.Task.Token"
    }
  },
  "TimeoutSeconds": 259200,
  "HeartbeatSeconds": 3600,
  "Catch": [{
    "ErrorEquals": ["States.Timeout"],
    "Next": "EscalateStaleReview"
  }],
  "Next": "ApplyDecision"
}

TimeoutSeconds is the maximum total lifetime of the task regardless of heartbeats. HeartbeatSeconds is the maximum gap between SendTaskHeartbeat calls, so a review app that periodically confirms the item is still in someone’s queue will fail fast when that app dies, rather than at the outer limit. AWS’s own guidance is to set the heartbeat below the task timeout for exactly this reason: a heartbeat failure tells you the worker died, a timeout tells you the work took too long, and those are different incidents. Catch States.Timeout and route to a real state. An unhandled timeout is just a differently-shaped silence.

One constraint worth knowing before you design around it: the callback pattern requires Standard workflows. Express workflows support request-response integrations only, so no .waitForTaskToken and no .sync. If you split your pipeline into a fast Express path and a Standard review path, the boundary between them is where the token has to live.

Failure family four: the human review service you can no longer sign up for

This one catches people copying a reference architecture, and it’s the reason to read publication dates on IDP blog posts.

Amazon Augmented AI, known as A2I, was the managed answer to human-in-the-loop review. It plugged directly into Textract’s AnalyzeDocument, watched confidence conditions, and spun up review tasks with a worker UI for you. It appears in a great many architecture diagrams for claims and lending workflows.

Per the AWS documentation, SageMaker A2I is no longer open to new customers. Existing customers can keep using it, and AWS continues security and availability work, but no new features are planned. If you’re standing up a new account today, that diagram does not deploy.

Be fair about what that costs you, because A2I genuinely removed real work: the task assignment logic, the worker UI, the private workforce plumbing through Cognito, result consolidation. Rebuilding it means owning all of that. What you get back is that the review queue becomes yours, which in practice means you can put claim-specific context on the screen instead of a generic key-value editor. For adjusters that difference is not cosmetic.

A minimal replacement is not exotic:

• A DynamoDB table of review items, each holding the claim identifier, the extracted values, the bounding boxes, and the Step Functions task token.
• A small web app for reviewers that renders the page image with the boxes overlaid, so a reviewer confirms placement rather than retyping values.
• An API that writes the corrected values and calls SendTaskSuccess with the stored token.
• Authentication in front of it. Amazon Cognito if you want to stay inside AWS, or an identity-aware proxy such as Cloudflare Access if your reviewers are external adjusters you’d rather not create AWS identities for.
• A sweeper that finds review items older than your heartbeat window and escalates them.

If the review app is a small internal tool with no data residency requirement of its own, it doesn’t have to live in the same account or even the same provider. A modest VPS from a host like Contabo or InterServer running behind a Cloudflare Tunnel is a legitimate answer for a reviewer console that talks to AWS over scoped API credentials. Just be honest about what crosses that boundary, which brings us to the next family.

Failure family five: claim data in places nobody decided to put it

Claims documents carry protected health information, financial identifiers, and often photographs of people and property. The pipeline you drew has three or four places that data lives. The pipeline you deployed has a dozen.

The ones that get missed:

• Lambda logs. One print of an event payload during a debugging session, and CloudWatch Logs is now a claims repository with a different retention policy and a different access model.
• Dead letter queues. A DLQ holds the full failed message. If that message carries extracted values, your DLQ is regulated data, and it is usually the least governed thing in the account.
• Step Functions execution history. State input and output are visible in the console and the history API. Passing extracted fields between states puts them there.
• Intermediate extraction output. The bucket you pointed OutputConfig at holds raw OCR of the whole packet, often with a lifecycle policy nobody wrote.
• Model invocation logging. Bedrock can log inputs and outputs to S3 or CloudWatch. Useful for debugging, and another copy of everything.

The pattern that keeps this manageable is passing pointers, not payloads. States carry an S3 key and a claim identifier; the values themselves stay in one encrypted bucket with one lifecycle policy and one access policy. It makes debugging marginally more annoying and it makes the data map fit on a page.

On regulated workloads, check the current AWS HIPAA-eligible services list and your executed BAA for every service in the path, in the specific region you’re deploying to. Eligibility is per service and it changes. Textract has long been used for claims workflows on that basis, and Bedrock is listed as HIPAA eligible, but “I read a blog post” is not a control. Pull the list yourself before PHI touches anything.

Two smaller things worth deciding early. Reviewers working from home should reach the console over something better than the open internet, whether that’s a corporate tunnel, a business VPN account from a provider like NordVPN or Surfshark, or an identity-aware proxy. And if reviewers ever download claim documents locally, agree what happens to those files afterward, because a deleted file is not an erased file. Tools such as O&O SafeErase exist for exactly that gap on Windows endpoints.

Failure family six: paying twice for the same page

Document AI services bill per page. That single fact reshapes how you think about retries, because in most pipelines a retry is free and here it isn’t.

The expensive patterns are all shaped the same way. A poison document fails a downstream parser, gets requeued, and is re-extracted on every attempt. A batch job re-runs over an entire prefix instead of a delta. A misconfigured S3 event delivers twice. An operator reprocesses a day’s intake to fix a mapping bug in the transform stage, when the extraction stage was fine all along.

The structural fix is separating extraction from interpretation. Extract once, write the raw result to S3 keyed by a content hash of the document, and let every downstream stage read from that. When the mapping bug shows up, you re-run interpretation over stored output and pay nothing. Combined with ClientRequestToken, most accidental double-charges disappear.

Also route the packet before you extract it. Forty pages of accident photographs do not need forms and tables analysis. Classification is cheaper than extraction, and page-level routing is often the single largest lever on the bill.

For attributing that spend, cost allocation tags on the buckets and functions give you the AWS-native view, and platforms like Vantage or CloudZero are worth a look if you need per-claim or per-client unit costs rather than per-service totals. Whatever you use, the metric that matters is cost per claim processed, split by straight-through versus reviewed. Those two numbers tell you whether the automation is earning its keep.

Troubleshooting an insurance claims processing pipeline on AWS

Symptoms you’ll actually see, and where to look first.

• Executions succeed but downstream amounts are wrong. Check whether required fields are present, not just confident. Diff the schema against the response keys for a sample of recent claims. This is failure family one until proven otherwise.
• Executions stuck in Running for days. A callback task with no timeout. List running executions ordered by start time and look for the state name of your review task.
• Throttling on the extraction stage under load. If you’re polling Get operations, move to OutputConfig and SNS notification and stop polling. If you’re already there, check the start-operation limits rather than assuming the whole service is slow.
• The same claim appearing twice. Look for a missing or per-invocation ClientRequestToken, and check whether your S3 event handler is idempotent. Delivery is at-least-once.
• Extraction quality dropped for one document type. Usually the form changed, not the model. Compare bounding boxes for the affected field against an older sample. If the box moved, that’s a layout change, and query aliases or a blueprint update is the fix.
• Review queue growing faster than reviewers clear it. Break the routing reasons apart. If most items are low confidence on one field, that’s an extraction problem wearing a staffing problem’s clothes.
• SNS notification arrives, handler can’t find the results. Confirm the notification role has permission to publish and the handler is reading the S3 prefix rather than calling Get with an expired job identifier.

Common mistakes

• Gating only on confidence, so a missing field is indistinguishable from a clean extraction.
• Defaulting absent values to zero or null in the transform layer instead of raising a routing decision.
• Copying an architecture diagram that includes A2I into a new AWS account.
• Callback tasks with no TimeoutSeconds and no heartbeat.
• Passing extracted claim values through Step Functions state rather than passing an S3 pointer.
• Treating a single global confidence threshold as adequate for every field. A name and a dollar amount do not carry the same downstream risk.
• Running forms and tables analysis over every page of a packet including the photographs.
• No metric for straight-through rate, so nobody notices when it quietly drops.

Best practices worth the effort

• Declare the schema, then validate presence, then confidence, then invariants. Four gates, four distinct rejection reasons, four things a reviewer can act on.
• Set per-field thresholds by consequence. Get the payable amount wrong and money moves. Get a street suffix wrong and a letter is slightly odd.
• Keep a labeled regression set. A few dozen real packets with known-correct values, run on every blueprint or query change. Bedrock Data Automation can use ground-truth examples to refine blueprint instructions, which only works if you maintain the ground truth.
• Store bounding boxes alongside values. They make review faster and they turn “quality dropped” from a guess into a comparison.
• Alarm on rates, not just errors. Straight-through rate, review-queue age, and cost per claim. A pipeline that stops approving anything is broken even though nothing threw.
• Make every stage idempotent on a content hash. Reprocessing is normal. It should be safe and cheap.
• Instrument the pipeline like a pipeline. CloudWatch covers the AWS surface; if you’re consolidating with on-premises claims systems, a platform like Grafana Cloud gives you one place to correlate both sides.

Frequently asked questions

Should I use Amazon Textract or Bedrock Data Automation for claims extraction?

Textract is the sharper tool when your documents are standardized forms and you want deterministic OCR with forms, tables and targeted queries. Bedrock Data Automation is stronger on mixed packets, because it splits along logical document boundaries, classifies each part, and applies a blueprint per document type, with confidence scores and visual grounding on the output. Claims intake is usually mixed packets, which tilts toward Data Automation, but the honest answer is to run both against a sample of your real documents. The evaluation costs a day and it decides your architecture.

What replaces Amazon A2I for human review?

For new AWS accounts, a custom review path: a queue or table of review items, a reviewer UI, and the Step Functions callback pattern to resume the workflow. It’s more code than A2I but not a large amount, and it gives you a review screen designed around claims rather than around generic key-value pairs. Existing A2I customers can continue as they are, though building on a service with no planned features is a decision to make deliberately rather than by default.

What straight-through processing rate should I expect?

Anyone quoting you a number without seeing your documents is guessing. It depends almost entirely on document quality and how many fields you require. What’s reliable is the method: measure your current rate, split failures by reason, and fix the largest reason. Requiring one rarely-present field can dominate everything else, and that’s a policy decision as much as an engineering one.

How do I keep an insurance claims processing pipeline on AWS HIPAA-aligned?

Start from the AWS HIPAA-eligible services list and an executed BAA, and confirm eligibility for each service in your specific region. Then do the unglamorous work: customer-managed KMS keys, no PHI in logs or state payloads, scoped IAM roles per stage, VPC endpoints where the service supports them, retention policies on every bucket and queue including dead letter queues, and CloudTrail configured so you can answer who accessed which claim. Eligibility is permission to build; the controls are yours.

Can I run the whole pipeline with Step Functions Express workflows?

Not the part that waits for a human. Express workflows support request-response integrations only, so the callback pattern requires Standard. A common split is Express for the high-volume deterministic stages and Standard for anything holding a task token, with the two connected by an event or a queue.

How do I stop duplicate claims from duplicate events?

Treat every trigger as at-least-once. Derive an idempotency key from the document itself, usually a content hash plus a version marker, pass it as ClientRequestToken to the extraction call, and use it as the conditional write key when you create the claim record. Then a duplicate event is a no-op rather than a second claim.

Is it worth starting from the AWS GenAI IDP Accelerator?

As a reference for structure and as a way to get a working pipeline in front of stakeholders quickly, yes. As a production system you inherit wholesale, be careful: you’re adopting someone else’s opinions about classification, review and storage, and you’ll be reading that code anyway the first time something behaves oddly. Read it, borrow the patterns, own what you deploy.

The one thing to take away

An insurance claims processing pipeline on AWS is not hard to build. Textract, Bedrock Data Automation, Step Functions and S3 will get you a working pipeline in a couple of weeks. What’s hard is making it fail in ways you can see.

Every expensive failure in this space shares one shape: the pipeline had no opinion about what it did not receive. A field that didn’t come back scored nothing, a callback that never fired errored nothing, a duplicate event failed nothing. If you take one design rule from this, take that one. Declare what a complete claim looks like, check for its absence explicitly, and route anything incomplete to a human with a reason attached.

Green executions are not evidence. Reconciled numbers are.

Need help with your claims pipeline?

I work with teams building document-heavy workflows on AWS, and claims pipelines are one of the places where a small amount of design care prevents a large amount of reconciliation work. Things I can help with:

• Reviewing an existing extraction pipeline for silent-failure paths, particularly missing-field handling and default values in the transform layer
• Designing the routing logic: schema gates, per-field thresholds, cross-field invariants, and the escalation rules that sit behind them
• Building a human review path with the Step Functions callback pattern, including timeouts, heartbeats and a sweeper for stale tasks
• Running a structured evaluation of Amazon Textract against Bedrock Data Automation on your actual documents, with a labeled regression set you keep afterward
• Tracing where claim data actually lands across logs, queues, execution history and intermediate buckets, then shrinking that footprint
• Instrumenting straight-through rate, review-queue age and cost per claim so regressions surface before finance finds them

If you’d like a second opinion, send me a state machine definition, a sample extraction response with the values redacted, or the routing code that decides what goes to review. That’s usually enough to spot the gap.

The post Building an Insurance Claims Processing Pipeline on AWS That Fails Loudly appeared first on John Nessime.

This post is a practical guide to how I’d extract contract data with Amazon Bedrock: the clauses, parties, dates, and obligations that feed contract lifecycle tools, obligation trackers, and compliance dashboards. It covers the two build paths Bedrock gives you, how to design the extraction schema, the three failure families that produce confident wrong answers, and the validation layer that catches them before they reach a system someone trusts.

Two ways to extract contract data with Amazon Bedrock

Bedrock offers two genuinely different routes to the same destination, and picking the wrong one costs you either flexibility or months of pipeline plumbing you didn’t need to build.

Path one: call a model directly

You send the contract to a foundation model, typically Anthropic Claude, through the Converse or InvokeModel API, with a prompt and a schema describing the fields you want back. Converse accepts PDF documents as a content block, so contract bytes go straight from S3 without a separate OCR step. One detail that catches people: full visual understanding of a PDF through Converse with Claude requires citations to be enabled. Without them the API falls back to plain text extraction, and anything living in a table or a scanned signature page silently disappears from the model’s view.

The piece that changed this path from fragile to dependable is structured outputs. Instead of asking the model nicely for JSON and writing defensive parsing code for the day it prepends an apology, you attach a JSON Schema to the request and Bedrock enforces it during token generation through constrained decoding. The response cannot fail to parse. One mechanic worth knowing: the first request with a new schema triggers a grammar compilation step, and compiled grammars are cached for a limited time, so schema churn has a latency cost.

Path two: Bedrock Data Automation

Bedrock Data Automation (BDA) is the managed intelligent document processing service. You define a blueprint, which is a list of fields with types and natural language instructions for how to find and normalize each one, attach it to a project, and invoke the async processing API against files in S3. BDA returns two things a raw model call does not give you out of the box: a confidence score per field and visual grounding, meaning bounding boxes pointing back to where on the page each value came from. A project can hold multiple document blueprints, so a mixed intake of NDAs, master service agreements, and amendments routes each document to the right extraction logic automatically.

My honest read on the choice: BDA wins when you want the boring parts managed and you need per-field confidence and grounding for a human review loop, which for contracts you almost always do. The direct model path wins when you need reasoning the blueprint format can’t express, like interpreting a clause against the definitions section. Plenty of real pipelines use both: BDA for the structured pass, a direct Claude call for the clauses that need interpretation. Amazon Textract still earns its place for high-volume standardized forms, but contracts are exactly the variable, long-tail documents layout-based extraction was never great at.

Design the schema before you touch the API

The schema is not a formality. With structured outputs, the field names, descriptions, and ordering you write become context the model generates against, so the schema is effectively part of the prompt. A vague schema produces structurally valid garbage: JSON that parses perfectly and means nothing.

Three design rules that pay for themselves. First, one concept per field: “renewal terms” as a single string is unusable downstream; split it into whether renewal exists, whether it is automatic, the duration, and the notice period to prevent it. Second, write field descriptions like instructions to a junior reviewer, including what the field is not. Third, make every field nullable and tell the model to return null when the contract is silent. A model forced to fill a required field will fill it with something.

Here is the shape I’d start with for a general commercial agreement, trimmed to the core:

{
  "type": "object",
  "properties": {
    "parties": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "legal_name": { "type": ["string", "null"],
            "description": "Full registered entity name from the preamble or signature block, not the defined shorthand" },
          "defined_term": { "type": ["string", "null"],
            "description": "The shorthand assigned in the preamble, e.g. Supplier, Client" },
          "role": { "type": ["string", "null"] }
        }
      }
    },
    "effective_date": { "type": ["string", "null"],
      "description": "Date the agreement takes effect. Null if only a signature date exists." },
    "governing_law": { "type": ["string", "null"] },
    "obligations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "obligor": { "type": ["string", "null"] },
          "description": { "type": ["string", "null"] },
          "deadline_text": { "type": ["string", "null"],
            "description": "The deadline exactly as written, e.g. within 30 days of the Effective Date" },
          "source_clause": { "type": ["string", "null"],
            "description": "Section number where this obligation appears" }
        }
      }
    }
  }
}

Notice deadline_text keeps the deadline as written instead of asking the model to compute a calendar date. That is deliberate, and the reason is the second failure family below.

Failure family one: parties that aren’t who they say they are

Contracts name their parties once, in the preamble, then never again. From that point on everything refers to “the Supplier” or “the Receiving Party.” A naive extraction returns the defined term as the party name, which is useless for matching against a vendor master or a CRM. Worse, the signature block sometimes names a different legal entity than the preamble, usually an affiliate signing on behalf of a subsidiary, and the model will happily pick whichever it saw last.

The fix is structural, not clever prompting. Extract the legal name and the defined term as separate fields, as in the schema above, with instructions to take legal names from the preamble and cross-check the signature block. Then validate outside the model: if the two entities differ, flag the document for review rather than picking one. That mismatch is occasionally a genuine drafting error in the contract itself, which makes it exactly the kind of thing you want surfaced, not smoothed over.

Failure family two: dates that mean different things

A contract carries several dates that look interchangeable and are not: execution, effective, commencement, expiry, and the deadlines derived from notice periods. The extraction failure that costs money is almost never a misread date string. It is a correctly read date assigned to the wrong concept, or a relative deadline flattened into a wrong absolute one.

Relative dates are the invisible one. “Either party may terminate by written notice no later than ninety days before the end of the then-current term” contains no date at all. If your schema demands a date type, the model computes one, and its arithmetic rests on assumptions about the term start that may be wrong, especially when an amendment has reset the term. That computed date looks exactly as trustworthy in your tracker as a real one. This is why I extract deadline language verbatim and compute calendar dates in code, where the arithmetic is testable and the inputs are auditable.

Cheap validation catches most of the rest. The effective date must not fall after the expiry. A notice period must be a plausible number of days. If the agreement auto-renews, a renewal duration should exist. Fields that fail these checks get flagged. None of this needs a model; it is twenty lines of code standing between extraction and the systems people trust.

Failure family three: obligations, cross-references, and the amendment problem

Obligations are the hardest target because they are relational. “The Supplier shall deliver the reports described in Schedule 3 within the period set out in Section 4.2, except as provided in Section 9” is one obligation spread across three locations. Chunk the contract carelessly and the model sees the reference with no Section 4.2 in sight. It will either drop the deadline or invent a plausible one, and the second outcome is far worse because nothing downstream looks broken.

Contracts resist chunking more than almost any other document type, because defined terms and cross-references make every section depend on distant ones. The practical answers, in order of preference:

1. Don’t chunk. Modern model context windows fit most commercial agreements whole, and whole-document extraction is the single biggest accuracy win available. Reach for chunking only when a document genuinely doesn’t fit.
2. If you must split, prepend the definitions section and the table of contents to every chunk, and split on section boundaries rather than fixed token counts so no clause straddles a cut.
3. Capture the source clause reference for every obligation, as in the schema above, so a reviewer can jump straight to the language instead of re-reading the contract.

Then there are amendments. An amendment is a short document that quietly rewrites the base agreement, and if you extract each file independently your dataset will assert the original terms with full confidence long after they stopped being true. Treat the contract family, not the file, as the unit of extraction: process the base agreement, process each amendment, and resolve them in order so later documents override earlier ones. BDA’s multi-blueprint routing helps here, since amendments have a recognizably different shape and can carry their own blueprint focused on what changed.

The validation layer that makes it trustworthy

Schema compliance tells you the output is well-formed. It says nothing about whether it is true. The trust comes from three checks layered after extraction.

• Cross-field consistency. Dates in order, notice periods plausible, every obligation carrying an obligor that matches an extracted party. Pure code, no model.
• Grounding. Every extracted value should point back to a location in the document. BDA gives you this natively through bounding boxes; on the direct model path, requiring a source clause reference per field is the lightweight version. A value that cannot be traced to the page is a value you cannot audit.
• Confidence-based routing. Use BDA’s per-field confidence scores to route low-confidence extractions to human review and let high-confidence ones flow through. The goal is not eliminating review; it is spending reviewer minutes only where the model is unsure. Party names and governing law typically extract very reliably, while obligations from unusual templates are where automation honestly ends and review begins.

Operationally this looks like S3 event notifications triggering a Step Functions workflow: extract, validate, then branch to auto-accept or a review queue. Instrument it from day one. Track the auto-accept rate, per-field confidence distributions, and validation failure counts; a slow drift in confidence is usually the first sign a new contract template has entered the intake. I ship these to Grafana Cloud with the rest of the pipeline metrics, and since direct model calls bill by tokens while BDA bills per unit of content processed, a cost dashboard in something like Vantage or CloudZero shows quickly whether whole-document extraction is worth its token count for your mix. It usually is.

Troubleshooting

Fields come back null for content that is clearly in the document. On the Converse path with PDFs, check whether you are getting text-only extraction; content in tables, stamps, or scanned pages needs the visual analysis mode. For scans, verify image quality before blaming the model.

The same contract extracts differently on reruns. Set temperature to zero for extraction work, then look for instructions the model can resolve more than one way. Ambiguity in a schema description is the usual culprit.

First requests with a new schema are slow. That is grammar compilation for structured outputs. Stabilize your schemas and warm them before batch runs rather than generating schemas dynamically per request.

BDA accuracy is mediocre on your templates. Blueprint instructions are tunable, and BDA can refine them automatically from a small set of example documents with known correct values. Feed it your hardest real examples, not your cleanest ones.

Common mistakes

• Treating schema-valid JSON as verified data. Structure and truth are separate properties, and the second one needs its own checks.
• Extracting the defined term as the party name and matching it against vendor records.
• Letting the model compute calendar dates from relative deadline language instead of extracting the language and computing in code.
• Chunking by token count through a document whose clauses reference each other across fifty pages.
• Processing amendments as standalone documents and silently keeping stale base-agreement terms.
• Making every schema field required, which converts “the contract is silent” into a fabricated answer.

Best practices

• Extract whole documents whenever the context window allows. Every chunking strategy is a workaround, not a feature.
• Keep a small labeled evaluation set of your own contracts and score every prompt or blueprint change against it before shipping.
• Require a source reference for every extracted value, whether that is a BDA bounding box or a clause number from the model.
• Route on confidence and validation results, and measure your auto-accept rate as a first-class metric.
• Version your schemas and store the schema version alongside every extraction, so you can tell which records predate a fix.
• Keep humans in the loop for high-stakes fields. The realistic goal is automating the bulk and focusing legal review, not replacing it.

FAQ

Should I use Bedrock Data Automation or call Claude directly for contract extraction?

BDA if you want managed parsing, per-field confidence scores, and visual grounding for a review workflow. Direct model calls if you need interpretive reasoning over clauses or conversational access to the document. Many production pipelines combine both.

Do I still need Amazon Textract?

Usually not as the primary extractor for contracts, since layout-based extraction struggles with variable legal drafting. It remains a good fit for high-volume standardized forms, and some pipelines route documents between Textract and generative extraction by type.

How accurate is contract data extraction in practice?

It varies sharply by field type. Parties, governing law, and explicit dates extract reliably. Obligations, renewal mechanics, and anything expressed through cross-references are meaningfully harder, which is why confidence routing and human review exist. Distrust any vendor quoting one accuracy number for “contracts.”

How do I extract obligations specifically, not just metadata?

Model each obligation as its own object with obligor, description, deadline language, and source clause. Extract against the whole document so cross-references resolve, and validate that every obligor matches an extracted party.

Can Bedrock handle scanned contracts?

Yes. BDA processes scanned documents as part of its parsing stage, and Claude’s visual document understanding reads scanned pages when invoked with visual analysis. Scan quality still matters; poor scans degrade every approach.

Is it safe to put contracts through Bedrock?

Bedrock runs within your AWS account boundary, and AWS states that customer content is not used to train the underlying models. For sensitive agreements the usual controls apply: VPC endpoints for private connectivity, KMS encryption on the buckets, and IAM scoped to the pipeline roles. Confirm the specifics against your own compliance requirements.

Conclusion

If you remember one thing, make it this: when you extract contract data with Amazon Bedrock, valid output is the starting line, not the finish. Structured outputs and BDA blueprints have genuinely solved the formatting problem, which means the remaining risk is concentrated in meaning: the wrong entity, the wrong date concept, the obligation whose deadline lives two sections away. Design the schema around those failures, validate outside the model, ground every value to the page, and route on confidence. That is the difference between a demo and a pipeline a legal team will actually trust.

Need help building a contract extraction pipeline on AWS?

I design and build document intelligence pipelines on AWS as a freelance DevOps engineer. If you’re extracting data from contracts or other legal documents, here’s what I can help with:

• Designing extraction schemas and BDA blueprints for your actual contract types, including amendments and multi-document families
• Building the full serverless pipeline: S3 intake, Step Functions orchestration, extraction, validation, and review routing
• Adding the validation and grounding layer that catches confident wrong answers before they reach your CLM or tracker
• Setting up evaluation sets and accuracy measurement so prompt and blueprint changes are tested, not guessed
• Locking down the security side: VPC endpoints, KMS encryption, and IAM scoping for sensitive documents
• Cost tuning across model choice, whole-document versus chunked extraction, and BDA versus direct model calls

Send me a sample contract (redacted is fine), your target field list, or the output of a pipeline that isn’t behaving, and I’ll tell you what I’d do with it.

The post Extracting Clauses, Parties, Dates and Obligations with Amazon Bedrock appeared first on John Nessime.