<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Construction Technology | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/construction-technology/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/construction-technology/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Sun, 25 Oct 2026 06:06:42 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>Construction Technology | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/construction-technology/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>The Delay Was in the Data Three Weeks Ago: Building a Construction Early Warning Dashboard</title>
		<link>https://john-nessime.com/blog/data-engineering/construction-early-warning-dashboard/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sun, 25 Oct 2026 06:06:42 +0000</pubDate>
				<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Monitoring]]></category>
		<category><![CDATA[Alerting]]></category>
		<category><![CDATA[Capital Projects]]></category>
		<category><![CDATA[Delay Analysis]]></category>
		<category><![CDATA[Earned Schedule]]></category>
		<category><![CDATA[Grafana]]></category>
		<category><![CDATA[Last Planner System]]></category>
		<category><![CDATA[Leading Indicators]]></category>
		<category><![CDATA[Owner Dashboard]]></category>
		<category><![CDATA[PostgreSQL]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[Procore]]></category>
		<category><![CDATA[Project Controls]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Total Float]]></category>
		<category><![CDATA[XER Files]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=1033</guid>

					<description><![CDATA[<p>Schedule delay rarely arrives as a surprise. It sits in float erosion, aging RFIs and slipping weekly commitments long before the monthly update moves the finish date. Here is how to build a dashboard that catches it while there is still float left to spend.</p>
<p>The post <a href="https://john-nessime.com/blog/data-engineering/construction-early-warning-dashboard/">The Delay Was in the Data Three Weeks Ago: Building a Construction Early Warning Dashboard</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The monthly schedule update lands on a Thursday. The planner has re-run the logic, and substantial completion has moved three weeks to the right. The project manager asks the obvious question in the next meeting: why is this the first time anyone is hearing about it?</p>



<p class="wp-block-paragraph">The uncomfortable answer is that people did know. The superintendent knew the drywall crew had been short for two weeks. The project engineer knew a structural RFI had been sitting with the design team well past its due date. The foreman knew the weekly commitments kept slipping. None of it was hidden. It just never reached one screen, in one place, while there was still float left to spend.</p>



<p class="wp-block-paragraph">This post walks through building a <strong>construction early warning dashboard</strong> that catches those signals while they are still cheap to act on. It covers why schedule delay shows up late, the one data decision that makes early warning possible, the signals worth tracking, how to pull data out of Primavera P6, Microsoft Project and Procore, and how to alert without training everyone to ignore the alerts.</p>



<h2 class="wp-block-heading">Why Construction Delay Shows Up Three Weeks Late</h2>



<p class="wp-block-paragraph">Delay rarely arrives as one event. It accumulates in small amounts across activities that do not look critical yet. Three things keep it invisible.</p>



<ol class="wp-block-list">
<li><strong>The schedule is updated in batches.</strong> If the CPM schedule is statused monthly, or even biweekly, the finish date is only recalculated at that cadence. Everything that happened in between lands at once.</li>



<li><strong>Progress is self-reported.</strong> Percent complete is an estimate from whoever statuses the activity. Optimism is not dishonesty, but it compounds. An activity that sits at a comfortable-looking percentage for three updates is telling you something.</li>



<li><strong>The data is overwritten in place.</strong> This is the one that bites hardest. Most schedule tools, and most dashboards built on top of them, show the current value of float, the current forecast finish and the current status. Last month&#8217;s values are gone unless someone saved the file.</li>
</ol>



<p class="wp-block-paragraph">That third point is why so many project dashboards look fine right up until they don&#8217;t. A current-state dashboard tells you where you are. Delay warnings live in the rate of change. An activity with twelve days of float looks healthy. The same activity with twelve days of float, down from thirty over the last two updates, is the next critical path.</p>



<h2 class="wp-block-heading">The Design Decision That Makes Early Warning Possible: Snapshot Everything</h2>



<p class="wp-block-paragraph">Before any chart or alert, decide that every schedule update, every RFI log pull and every weekly plan gets stored as a dated snapshot. You append. You never overwrite. Storage for this is trivial; a few hundred activities per update is nothing for any database.</p>



<p class="wp-block-paragraph">PostgreSQL is what I reach for first here. It runs happily on a small managed instance like Amazon RDS or DigitalOcean Managed Databases, or on a modest VPS from Hetzner, Contabo or Interserver if you are comfortable handling backups yourself. A minimal activity snapshot table looks like this:</p>



<pre class="wp-block-code"><code>CREATE TABLE activity_snapshot (
    project_code     text        NOT NULL,
    data_date        date        NOT NULL,
    task_code        text        NOT NULL,
    task_name        text,
    total_float_days numeric,
    early_start      date,
    early_finish     date,
    is_complete      boolean     NOT NULL DEFAULT false,
    loaded_at        timestamptz NOT NULL DEFAULT now(),
    PRIMARY KEY (project_code, data_date, task_code)
);</code></pre>



<p class="wp-block-paragraph">The primary key does the important work. Each activity appears once per data date, so loading the same export twice fails loudly instead of silently doubling the history. The <code>data_date</code> is the schedule&#8217;s status date, not the day you loaded the file, because trends must be plotted against when the schedule says it was true.</p>



<p class="wp-block-paragraph">Keying on <code>task_code</code>, the activity ID the planner sees, rather than an internal database ID matters too. Internal IDs can change when a schedule is re-imported. Activity IDs usually survive, and when they don&#8217;t, you want to know (more on that in troubleshooting).</p>



<h2 class="wp-block-heading">Early Warning Signals That Move Before the Finish Date Does</h2>



<p class="wp-block-paragraph">Each of these is a leading indicator of schedule delay. None is perfect alone. Together they give you weeks of notice instead of a surprise in the monthly update.</p>



<h3 class="wp-block-heading">Float Erosion on Near-Critical Paths</h3>



<p class="wp-block-paragraph">Watching the critical path is necessary but late. By the time an activity is critical, its float is already gone. The useful signal is near-critical work losing float quickly. This query compares each open activity to its own previous snapshot:</p>



<pre class="wp-block-code"><code>WITH history AS (
    SELECT
        project_code,
        task_code,
        task_name,
        data_date,
        total_float_days,
        LAG(total_float_days) OVER (
            PARTITION BY project_code, task_code
            ORDER BY data_date
        ) AS prev_float_days
    FROM activity_snapshot
    WHERE NOT is_complete
)
SELECT
    project_code,
    task_code,
    task_name,
    prev_float_days,
    total_float_days,
    prev_float_days - total_float_days AS float_lost_days
FROM history
WHERE data_date = (SELECT max(data_date) FROM activity_snapshot)
  AND prev_float_days IS NOT NULL
  AND total_float_days &lt;= 20
  AND prev_float_days - total_float_days &gt;= 5
ORDER BY float_lost_days DESC;</code></pre>



<p class="wp-block-paragraph">The thresholds (twenty days remaining, five days lost) are starting points, not rules. Tune them to your update cadence and project length. What matters is that the query looks at movement, which a current-state report cannot.</p>



<h3 class="wp-block-heading">Constraint Aging: RFIs and Submittals Against Need-By Dates</h3>



<p class="wp-block-paragraph">An overdue RFI on its own is noise. Every project has them. An overdue RFI that blocks an activity starting in ten days is a delay you can see coming.</p>



<p class="wp-block-paragraph">The hard part is the link. RFI and submittal logs in Procore or Autodesk Construction Cloud rarely point at schedule activity IDs. Someone has to maintain a small mapping table from RFI or submittal to activity, or at least to area and spec section. I would not skip it. Without the link you get a list of late paperwork; with it you get &#8220;this answer is needed in eight days and the ball has been in the architect&#8217;s court for three weeks.&#8221;</p>



<p class="wp-block-paragraph">The signal to chart is days of slack between the expected response and the need-by date of the linked activity. When it goes negative, the RFI is now on the schedule&#8217;s path whether the schedule knows it or not.</p>



<h3 class="wp-block-heading">Commitment Reliability: PPC and Reasons for Variance</h3>



<p class="wp-block-paragraph">If the field runs the Last Planner System, you already have a strong signal. Percent Plan Complete (PPC) is the share of weekly commitments finished as promised. Each week&#8217;s PPC is a lagging number, but the trend across weeks leads the CPM schedule, because missed promises show up in the field long before they show up as recalculated dates.</p>



<p class="wp-block-paragraph">Store reasons for variance as categories, not free text. A PPC dip caused by weather tells you something different from three straight weeks of &#8220;material not on site.&#8221; The second one is a procurement problem that will keep producing delay until someone fixes it.</p>



<h3 class="wp-block-heading">Look-Ahead Churn</h3>



<p class="wp-block-paragraph">Count how many times an activity&#8217;s planned start has moved across snapshots of the three- or six-week look-ahead. An activity pushed out once is normal planning. One that has been pushed four updates running is being deferred, not planned, and the reason is usually a constraint nobody has named yet.</p>



<h3 class="wp-block-heading">Earned Value, Read Carefully</h3>



<p class="wp-block-paragraph">If you track earned value, the Schedule Performance Index (SPI = EV / PV) belongs on the dashboard, with a caveat many dashboards skip. Classic SPI drifts back toward 1.0 as the project nears completion, even when the job is late, because earned value and planned value both end at the budget at completion. A late-stage SPI that looks like recovery may just be arithmetic.</p>



<p class="wp-block-paragraph">Earned Schedule fixes this by measuring in time rather than money. It asks at what point in the baseline your current earned value was supposed to be reached, then divides that by the actual time elapsed to get SPI(t). It needs no new data collection, only different math on the same numbers. If you only plot one schedule index late in a job, plot SPI(t).</p>



<h3 class="wp-block-heading">Claimed Progress Versus Manpower</h3>



<p class="wp-block-paragraph">Daily logs record who was on site. When an activity&#8217;s reported percent complete keeps rising while the manpower logged against that trade stays well below the plan, one of those numbers is wrong. Usually it is the percent complete. This comparison is crude, and it needs a consistent trade mapping, but it is one of the few checks that uses data nobody is trying to make look good.</p>



<h2 class="wp-block-heading">Getting the Data Out of P6, Microsoft Project and Procore</h2>



<p class="wp-block-paragraph">A construction project dashboard lives or dies on boring extraction work. Plan for it.</p>



<h3 class="wp-block-heading">Primavera P6 XER Exports</h3>



<p class="wp-block-paragraph">An XER file is plain tab-delimited text. Lines beginning with <code>%T</code> name a table, <code>%F</code> lists its fields, and <code>%R</code> lines are rows. Activities live in the <code>TASK</code> table. This reader pulls one table into a list of dictionaries:</p>



<pre class="wp-block-code"><code>def read_xer_table(path, table_name):
    rows, fields, current = [], None, None
    # XER exports often use a Windows code page, not UTF-8
    with open(path, encoding="cp1252", errors="replace") as fh:
        for line in fh:
            parts = line.rstrip("rn").split("t")
            tag = parts[0]
            if tag == "%T":
                current = parts[1]
            elif tag == "%F" and current == table_name:
                fields = parts[1:]
            elif tag == "%R" and current == table_name:
                rows.append(dict(zip(fields, parts[1:])))
    return rows

tasks = read_xer_table("update.xer", "TASK")
for t in tasks[:5]:
    print(t["task_code"], t["task_name"], t["total_float_hr_cnt"])</code></pre>



<p class="wp-block-paragraph">Two details catch people. Float and durations in XER are stored in hours (<code>total_float_hr_cnt</code>), so converting to days means dividing by the hours per day of each activity&#8217;s calendar, not a blanket eight. And an activity with a value in <code>act_end_date</code> has an actual finish, which is a clean way to set <code>is_complete</code> without relying on status labels.</p>



<p class="wp-block-paragraph">Use the float P6 stored in the export rather than recomputing the critical path yourself. P6&#8217;s result reflects its scheduling options, constraints and calendars. Your recompute probably won&#8217;t.</p>



<h3 class="wp-block-heading">Microsoft Project</h3>



<p class="wp-block-paragraph">Microsoft Project can save a schedule as XML, which carries tasks, dates and total slack. The same snapshot logic applies. Save one file per status date, load it with its data date, and never replace the previous one.</p>



<h3 class="wp-block-heading">Procore and Other Project Management Platforms</h3>



<p class="wp-block-paragraph">Procore&#8217;s REST API exposes RFIs and submittals per project, including status, due dates and ball-in-court. Two warnings. Most useful endpoints are scoped to a single project, so a portfolio view means iterating over the project list. And submittal status values can be configured per company, so map them explicitly rather than hard-coding what you saw on one account. Autodesk Construction Cloud has its own RFI API with a similar shape.</p>



<h3 class="wp-block-heading">The Spreadsheet Nobody Wants to Admit Exists</h3>



<p class="wp-block-paragraph">The weekly work plan is very often an Excel file or a Smartsheet. Accept that. A scheduled job that reads a fixed-layout sheet from a shared folder is less elegant than an API and far more likely to get used. Fighting the field&#8217;s tools is how early warning systems die.</p>



<h2 class="wp-block-heading">Building the Dashboard and Alerts People Don&#8217;t Mute</h2>



<p class="wp-block-paragraph">For visualization I lean toward Grafana, self-hosted or on Grafana Cloud, because its alert rules can run SQL directly against PostgreSQL and route results to email, Slack or Microsoft Teams. You can see the style of dashboard in this <a href="https://monitor.john-nessime.com/public-dashboards/4c1463601230401ab55159059364b739" target="_blank" rel="noreferrer noopener">live Grafana demo</a>.</p>



<p class="wp-block-paragraph">Power BI deserves a fair hearing. Many contractors already license it, project controls teams know it, and its reporting and drill-down are excellent for monthly reviews. Where it is weaker is rule-based alerting on arbitrary queries. If your organization lives in Microsoft 365, a sensible split is Power BI for the reporting pack and a small scheduled job or Grafana instance for the alerts. Metabase is another reasonable middle ground for smaller teams.</p>



<p class="wp-block-paragraph">Whatever the tool, the alert design matters more than the charts:</p>



<ul class="wp-block-list">
<li><strong>Send a weekly digest, not a page.</strong> Schedule risk is not an outage. A Monday summary of every signal that crossed a threshold gets read. Twenty individual notifications get filtered.</li>



<li><strong>Name the owner in the alert.</strong> &#8220;RFI 142 blocks level 3 slab pour, need-by in 8 days, ball in court: structural engineer&#8221; is actionable. &#8220;Constraint risk: high&#8221; is not.</li>



<li><strong>Compare the project to itself.</strong> Portfolio-wide thresholds punish complex jobs and let simple ones coast. A drop from its own recent average is the better trigger.</li>



<li><strong>Alert on stale data.</strong> If no schedule update has loaded in longer than your cadence, that is itself a warning. An early warning dashboard showing old data is worse than none.</li>
</ul>



<h2 class="wp-block-heading">Troubleshooting</h2>



<h3 class="wp-block-heading">Float values look wrong compared to P6</h3>



<p class="wp-block-paragraph">Check the hours-to-days conversion first. Then check which float definition the schedule uses. A project-level &#8220;must finish by&#8221; date or activity constraints can produce negative float that looks like a bug but is correct. Also confirm whether the planner defines critical as longest path or as float below a threshold.</p>



<h3 class="wp-block-heading">Trend lines break after a re-baseline or renumbering</h3>



<p class="wp-block-paragraph">If activity IDs change, <code>LAG()</code> finds no previous row and the activity appears brand new. Add a check that counts activities present in the previous snapshot but missing from the current one. A sudden jump means the history needs a mapping table, not a fresh start.</p>



<h3 class="wp-block-heading">PPC sits at or near 100 percent every week</h3>



<p class="wp-block-paragraph">That usually means commitments are being made at the end of the week, or tasks are so loose they cannot fail. A PPC that never moves carries no signal. Look at how commitments are recorded before trusting the number.</p>



<h3 class="wp-block-heading">The dashboard is green but the field says you are behind</h3>



<p class="wp-block-paragraph">Check how percent complete is calculated. P6 supports duration, physical and units percent complete types, and they can tell very different stories about the same activity. Believe the field, then find which input is out of step.</p>



<h2 class="wp-block-heading">Common Mistakes</h2>



<ul class="wp-block-list">
<li>Building on the live schedule only, with no history, so every trend has to be reconstructed by hand.</li>



<li>Watching only the critical path and missing the near-critical work that becomes critical next month.</li>



<li>Trusting classic SPI in the final third of a project, when it drifts toward 1.0 regardless of performance.</li>



<li>Tracking overdue RFIs without linking them to the activities they block.</li>



<li>Recomputing float yourself instead of using what the scheduling tool calculated.</li>



<li>Sending so many alerts that the project team creates an inbox rule to hide them.</li>
</ul>



<h2 class="wp-block-heading">Best Practices</h2>



<ul class="wp-block-list">
<li>Append every export with its data date. Never overwrite history.</li>



<li>Start with two signals, float erosion and constraint aging, and add more once those are trusted.</li>



<li>Agree with the planner on update cadence. The dashboard cannot be fresher than its inputs.</li>



<li>Keep the RFI-to-activity mapping small and owned by one named person.</li>



<li>Review thresholds after the first month. Early numbers are guesses.</li>



<li>Show each signal with its source and data date, so nobody argues about whether it is current.</li>
</ul>



<h2 class="wp-block-heading">Frequently Asked Questions</h2>



<h3 class="wp-block-heading">What are leading indicators of construction delay?</h3>



<p class="wp-block-paragraph">The most useful are float erosion on near-critical activities, RFIs and submittals aging toward the start of the work they block, a falling PPC trend, activities repeatedly pushed out of the look-ahead, and reported progress outpacing logged manpower.</p>



<h3 class="wp-block-heading">How often should the schedule be updated for early warning to work?</h3>



<p class="wp-block-paragraph">More often than monthly if you can manage it. Weekly or biweekly updates give the trend queries enough points to show movement. Signals from RFIs and weekly plans can refresh daily regardless of schedule cadence.</p>



<h3 class="wp-block-heading">Is SPI a good early warning metric?</h3>



<p class="wp-block-paragraph">Early and mid-project, yes, alongside float and critical path analysis. Late in a project, classic SPI drifts toward 1.0 even on late jobs, so the time-based SPI(t) from Earned Schedule is more trustworthy.</p>



<h3 class="wp-block-heading">Can I build this with Power BI instead of Grafana?</h3>



<p class="wp-block-paragraph">Yes for the reporting side, and it is often the better choice where the team already uses it. The snapshot database and trend logic are the same either way. Rule-based alerting is where you may want a separate tool.</p>



<h3 class="wp-block-heading">Do I need Primavera P6 for a construction early warning dashboard?</h3>



<p class="wp-block-paragraph">No. Any scheduling tool that exports activities with dates and float works, including Microsoft Project. What you need is a consistent export on every update and somewhere to keep the history.</p>



<h3 class="wp-block-heading">How do I link RFIs to schedule activities?</h3>



<p class="wp-block-paragraph">Usually with a small mapping table maintained by the project engineer: RFI or submittal number, activity ID, and need-by date. Mapping by area and spec section is a workable first step if activity-level links are too much at the start.</p>



<h2 class="wp-block-heading">Conclusion</h2>



<p class="wp-block-paragraph">The project manager who learns about delay three weeks late usually isn&#8217;t short of data. The problem is that the data is spread across a schedule, an RFI log and a weekly plan, and each one only remembers its latest state. A construction early warning dashboard works because it keeps the history and watches the movement: float draining from work that isn&#8217;t critical yet, answers arriving after the work that needs them, promises slipping week after week. Snapshot first. Everything else is a query on top.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need Help Building a Construction Early Warning Dashboard?</h2>



<p class="wp-block-paragraph">I build and host the data side of this for construction teams and project controls groups. Here is where I can help:</p>



<ul class="wp-block-list">
<li>Setting up a snapshot database that stores every P6 or Microsoft Project update with its data date.</li>



<li>Writing XER and XML loaders that handle hours-based float, calendars and renumbered activities.</li>



<li>Pulling RFIs and submittals from Procore or Autodesk Construction Cloud and linking them to activities.</li>



<li>Building Grafana dashboards for float erosion, constraint aging, PPC and SPI(t) trends.</li>



<li>Designing weekly digest alerts with named owners, sent to email, Slack or Teams.</li>



<li>Hosting and maintaining the dashboard so your team reads it instead of running it.</li>
</ul>



<p class="wp-block-paragraph">If you want to scope it first, the <a href="https://john-nessime.com/tools/engagement-builder/">Engagement Builder</a> lays out phases and effort. Or send me one recent XER export and your RFI log, and I will tell you which signals your data can already support.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://john-nessime.com/projects/hosted-monitoring/">See Hosted Monitoring</a></div>



<div class="wp-block-button is-style-outline is-style-outline--1"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/data-engineering/construction-early-warning-dashboard/">The Delay Was in the Data Three Weeks Ago: Building a Construction Early Warning Dashboard</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Primavera P6 vs Unifier vs Aconex vs Textura: Which Oracle Product Do You Actually Need?</title>
		<link>https://john-nessime.com/blog/devops/primavera-p6-vs-unifier-vs-aconex/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Fri, 28 Aug 2026 18:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[BIM]]></category>
		<category><![CDATA[Capital Projects]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Common Data Environment]]></category>
		<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[Enterprise Software]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Oracle]]></category>
		<category><![CDATA[Oracle Aconex]]></category>
		<category><![CDATA[Oracle Textura]]></category>
		<category><![CDATA[Primavera Gateway]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[Primavera Unifier]]></category>
		<category><![CDATA[Project Controls]]></category>
		<category><![CDATA[REST API]]></category>
		<category><![CDATA[Self Hosting]]></category>
		<category><![CDATA[Software Selection]]></category>
		<category><![CDATA[System of Record]]></category>
		<category><![CDATA[Vendor Lock-In]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=252</guid>

					<description><![CDATA[<p>All four Oracle Construction and Engineering products overlap on purpose, and three of them hold cost. The damage comes from owning two and never deciding which one is authoritative. A practical comparison by whose record each product holds, plus a decision procedure and the integration constraints nobody scopes.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/primavera-p6-vs-unifier-vs-aconex/">Primavera P6 vs Unifier vs Aconex vs Textura: Which Oracle Product Do You Actually Need?</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The forecast in the monthly report doesn&#8217;t match the forecast in the cost system. Someone asks which one is right, and the honest answer is that nobody knows, because there are three places holding a number and nobody ever wrote down which one wins.</p>



<p class="wp-block-paragraph">That conversation is usually the first sign that an organisation bought two Oracle products without deciding what each one was for. It is a common outcome, and it is not because the products are bad. It&#8217;s because they are sold as a suite, they overlap on purpose, and the sales conversation is about capability rather than authority.</p>



<p class="wp-block-paragraph">This post is a working comparison of the four Oracle Construction and Engineering products people most often confuse: <strong>Primavera P6 vs Unifier vs Aconex</strong>, plus Textura. One profile each, what it genuinely owns, where it wins, where it doesn&#8217;t, and then the part nobody covers: how to decide which system holds the authoritative number when three of them offer a cost module.</p>



<h2 class="wp-block-heading">The question that actually sorts them</h2>



<p class="wp-block-paragraph">Feature lists will not separate these products. All four manage documents. Three of them track cost. Two of them hold a schedule. If you compare capability columns you will conclude that you need all four, which is exactly the conclusion the suite is designed to produce.</p>



<p class="wp-block-paragraph">The question that does separate them is: <em>whose record is this, and who else needs to write to it?</em></p>



<p class="wp-block-paragraph">Two of these products are single-organisation systems. You run them, you own the data, and outside parties get in only if you give them a seat and they accept your governance. One of them is deliberately multi-organisation, where every company on the project holds its own workspace and decides what it shares. And one is a transactional network that touches money leaving your bank account, which makes it a finance system wearing a construction badge.</p>



<p class="wp-block-paragraph">Get that axis right and the rest of the decision mostly falls out.</p>



<h2 class="wp-block-heading">Primavera P6: the schedule, and only the schedule</h2>



<p class="wp-block-paragraph">P6 is a critical path method scheduling engine with resource and cost loading on top. It exists to answer questions about time: what drives the finish date, what happens if this activity slips, what does the resource histogram look like if we accelerate.</p>



<p class="wp-block-paragraph">Two things confuse people here. First, P6 comes in two shapes. P6 Professional is the desktop client that planners actually live in. P6 EPPM is the enterprise web layer with the shared database, portfolio views and web services. Most organisations run both against the same database, and most planners will fight you to keep the desktop client.</p>



<p class="wp-block-paragraph">Second, Oracle Primavera Cloud is a separate product, not P6 in a browser. Oracle says so on its own product page, which is unusually direct for vendor copy. Primavera Cloud shares a lineage with P6 and is positioned as the cloud destination, but the data model is not identical and things planners rely on in P6 do not all have equivalents. If someone tells you a move to Primavera Cloud is a hosting change, they have not tried it.</p>



<h3 class="wp-block-heading">Where P6 wins</h3>



<ul class="wp-block-list">
<li>Contractual programme scheduling, where the schedule is a deliverable and gets audited</li>

<li>Forensic delay work, because the XER and PMXML formats are what every expert and every third-party analysis tool expects</li>

<li>Resource-loaded planning across a portfolio with a shared resource pool</li>

<li>Anywhere the client&#8217;s contract names it, which is more places than you would expect</li>
</ul>



<h3 class="wp-block-heading">Where P6 doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Cost control. P6 has cost fields and they are useful for cash flow shape, but a resource-loaded schedule is not a commitment ledger. It has no concept of a purchase order lifecycle, a variation approval or a payment certificate</li>

<li>Correspondence and document control. There is nothing here for transmittals, RFIs or drawing registers</li>

<li>Collaboration across companies. Subcontractors do not get P6 logins, and you would not want them writing to your programme if they did</li>
</ul>



<h2 class="wp-block-heading">Primavera Unifier: a business process engine that happens to know about projects</h2>



<p class="wp-block-paragraph">Unifier is the one people describe worst, usually as &#8220;the cost one&#8221;. That undersells it and also sets bad expectations.</p>



<p class="wp-block-paragraph">Unifier is a configurable workflow platform for capital programmes. You define business processes: a change order, a funding request, an invoice, a payment application, a site instruction. Each one has a form, a workflow with approval steps and routing rules, and a defined effect on the cost sheet. Oracle ships preconfigured processes and a no-code designer called uDesigner for changing them or building your own. Alongside the business processes sit the managers, including the Cost Manager, Schedule Manager and Document Manager.</p>



<p class="wp-block-paragraph">It&#8217;s sold as base products, principally Project Controls and Facilities and Asset Management, which matters because Unifier&#8217;s remit runs past handover into operating the asset.</p>



<p class="wp-block-paragraph">The honest trade-off with Unifier is this: its configurability is the reason to buy it and the reason implementations run long. A platform that can model any approval workflow will not tell you what your approval workflow should be. If your organisation has not written its cost control process down, Unifier will faithfully automate the argument you are having instead of the process you wanted.</p>



<h3 class="wp-block-heading">Where Unifier wins</h3>



<ul class="wp-block-list">
<li>Owner-side capital programme control, where budget, funding, commitments, forecast and actuals need to roll up across many projects</li>

<li>Anywhere approvals need an audit trail with defined routing, delegation and thresholds</li>

<li>Public sector and utility programmes with fund tracking, where money has strings attached and the strings must be reportable</li>

<li>Organisations that want one governed process applied to every project rather than per-project improvisation</li>
</ul>



<h3 class="wp-block-heading">Where Unifier doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Supply chain collaboration. It&#8217;s your instance, your governance, your seats. External parties can be given access, but they are guests in your system and they behave like it</li>

<li>Drawing and model control at scale. The Document Manager is real, but it is not a common data environment and it is not built for design coordination</li>

<li>Fast starts. Configuration is a project in its own right, and treating it as an install is the most common way to burn a first year</li>
</ul>



<h2 class="wp-block-heading">Aconex: the project record, owned by nobody in particular</h2>



<p class="wp-block-paragraph">Aconex is a common data environment for correspondence, documents, workflows and models across every organisation on a project. Document control, transmittals, RFIs, submittals, drawing registers, model coordination, field inspections.</p>



<p class="wp-block-paragraph">Its defining property is not a feature. It is the data ownership model. Every organisation on the project gets its own workspace and controls what it shares and when. That neutrality is the entire commercial argument, and it is the thing you cannot replicate by inviting subcontractors into your instance of something else.</p>



<p class="wp-block-paragraph">This matters more than it sounds. A designer who believes the main contractor can read their internal drafts will keep their real work somewhere else, and your &#8220;single source of truth&#8221; quietly becomes a partial one. The reason Aconex accumulates a complete project record is that nobody is afraid of it. Oracle&#8217;s own materials lean on that neutrality hard, and in this case the marketing claim and the engineering consequence are the same thing.</p>



<p class="wp-block-paragraph">On the model side, Aconex Model Coordination is design-neutral and built on open BIM standards including IFC and BCF, so it sits alongside the authoring tools rather than replacing them.</p>



<h3 class="wp-block-heading">Where Aconex wins</h3>



<ul class="wp-block-list">
<li>Multi-party projects where the correspondence record will be read by lawyers later</li>

<li>Design-build and joint venture delivery, where no single party should hold everyone&#8217;s data</li>

<li>Document and model volume, which Oracle&#8217;s datasheets describe as unbounded on data and participants</li>

<li>Handover, because the audit trail and the linked document set are the deliverable</li>
</ul>



<h3 class="wp-block-heading">Where Aconex doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Portfolio-level capital planning. Aconex is organised around projects, not around a programme of investment decisions</li>

<li>Asset lifecycle after handover. The record ends where the project ends</li>

<li>Deep enterprise cost governance. Connected Cost exists and is genuinely useful for project cost, but if your organisation needs funding, capital planning and multi-year rollup, that is Unifier&#8217;s shape</li>
</ul>



<h2 class="wp-block-heading">Textura: money leaving the building</h2>



<p class="wp-block-paragraph">Textura came into Oracle by acquisition and still feels different from the rest of the suite, because it is solving a different class of problem.</p>



<p class="wp-block-paragraph">Textura Payment Management runs the construction payment cycle: initiating a draw, subcontractor pay applications, markups and approvals, owner billing, collecting lien waivers and compliance documents, enforcing compliance holds, payment authorisation and disbursement. It&#8217;s a network product. Your subcontractors transact in it, and the value comes from them being there.</p>



<p class="wp-block-paragraph">The compliance side is the part that decides whether it&#8217;s worth buying. If your risk is a subcontractor&#8217;s insurance lapsing quietly and nobody noticing until a claim, an automated payment hold tied to a missing document is a control you cannot easily build with a spreadsheet and good intentions.</p>



<h3 class="wp-block-heading">Where Textura wins</h3>



<ul class="wp-block-list">
<li>General contractors with a wide subcontractor base and a monthly draw cycle that eats accounts payable alive</li>

<li>Jurisdictions with real lien exposure, where waiver collection is a legal control rather than paperwork</li>

<li>Compliance enforcement at the point of payment, which is the only moment anyone reliably pays attention</li>

<li>Audit trails on who approved what, signed what and released what</li>
</ul>



<h3 class="wp-block-heading">Where Textura doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>It is not a cost control system. It handles the payment leg, not the budget, forecast or commitment ledger</li>

<li>It does not replace your ERP. It integrates with it, and that integration is part of the implementation cost</li>

<li>Adoption is a supply chain problem, not a software problem. If your subcontractors will not onboard, you own a slower process than the one you replaced</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The overlap trap, and why the cost module is where it bites</h2>



<p class="wp-block-paragraph">Here&#8217;s the failure mode that is invisible at procurement and expensive eighteen months later.</p>



<p class="wp-block-paragraph">Three of these products can hold cost. P6 has cost-loaded activities. Unifier has a full Cost Manager. Aconex has Connected Cost. Nobody sets out to run three cost systems, but it happens gradually: the schedule gets cost-loaded because the client asked for an S-curve, the finance team runs the commitment ledger in Unifier, and the project team starts logging variations in Aconex because that&#8217;s where the correspondence already is.</p>



<p class="wp-block-paragraph">Now you have three forecasts. They disagree. Not because anyone is wrong, but because they are answering different questions with different cut-off dates and different accrual rules. And the reconciliation lands in a spreadsheet maintained by one person, which is the outcome every one of these products was bought to prevent.</p>



<p class="wp-block-paragraph">The fix is a decision, not a tool. Before anything is configured, write down one sentence per data class: which system is authoritative, and what the others are allowed to hold.</p>



<ul class="wp-block-list">
<li><strong>Schedule dates and logic:</strong> P6 is authoritative. Everything else receives dates, nothing else edits them</li>

<li><strong>Budget, commitments and forecast:</strong> pick one. If you own Unifier, it is Unifier. If you don&#8217;t, Connected Cost is a reasonable home, but say so out loud</li>

<li><strong>Correspondence and controlled documents:</strong> Aconex if it&#8217;s on the project, and if it is, do not run a parallel drawing register anywhere else</li>

<li><strong>Subcontractor payment and compliance status:</strong> Textura, with the resulting actuals flowing to whichever system owns cost</li>

<li><strong>Everything the authoritative system does not own:</strong> read-only, refreshed by integration, and clearly labelled in reports as a copy</li>
</ul>



<p class="wp-block-paragraph">That list takes an afternoon and saves a year. It&#8217;s also the single most common thing missing from Oracle Primavera implementations I get asked to look at.</p>



<h2 class="wp-block-heading">Integration is the real cost, and it has a shape</h2>



<p class="wp-block-paragraph">Once you own two of these, you own an integration. Budget for it as a workstream, not a checkbox.</p>



<p class="wp-block-paragraph">Oracle offers more than one path. <strong>Primavera Gateway</strong> is the established broker for P6 and Unifier, with built-in providers per application and a canonical intermediate format so mappings survive version changes. In the P6 to Unifier direction, Gateway maps the P6 Project ID to Unifier&#8217;s Project Number, and what flows depends on how the schedule is built: a duration-based schedule sends durations, a resource-loaded one sends durations and units. Oracle also has a lighter integration platform for exchanging data between Unifier, P6, Aconex and Primavera Cloud, so check which one your version and deployment model actually supports before designing anything.</p>



<p class="wp-block-paragraph">Aconex Connected Cost takes a different route, talking to P6 over its web services API. Two constraints from that integration are worth knowing before you draw an architecture diagram:</p>



<ul class="wp-block-list">
<li>You can configure both a P6 and a Primavera Cloud integration for the organisation, but an individual project can only link to one of them at a time</li>

<li>Self-hosted or third-party-hosted P6 servers have to be approved by Oracle and reachable over the internet before the integration can be established. Oracle-hosted P6 skips that approval step</li>
</ul>



<p class="wp-block-paragraph">That second one has teeth if you are running P6 EPPM on your own infrastructure. &#8220;Internet accessible&#8221; and &#8220;exposed&#8221; are not the same thing, and the gap between them is where your security review lives. If you&#8217;re standing up middleware or a reverse proxy in front of an on-premise P6 or Unifier deployment, that box needs to be treated as production: a dedicated host from a provider like Contabo or InterServer rather than a corner of an existing server, TLS terminated properly, access restricted to known networks, and remote administrative access over a business VPN such as NordLayer or Surfshark rather than a port forward someone opened for a consultant and forgot about.</p>



<p class="wp-block-paragraph">Back up the database host properly too, and test a restore. On-premise P6 and Unifier estates are frequently the last thing anyone thinks about until they need it, and imaging tools from vendors like O&amp;O Software earn their keep on exactly this kind of long-lived, rarely-touched server.</p>



<h2 class="wp-block-heading">How I&#8217;d decide</h2>



<p class="wp-block-paragraph">Work through this in order and stop at the first answer that fits. Buying more than you need is far more common than buying too little.</p>



<ol class="wp-block-list">
<li><strong>Name the pain in one sentence.</strong> &#8220;We cannot defend our programme in a delay claim&#8221; points at P6. &#8220;We cannot tell the board what the capital programme will cost&#8221; points at Unifier. &#8220;We cannot find the approved revision&#8221; points at Aconex. &#8220;Subcontractor payments take six weeks and compliance is a filing cabinet&#8221; points at Textura</li>

<li><strong>Count the organisations that must write to the system.</strong> One, and you are choosing between P6 and Unifier. Many, and you are choosing between Aconex and Textura</li>

<li><strong>Ask whether the problem outlives the project.</strong> Assets, maintenance and multi-year capital planning are Unifier&#8217;s territory. Anything that ends at handover is not</li>

<li><strong>Check what the contract already mandates.</strong> Plenty of clients name a scheduling tool or a project CDE in the contract documents. That decision is made, so spend your effort elsewhere</li>

<li><strong>Ask the reseller how each product meters.</strong> Per named user, per project, per transaction or per value processed changes the architecture, not just the invoice. A per-user product pushes you towards fewer, more senior users, and that changes who does data entry</li>

<li><strong>Only then decide on a second product,</strong> and write the system-of-record sentence for every data class the two of them share before signing anything</li>
</ol>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<ul class="wp-block-list">
<li><strong>&#8220;Unifier can do document control, so we don&#8217;t need Aconex.&#8221;</strong> Technically true, practically false on multi-party projects. The obstacle is not features, it is that other companies will not put their real correspondence in your instance</li>

<li><strong>&#8220;Aconex has cost, so we don&#8217;t need Unifier.&#8221;</strong> Fine for project-level cost. It falls down when you need portfolio funding, capital planning and asset lifecycle</li>

<li><strong>&#8220;Primavera Cloud is just P6 in the browser.&#8221;</strong> Oracle itself states that Primavera Cloud is not P6. Treat any migration as a data model exercise with a gap analysis, not a lift and shift</li>

<li><strong>&#8220;We&#8217;ll integrate it later.&#8221;</strong> Later means after users have created data under two incompatible breakdown structures. Agree the project and cost breakdown codes on day one, even if the integration itself waits</li>

<li><strong>&#8220;The suite integrates out of the box.&#8221;</strong> Connectors exist and they are good. Mapping your codes, your workflow states and your calendars is still work, and it is work that needs someone who understands both sides</li>
</ul>



<h2 class="wp-block-heading">Practices worth adopting early</h2>



<ul class="wp-block-list">
<li>Standardise the project ID before anything else. Gateway maps P6&#8217;s Project ID to Unifier&#8217;s Project Number, and a free-text naming convention will hurt on the day you connect them</li>

<li>Keep the schedule in one place and let it flow outward. Bidirectional date editing across two systems is a support ticket generator</li>

<li>Pilot Unifier configuration on one real process end to end before configuring twenty. uDesigner makes it easy to build the wrong thing quickly</li>

<li>Onboard the supply chain before go-live on Aconex or Textura. Network products fail on adoption, not on uptime</li>

<li>Export your own data on a schedule. XER and PMXML for schedules, and regular extracts from the cloud products, so you are not negotiating from zero at renewal</li>

<li>Write the system-of-record statement into the implementation charter, where people will actually see it</li>
</ul>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">What is the difference between Primavera P6 and Unifier?</h3>



<p class="wp-block-paragraph">P6 is a critical path scheduling tool that answers questions about time and resources. Unifier is a configurable business process and cost platform that answers questions about money, approvals and governance across a capital programme. They overlap slightly because P6 can hold cost data and Unifier has a Schedule Manager, but neither is a substitute for the other.</p>



<h3 class="wp-block-heading">Do I need Aconex if I already have Unifier?</h3>



<p class="wp-block-paragraph">If the project involves several independent organisations exchanging drawings and correspondence, usually yes. Unifier&#8217;s Document Manager works well for your own records, but external parties are guests in your instance. Aconex gives every organisation its own workspace and control over what it shares, which is what makes the project record complete rather than partial.</p>



<h3 class="wp-block-heading">Is Oracle Primavera Cloud replacing P6?</h3>



<p class="wp-block-paragraph">Oracle positions Primavera Cloud as the cloud platform and publishes migration guidance from P6, while continuing to sell and support P6 EPPM. Oracle also states plainly that Primavera Cloud is not P6. Plan on the two coexisting for now, and treat any migration as a data model exercise with a proper gap analysis rather than a hosting change.</p>



<h3 class="wp-block-heading">Can Textura replace our accounting system?</h3>



<p class="wp-block-paragraph">No. Textura handles the construction payment cycle: pay applications, approvals, lien waivers, compliance holds and disbursement. It integrates with ERP systems rather than replacing them, and that integration is a real line item in the implementation.</p>



<h3 class="wp-block-heading">Which product should a general contractor start with?</h3>



<p class="wp-block-paragraph">It depends on which pain is loudest. If schedules are being disputed, P6. If the project record is scattered across email, Aconex. If the monthly draw cycle and subcontractor compliance are consuming the finance team, Textura. Contractors reach for Unifier less often than owners do, because Unifier&#8217;s strengths are programme-level funding and governance.</p>



<h3 class="wp-block-heading">How hard is it to integrate P6 with Unifier?</h3>



<p class="wp-block-paragraph">The mechanism is well trodden through Primavera Gateway, which ships providers for both. The hard part is not the connector, it&#8217;s agreement: matching project identifiers, deciding which system owns dates, and reconciling breakdown structures that were designed independently. Expect the mapping work to take longer than the technical setup.</p>



<h3 class="wp-block-heading">What data should I be exporting for portability?</h3>



<p class="wp-block-paragraph">For schedules, take regular XER and PMXML exports, since those formats are what independent analysis tools and delay experts read. For the cloud products, set up scheduled extracts of documents, transaction records and audit trails. The point is not distrust of the vendor, it&#8217;s that a project record you cannot read without an active subscription is a liability during a dispute.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">The <strong>Primavera P6 vs Unifier vs Aconex</strong> comparison, with Textura alongside them, is not really a features question. Each of these products is good at the thing it was built for, and Oracle is not trying to trick anyone by giving three of them a cost module.</p>



<p class="wp-block-paragraph">The damage comes from owning two of them and never deciding which one is authoritative. Sort the products by whose record they hold and how many organisations write to them, pick the smallest set that covers your loudest pain, and write down the system of record for every data class before a single workflow gets configured.</p>



<p class="wp-block-paragraph">Do that and the monthly report matches the cost system. Skip it and you&#8217;ll meet the spreadsheet.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help untangling this?</h2>



<p class="wp-block-paragraph">Most of the work in this area is not clicking through screens, it&#8217;s the plumbing and the decisions around it. That part I can help with:</p>



<ul class="wp-block-list">
<li>Writing the system-of-record map for your data classes before you commit to a second Oracle product, so cost and dates have one owner each</li>

<li>Designing and building integrations between P6, Unifier, Aconex, Textura and your ERP, including the field mapping and error handling nobody scopes</li>

<li>Hardening on-premise P6 EPPM or Unifier deployments that need to be reachable for a cloud integration, with proper TLS, network restrictions and audited access</li>

<li>Automating schedule and cost data extraction into a warehouse or BI layer, so reporting stops depending on someone manually exporting on a Friday</li>

<li>Building repeatable backup, restore-testing and disaster recovery for self-hosted Primavera estates</li>

<li>Second-opinion reviews on an implementation or integration design that already exists and is not behaving</li>
</ul>



<p class="wp-block-paragraph">If you&#8217;re stuck on something concrete, send it over: an integration mapping spec, a Gateway error, a network diagram for an on-premise P6 that needs to talk to a cloud service. Easier to give you something useful when I can see the actual thing.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/primavera-p6-vs-unifier-vs-aconex/">Primavera P6 vs Unifier vs Aconex vs Textura: Which Oracle Product Do You Actually Need?</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>As-Planned vs As-Built Analysis: Building a Platform That Survives Cross-Examination</title>
		<link>https://john-nessime.com/blog/technical-guides/as-planned-vs-as-built-analysis-platform/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Mon, 24 Aug 2026 13:00:00 +0000</pubDate>
				<category><![CDATA[Case Studies]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[As-Built Schedule]]></category>
		<category><![CDATA[Audit Logging]]></category>
		<category><![CDATA[Capital Projects]]></category>
		<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Data Lineage]]></category>
		<category><![CDATA[Delay Analysis]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Extension of Time]]></category>
		<category><![CDATA[Forensic Schedule Analysis]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[PostgreSQL]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[Project Controls]]></category>
		<category><![CDATA[Python]]></category>
		<category><![CDATA[Schema Design]]></category>
		<category><![CDATA[XER Files]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=274</guid>

					<description><![CDATA[<p>Most as-planned vs as-built analysis compares the baseline to the last P6 update and calls the result an as-built. It isn't one. This is how to build the data layer underneath a delay analysis: versioned XER ingestion, activity identity across renumbering, a first-appearance table that proves when every actual date entered the record, calendar-safe float, and record linking that proposes candidates instead of asserting cause.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/as-planned-vs-as-built-analysis-platform/">As-Planned vs As-Built Analysis: Building a Platform That Survives Cross-Examination</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The question that kills a delay analysis is never about methodology. It is five words from the other side&#8217;s expert: &#8220;which update did that come from?&#8221;</p>



<p class="wp-block-paragraph">Picture the experts&#8217; meeting. Your exhibit shows activity C-2140, structural steel erection to level four, finishing well behind where the baseline put it. He has the same XER files you do. He opens update 19 and points out that C-2140 carries no actual finish. Same in update 20. In update 21 an actual finish appears, and the date on it sits three weeks in the past, inside the window you already closed out.</p>



<p class="wp-block-paragraph">So which file is the record? And if one activity was back-filled a month after the fact, what does that say about the other four thousand?</p>



<p class="wp-block-paragraph">That is the failure an <strong>as-planned vs as-built analysis</strong> has to survive, and no amount of care inside the Excel workbook fixes it. It is a data problem before it is a scheduling problem. This post covers the platform underneath the analysis: how to ingest a long series of Primavera P6 updates, how to prove where every actual date came from, and how to produce output a tribunal can audit instead of having to trust.</p>



<h2 class="wp-block-heading">The as-built is not a file, and treating it as one is the invisible failure</h2>



<p class="wp-block-paragraph">Almost every quick as-planned vs as-built comparison does the same thing: take the approved baseline, take the last update in the sequence, join on activity ID, subtract the dates, sort by variance. It produces a clean-looking table in an afternoon.</p>



<p class="wp-block-paragraph">It is also the weakest possible construction of an as-built, for three reasons that never show up in the output.</p>



<ul class="wp-block-list">
<li><strong>A P6 update is a forecast document.</strong> It was produced to tell the employer when the job would finish, not to record what happened. Actual dates are a by-product of that exercise, entered by a planner working to a monthly deadline.</li>

<li><strong>The last update has been edited the most.</strong> By the end of a long job it has usually been re-baselined, partially renumbered, had calendars swapped, had activities merged or deleted. Every one of those edits is invisible in a two-file comparison.</li>

<li><strong>Actual dates get back-filled.</strong> A date typed in month 21 describing month 19 is not contemporaneous evidence in the same sense as a date typed in month 19. It might still be right. It is a different quality of record, and the difference is exactly what gets probed in cross-examination.</li>
</ul>



<p class="wp-block-paragraph">AACE International&#8217;s Recommended Practice 29R-03 places the as-planned versus as-built family in its observational, static group: you compare a planned network against an as-built without inserting or removing delay events. The SCL Delay and Disruption Protocol, second edition, is blunter about what makes any of it work, pushing hard on agreeing a record-keeping regime up front and never overwriting a programme version. Both point at the same requirement. You need the whole sequence of updates, intact, and you need to know which one told you what.</p>



<p class="wp-block-paragraph">The platform&#8217;s whole job is to make that sequence queryable.</p>



<h2 class="wp-block-heading">A worked example: what thirty-four updates actually look like</h2>



<p class="wp-block-paragraph">Take a job shaped like this. Three years of monthly updates, call it thirty-four XER exports. Around four thousand activities in the current programme. Three baseline revisions, two agreed and one submitted but never accepted. Somewhere near month fourteen the contractor moved most trades from a five-day to a six-day calendar to recover. The numbers are illustrative rather than from any particular project, but the shape is ordinary.</p>



<p class="wp-block-paragraph">Run the naive comparison and you get a variance table. Run the sequence properly and four separate questions fall out, none of which the variance table can answer:</p>



<ol class="wp-block-list">
<li>Which activities existed in the baseline, disappeared mid-job, and came back under a different activity ID?</li>

<li>For each actual date in the final update, in which update did that value first appear, and how far behind the event was it?</li>

<li>When float moved, was it because work moved, or because someone changed a calendar or a duration?</li>

<li>What contemporaneous record sits underneath each actual date, and does it agree?</li>
</ol>



<p class="wp-block-paragraph">Those four questions are the platform. Everything below is how to answer each one without hand-checking four thousand rows thirty-four times.</p>



<h2 class="wp-block-heading">Failure family one: activity identity across versions</h2>



<p class="wp-block-paragraph">An XER file is tab-delimited text. It opens in a text editor. The structure is a header line beginning <code>ERMHDR</code>, then a run of tables, each introduced by <code>%T</code> with the table name, a <code>%F</code> line carrying the field names, and <code>%R</code> lines carrying rows, closed by <code>%E</code>. The tables you care about first are <code>PROJECT</code>, <code>TASK</code>, <code>TASKPRED</code> and <code>CALENDAR</code>.</p>



<pre class="wp-block-code"><code>ERMHDR	19.12	2026-01-15	Project	admin	Admin User	dbxDatabaseNoName	Project Management	USD
%T	TASK
%F	task_id	proj_id	wbs_id	clndr_id	task_code	task_name	status_code	act_start_date	act_end_date	target_start_date	target_end_date	total_float_hr_cnt
%R	1001	100	200	1	C-2140	Structural Steel Erection L4	TK_Complete	...
%E</code></pre>



<p class="wp-block-paragraph">Here is the trap. <code>task_id</code> is the primary key of the P6 database the file was exported from. It is not a stable identifier for an activity across time. If the project was ever copied between databases, restored, or exported from a different P6 instance, the same physical activity can carry a different <code>task_id</code>. Worse, two XERs from different sources can reuse the same <code>task_id</code> for completely unrelated activities.</p>



<p class="wp-block-paragraph"><code>task_code</code> is the Activity ID a human sees, and it is stable right up until someone renumbers. Neither field alone gives you identity. So do not pick one. Build an identity resolution layer and keep the evidence for every decision it makes:</p>



<ul class="wp-block-list">
<li>Use <code>(proj_id, task_code)</code> as the working join key, scoped to a single source database.</li>

<li>Store <code>task_id</code>, <code>task_name</code> and <code>wbs_id</code> on every snapshot row so a rename or a WBS move is detectable after the fact.</li>

<li>Emit an explicit <em>unresolved</em> record when an activity vanishes or appears mid-sequence. Do not guess.</li>
</ul>



<p class="wp-block-paragraph">That last point is the one people get wrong. When C-2140 disappears in update 22 and an activity with the same name but the ID C-2140A appears in update 23, there are three candidate explanations: deletion and replacement, renumbering, or a split. The platform cannot know which. A tool that silently picks one is worse than a tool that flags it, because the analyst never learns there was a question. Surface it, hand it to a human, record the decision and the reason.</p>



<h2 class="wp-block-heading">Failure family two: date provenance, and the table that makes the whole thing worth building</h2>



<p class="wp-block-paragraph">If you build one thing, build this. For every activity and every date field, record the first update in which that value appeared, and every subsequent change to it.</p>



<p class="wp-block-paragraph">Assume a snapshot table with one row per activity per update, appended and never modified. A window function does the work:</p>



<pre class="wp-block-code"><code>-- First appearance of each distinct actual finish value,
-- and how far the reported date lags the update that reported it.
WITH changes AS (
  SELECT
    task_code,
    version_no,
    data_date,
    act_end_date,
    LAG(act_end_date) OVER (
      PARTITION BY task_code ORDER BY version_no
    ) AS prev_act_end
  FROM activity_snapshot
)
SELECT
  task_code,
  version_no          AS first_reported_in_update,
  act_end_date        AS reported_actual_finish,
  data_date           AS update_data_date,
  data_date::date - act_end_date::date AS reporting_lag_days
FROM changes
WHERE act_end_date IS NOT NULL
  AND (prev_act_end IS NULL OR prev_act_end &lt;&gt; act_end_date)
ORDER BY reporting_lag_days DESC;</code></pre>



<p class="wp-block-paragraph">Three things arrive at once. You can answer the question that opened this post, for any activity, in a second: C-2140&#8217;s actual finish first appeared in update 21, carrying a date twenty-three days behind that update&#8217;s data date.</p>



<p class="wp-block-paragraph">The distribution of that lag column is then a finding in its own right. If most activities report within days and one subcontractor&#8217;s work consistently reports a month late, that is not a rounding artefact, it is a records issue you can evidence. And any actual date that changes after it was first stated deserves a hard look. Actuals are supposed to lock. A value that moves has either been corrected, which should have a paper trail, or overwritten, which should not have happened.</p>



<p class="wp-block-paragraph">P6 ships a Schedule Comparison tool, the one that used to be called Claim Digger and now lives inside Visualizer. It does a field-by-field diff and dumps a large HTML table, which is genuinely useful for checking one submitted update against the last. That is what it was built for. It was not built to reason across thirty-four files at once, and pushing its output into Excel does not change that. Deltek&#8217;s Acumen Fuse and Steelray&#8217;s Delay Analyzer go further, with what the vendors describe as half-step analysis, separating progress from scope revisions across successive versions. If your instruction allows the licence and the timetable allows the learning curve, look at them seriously before writing any code. Neither is priced publicly, both target enterprise project controls teams, and neither does the records-linking work in the last section for you.</p>



<h2 class="wp-block-heading">Failure family three: calendars, and float that quietly means nothing</h2>



<p class="wp-block-paragraph">Durations and float in an XER are stored in hours. <code>total_float_hr_cnt</code> holds hours, not days. A value of 40 is five days only if the activity runs an eight-hour calendar. Divide everything by eight and a job with mixed shift patterns produces float numbers that are confidently wrong.</p>



<p class="wp-block-paragraph">The calendar itself lives in the <code>CALENDAR</code> table, in a <code>clndr_data</code> field holding a proprietary parenthesised blob describing work weeks, exceptions and holidays. It is the hardest part of the format to parse correctly, and getting it wrong produces durations and dates that look plausible and are not.</p>



<p class="wp-block-paragraph">Two rules follow, and both are about the read path, not the maths.</p>



<ul class="wp-block-list">
<li>Store float and durations in hours exactly as exported. Convert at presentation time, using that activity&#8217;s own calendar, and print the calendar name next to the number. If you cannot resolve the calendar, print hours and say so.</li>

<li>Track calendar assignment as a versioned attribute, the same as any date. A change from a five-day to a six-day calendar changes every float figure downstream of it, with no change to logic and no change to progress. An activity can go from critical to non-critical because of an administrative edit, and a variance table will attribute that shift to the works.</li>
</ul>



<p class="wp-block-paragraph">This is also why an as-built critical path is not simply the chain with zero float in the last update. Retrospective longest path and as-planned versus as-built windows analysis exist as separate methods precisely because the float figures in a progressed programme are a function of how the programme was maintained.</p>



<h2 class="wp-block-heading">Failure family four: separating movement from editing</h2>



<p class="wp-block-paragraph">Every delta between two consecutive updates falls into one of a small number of categories, and collapsing them into a single &#8220;variance&#8221; number is where analyses lose their defensibility. Classify each change explicitly:</p>



<ul class="wp-block-list">
<li><strong>Progress:</strong> an actual start or finish appeared, or remaining duration reduced consistent with work done.</li>

<li><strong>Duration change:</strong> original or remaining duration edited without corresponding progress.</li>

<li><strong>Logic change:</strong> a relationship added, removed, retyped, or its lag altered.</li>

<li><strong>Calendar change:</strong> activity reassigned, or the calendar definition itself edited.</li>

<li><strong>Constraint change:</strong> a date constraint applied, moved or removed.</li>

<li><strong>Scope change:</strong> activity added or deleted.</li>
</ul>



<p class="wp-block-paragraph">Six buckets, one row per change, with the before value, the after value and the version pair. Once that table exists, a lot of the argument stops being an argument. &#8220;The completion date moved out eighteen days in window 12&#8221; becomes &#8220;of which fourteen days sit with progress on the critical chain and four with two logic edits applied in the same submission.&#8221; The second version is the one that gets tested and holds.</p>



<p class="wp-block-paragraph">Logic changes deserve their own view. Pull <code>TASKPRED</code> for every version, key each relationship on the predecessor and successor activity codes plus the relationship type, and diff the sets. Relationships that appear late in a job, on activities that are already in progress, are worth reading one by one.</p>



<h2 class="wp-block-heading">Failure family five: joining the schedule to the records</h2>



<p class="wp-block-paragraph">The schedule tells you when. It never tells you why. Cause comes from daily reports, site instructions, RFIs, correspondence, minutes, inspection records, weather data, and delivery notes. That material arrives as scanned PDFs and email exports, and it does not carry activity IDs. Nobody has ever written &#8220;C-2140&#8221; on a site diary.</p>



<p class="wp-block-paragraph">So the join key is not the activity ID. It is a fuzzy composite of date, location or area, and trade or discipline. The practical approach:</p>



<ol class="wp-block-list">
<li>Extract structured fields from each record: document date, report date, area, trade, headcount if present, and the narrative text. Text extraction plus a layout-aware OCR pass handles most of it. Handwriting on older site diaries will not fully automate, and you should budget for that.</li>

<li>Normalise area and trade against the schedule&#8217;s own coding. Activity codes and WBS paths are usually the best available crosswalk, since they encode area and discipline already.</li>

<li>Generate <em>candidate</em> links: for each activity&#8217;s as-built window, every record whose date falls inside it and whose area or trade matches.</li>

<li>Have a human confirm, reject or annotate each candidate. Store who decided and when.</li>
</ol>



<p class="wp-block-paragraph">Step four is not optional and it is not a nicety. A machine-proposed link is a search result. Causation is an opinion, and the expert has to own it. Build the tool so it accelerates the search and refuses to state the conclusion, and you get the productivity without handing opposing counsel an argument about black boxes.</p>



<h2 class="wp-block-heading">The data model, in about six tables</h2>



<p class="wp-block-paragraph">The grain that makes everything else easy: one immutable row per activity per schedule version. Nothing is ever updated in place.</p>



<pre class="wp-block-code"><code>CREATE TABLE schedule_version (
  version_id     bigserial PRIMARY KEY,
  version_no     integer     NOT NULL,   -- ordinal in the sequence
  source_file    text        NOT NULL,
  file_sha256    char(64)    NOT NULL,   -- proves the file was not altered
  p6_export_ts   timestamptz,            -- from the ERMHDR line
  data_date      timestamptz NOT NULL,
  ingested_at    timestamptz NOT NULL DEFAULT now(),
  notes          text
);

CREATE TABLE activity_snapshot (
  version_id           bigint  NOT NULL REFERENCES schedule_version,
  task_code            text    NOT NULL,
  task_id              bigint  NOT NULL,
  task_name            text,
  wbs_path             text,
  clndr_id             bigint,
  status_code          text,
  target_start_date    timestamptz,
  target_end_date      timestamptz,
  act_start_date       timestamptz,
  act_end_date         timestamptz,
  total_float_hr_cnt   numeric,
  PRIMARY KEY (version_id, task_code)
);</code></pre>



<p class="wp-block-paragraph">Add <code>relationship_snapshot</code> keyed on <code>(version_id, pred_task_code, succ_task_code, rel_type)</code>, a <code>calendar_snapshot</code> holding the raw <code>clndr_data</code> alongside your parsed working pattern, a <code>record_document</code> table for the contemporaneous material, and a <code>record_link</code> table carrying the human decision on each candidate link.</p>



<p class="wp-block-paragraph">Thirty-four updates at four thousand activities is under a hundred and fifty thousand rows. This is not a big-data problem and it should not be built like one. PostgreSQL on a single machine, with the parsing done in Python, will run every query in this post fast enough that you stop thinking about it.</p>



<p class="wp-block-paragraph">On the parsing side, an XER is simple enough to read with the standard library, and there are open-source parsers such as PyP6Xer if you would rather not write the tokeniser yourself. Whichever route you take, keep the raw file. Hash it on ingest, store the hash in <code>schedule_version</code>, and never write to the original. Chain of custody over the source files is the cheapest credibility you will ever buy.</p>



<h2 class="wp-block-heading">Where to run it, and the confidentiality problem nobody budgets for</h2>



<p class="wp-block-paragraph">A modest VPS is plenty. Contabo and InterServer both sell machines with more than enough memory and disk for a job this size, and the specification matters far less than the question you should ask first: does your confidentiality undertaking actually permit the disclosure material to sit on that machine, in that jurisdiction? Often the answer is no, and the analysis has to run on a workstation inside the client&#8217;s own environment. Find that out before you build anything.</p>



<p class="wp-block-paragraph">Three practical points that follow from the material being privileged rather than ordinary project data:</p>



<ul class="wp-block-list">
<li><strong>Full-disk encryption on anything that holds the files</strong>, including the laptop you take to the hearing. This is table stakes and it is still routinely skipped.</li>

<li><strong>A VPN for untrusted networks.</strong> You will open the data room from a hotel or an airport at some point. NordVPN and Surfshark both handle that case. It protects the transport; it does nothing about where the data comes to rest, so do not let it substitute for the jurisdiction question above.</li>

<li><strong>A disposal plan.</strong> Undertakings usually require destruction or return at the close of the engagement, and deleting a file does not destroy it. Overwrite-based tools such as O&amp;O SafeErase cover the workstation case. If the volume was encrypted from the start, destroying the key is faster and cleaner than overwriting the data, and easier to certify.</li>
</ul>



<p class="wp-block-paragraph">If cloud storage is permitted, write-once object storage with a retention lock suits the ingested originals well. It gives you an immutability story that does not rest on your own discipline.</p>



<h2 class="wp-block-heading">Troubleshooting the ingest</h2>



<ul class="wp-block-list">
<li><strong>The parser dies on a decode error.</strong> XER files are not reliably UTF-8, particularly when they came from a database with a non-English locale. Try a single-byte fallback encoding before assuming the file is corrupt, and log which encoding succeeded against the version record.</li>

<li><strong>Row and field counts do not match.</strong> Activity names can contain characters that confuse a naive split, and some exports wrap fields inconsistently. Parse defensively on field count per <code>%F</code> line rather than assuming.</li>

<li><strong>Duplicate <code>task_code</code> in one file.</strong> One XER can carry several projects plus the shared global data they depend on. Always scope by <code>proj_id</code> and confirm you have the right project before anything else.</li>

<li><strong>Activity counts drop between updates for no reason.</strong> Check whether the export was filtered or taken at a WBS level rather than the full project. A partial export looks exactly like a deletion event in your diff.</li>

<li><strong>Dates shift by a few hours.</strong> P6 stores times, not just dates, and calendar start hours vary. Normalise to the activity calendar&#8217;s working day before comparing, or every finish looks a shift early.</li>

<li><strong>Float figures look wrong across the board.</strong> Check the hours-versus-days conversion and the calendar assignment before checking anything else. It is almost always one of those two.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating the final update as the as-built without stating that assumption anywhere in the report.</li>

<li>Joining on <code>task_id</code> because it is the primary key, and inheriting whatever database history came with the file.</li>

<li>Letting the platform resolve an ambiguous activity match silently, so the analyst never sees the ambiguity existed.</li>

<li>Converting float to days with a fixed divisor, then reporting a single variance figure that mixes progress with programme edits.</li>

<li>Automating the record-to-activity link all the way to a stated cause, which turns an evidence tool into a target.</li>

<li>Modifying source XERs in place, then having no way to demonstrate they are as received.</li>
</ul>



<h2 class="wp-block-heading">Best practices worth the effort</h2>



<ul class="wp-block-list">
<li>Hash every source file on ingest and record the hash next to the version. It takes one line and it answers a question you will otherwise answer badly.</li>

<li>Make every table append-only. Corrections become new rows with a reason, never overwrites.</li>

<li>Build the first-appearance view early. It reframes the analysis from &#8220;what do the files say&#8221; to &#8220;when did the files start saying it&#8221;, which is the more defensible question.</li>

<li>Make every exhibit reproducible from one query, so a challenged figure traces back to a file and a row rather than a spreadsheet cell.</li>

<li>Keep a written record of every judgement call the platform surfaced and a human resolved. That log is often more useful in the hearing than the analysis itself.</li>

<li>Validate against the commercial tools where you can. If Acumen Fuse or Schedule Comparison disagrees with your diff on a given version pair, one of you is wrong and you want to know which before the report goes out.</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Is as-planned vs as-built analysis still accepted in construction disputes?</h3>



<p class="wp-block-paragraph">Yes, as one of a recognised family of methods. AACE Recommended Practice 29R-03 includes it in its taxonomy as an observational method, and the SCL Protocol&#8217;s second edition deliberately stopped naming a single preferred technique for retrospective analysis, setting out selection factors instead. Acceptance in a given dispute turns on the records available, the contract and the forum, not on the method&#8217;s reputation in the abstract.</p>



<h3 class="wp-block-heading">How many schedule updates do I need before building a platform is worth it?</h3>



<p class="wp-block-paragraph">Roughly: below about eight updates, a careful analyst with a workbook is faster. Above twenty, manual comparison stops being reliable, not because people are careless but because the number of pairwise comparisons grows past what anyone can hold. The crossover in practice sits somewhere in the low teens, and it moves earlier if there are multiple baselines or a renumbering event.</p>



<h3 class="wp-block-heading">Can I do as-planned vs as-built analysis without a Primavera P6 licence?</h3>



<p class="wp-block-paragraph">For the data work, yes. An XER is plain text and parses without P6 installed, and browser-based and open-source viewers exist. You will still want licensed access somewhere in the engagement for anything that requires recalculating the network, because reproducing P6&#8217;s scheduling engine faithfully is not a project you want inside a disputes timetable.</p>



<h3 class="wp-block-heading">What is the difference between as-planned vs as-built and a windows analysis?</h3>



<p class="wp-block-paragraph">The simple form of as-planned versus as-built compares planned dates against actual dates observationally, without necessarily using network logic. A windows analysis divides the project into periods, usually monthly, and examines the contemporaneous critical path and the critical delay in each period before investigating the cause. Windows demands more of the records and generally carries more weight where the programme was maintained properly. This platform supports both, because both need the same version history underneath.</p>



<h3 class="wp-block-heading">What do I do when the contractor&#8217;s updates were not maintained properly?</h3>



<p class="wp-block-paragraph">State it, quantify it, and let it drive method selection. That is exactly what the first-appearance table is for: reporting lag, actuals that moved after being stated, out-of-sequence progress and unexplained logic edits are all measurable. Recommended practice treats recreating a schedule from records as a last resort, used when contemporaneous data is unavailable, and recreated schedules carry less evidential weight. Showing the deficiency with numbers is stronger than asserting it.</p>



<h3 class="wp-block-heading">Should I build this or buy commercial delay analysis software?</h3>



<p class="wp-block-paragraph">Buy for schedule diagnostics and version comparison. That problem is well solved and the established tools have years of edge cases behind them. Build for the parts specific to your instruction: the record linking, the exhibit generation in your house format, and the audit trail. The hybrid is usually right, and the deciding factor is more often the licence and the timescale than the technical fit.</p>



<h3 class="wp-block-heading">Does the platform need to reproduce P6&#8217;s critical path calculation?</h3>



<p class="wp-block-paragraph">No, and it should not try. Read the float and date values P6 already calculated and stored in each export, and treat them as evidence of what the programme said at that moment. Anywhere you genuinely need a recalculated network, do it in the scheduling tool so the result is reproducible by anyone with the same file.</p>



<h2 class="wp-block-heading">The one thing to take away</h2>



<p class="wp-block-paragraph">A credible as-planned vs as-built analysis is not a better comparison between two files. It is a complete, immutable, queryable history of what every schedule update said and when it started saying it. Build the version history and the first-appearance table first, and the exhibits fall out of it. Build the exhibits first and you will spend the hearing defending a spreadsheet whose provenance you cannot reconstruct.</p>



<p class="wp-block-paragraph">The scheduling judgement stays with the expert. The platform&#8217;s job is narrower and duller: make sure that when someone asks where a date came from, the answer takes one query and not one week.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need the data side of this built?</h2>



<p class="wp-block-paragraph">I build the engineering layer underneath delay and quantum work, so the expert spends their time on opinion rather than on data wrangling. That usually looks like:</p>



<ul class="wp-block-list">
<li>Ingesting a full sequence of P6 XER or XML updates into a versioned, append-only PostgreSQL model with hashing and chain of custody on every source file.</li>

<li>Building the first-appearance and reporting-lag views that show when each actual date entered the record, and flagging actuals that moved after being stated.</li>

<li>Classifying every between-update change into progress, duration, logic, calendar, constraint or scope, with before and after values on each row.</li>

<li>Extracting structured fields from daily reports, RFIs and correspondence with OCR and document processing, then generating candidate activity links for an analyst to confirm or reject.</li>

<li>Producing exhibit-grade output in Power BI or as static deliverables, where every figure traces back to a query, a file and a row.</li>

<li>Standing it up on infrastructure that fits the confidentiality regime you work under, whether that is a locked-down VPS or a machine inside the client&#8217;s own environment.</li>
</ul>



<p class="wp-block-paragraph">If you have a stack of XERs and a deadline, send me two consecutive updates and whatever the record-keeping looked like, and I will tell you what the sequence can and cannot support.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/as-planned-vs-as-built-analysis-platform/">As-Planned vs As-Built Analysis: Building a Platform That Survives Cross-Examination</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Designing a Construction Dispute Intelligence Platform That Survives Cross-Examination</title>
		<link>https://john-nessime.com/blog/technical-guides/construction-dispute-intelligence-platform/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 13 Aug 2026 20:19:52 +0000</pubDate>
				<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[As-Built Schedule]]></category>
		<category><![CDATA[Audit Logging]]></category>
		<category><![CDATA[Bates Numbering]]></category>
		<category><![CDATA[Capital Projects]]></category>
		<category><![CDATA[Chain of Custody]]></category>
		<category><![CDATA[Common Data Environment]]></category>
		<category><![CDATA[Data Lineage]]></category>
		<category><![CDATA[Delay Analysis]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Embeddings]]></category>
		<category><![CDATA[Extension of Time]]></category>
		<category><![CDATA[Forensic Schedule Analysis]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Hybrid Search]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[Litigation Hold]]></category>
		<category><![CDATA[Load Files]]></category>
		<category><![CDATA[Metadata Filtering]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[Oracle Aconex]]></category>
		<category><![CDATA[pgvector]]></category>
		<category><![CDATA[PostgreSQL]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[Project Controls]]></category>
		<category><![CDATA[Provenance]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[Reciprocal Rank Fusion]]></category>
		<category><![CDATA[S3 Object Lock]]></category>
		<category><![CDATA[Schema Design]]></category>
		<category><![CDATA[System of Record]]></category>
		<category><![CDATA[Tenant Isolation]]></category>
		<category><![CDATA[Vector Database]]></category>
		<category><![CDATA[XER Files]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=336</guid>

					<description><![CDATA[<p>A layer-by-layer architecture for a construction dispute intelligence platform: hashed intake and immutable originals, OCR with page coordinates, a versioned Primavera P6 schedule plane, an event model that joins documents to activities, hybrid retrieval on pgvector, and a generation layer bounded by human confirmation. Written around what each layer prevents.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/construction-dispute-intelligence-platform/">Designing a Construction Dispute Intelligence Platform That Survives Cross-Examination</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The question that kills these systems is always the same one, and it always comes from the expert, not the engineer: &#8220;Where did this date come from?&#8221;</p>



<p class="wp-block-paragraph">You have a screen showing a beautiful chronology. Two hundred events, colour-coded by discipline, cross-referenced against the programme. The delay expert points at row 47, the one that says the piling rig arrived late, and asks which document that came from, who produced it, what page, and whether the other side has seen it. If the answer involves anyone opening a folder and searching, the platform has already failed. Not technically. Evidentially. A chronology nobody can trace back to a produced document is a nice picture, and a nice picture is worth nothing in an adjudication.</p>



<p class="wp-block-paragraph">That is the constraint that shapes everything else. A construction dispute intelligence platform is not a search engine with a construction theme on it. It is a provenance system that happens to do search. Get that ordering wrong and you build something impressive that no expert will sign their name to.</p>



<p class="wp-block-paragraph">This post walks through the architecture layer by layer: intake and chain of custody, normalization, the schedule data plane, the chronology model that joins the two, retrieval, and the tightly bounded generation layer on top. For each one I&#8217;ll say what it prevents, because that is the only reason any of these layers exist.</p>



<h2 class="wp-block-heading">What a construction dispute intelligence platform actually has to do</h2>



<p class="wp-block-paragraph">Strip away the vocabulary and the job is narrow. Take a pile of heterogeneous records from a capital project, some of them structured and some of them scanned faxes, and let a small number of expensive humans answer three questions quickly and defensibly:</p>



<ul class="wp-block-list">
<li>What happened, in what order, and who said so at the time?</li>



<li>What did the programme say before and after each of those things happened?</li>



<li>Which contemporaneous record supports every single assertion in the report?</li>
</ul>



<p class="wp-block-paragraph">The third one is not a nice-to-have. Both of the industry references people actually argue from lean on it. The Society of Construction Law Delay and Disruption Protocol raised record keeping to a core principle in its second edition and stopped expressing a preference between retrospective delay analysis methods, on the reasoning that the records you have largely determine the method available to you. AACE International&#8217;s Recommended Practice 29R-03 codifies forensic schedule analysis into a taxonomy of Method Implementation Protocols, and its minimum implementation protocols are essentially statements about which source schedules you must be able to produce. Both documents are, underneath, about evidence handling. So is your architecture.</p>



<h2 class="wp-block-heading">Layer one: intake that a forensic expert can defend</h2>



<p class="wp-block-paragraph">Everything upstream of this layer is somebody else&#8217;s mess. Site photos on a phone, a shared drive nobody pruned since mobilization, an Aconex export, a hard drive that arrived in an envelope. Your job at intake is to make an irreversible statement about what you received and when, before anyone touches it.</p>



<p class="wp-block-paragraph">Hash every file on arrival and store the hash next to the object, not in a spreadsheet. SHA-256 over the raw bytes, computed before any conversion, deduplication or renaming. The hash becomes the document&#8217;s identity for the rest of its life.</p>



<pre class="wp-block-code"><code># Build an arrival manifest before anything else runs
find ./ACME-INTAKE-001 -type f -print0 
  | xargs -0 sha256sum &gt; manifest-ACME-INTAKE-001.txt

# Re-verify later, after the files have moved through processing
sha256sum -c manifest-ACME-INTAKE-001.txt</code></pre>



<p class="wp-block-paragraph">Then put the originals somewhere they cannot be quietly edited. On AWS, S3 Object Lock in compliance mode with a retention period is the blunt instrument: once set, the object version cannot be overwritten or deleted by anyone, including the root account, until retention expires. Governance mode is the softer variant, where a specifically permissioned principal can lift the lock, which is fine for internal working copies and wrong for originals. If you are self-hosting on a Contabo or InterServer box because the client will not put project data in a hyperscaler, the equivalent discipline is a write-once staging volume plus an append-only manifest log, and you accept that you are relying on process rather than on the storage layer refusing the write.</p>



<h3 class="wp-block-heading">The manifest is the product</h3>



<p class="wp-block-paragraph">If the matter is already in formal disclosure, you will be handed productions rather than raw folders, and those arrive as load files. A Concordance DAT carries one row per document with fields like BegBates, EndBates, Custodian, BegAttach, EndAttach and an MD5 hash. An Opticon OPT carries one row per page, mapping Bates numbers to image paths. The delimiters are deliberately weird characters that real text rarely contains, and different tools disagree about the exact code points, so parse defensively and log what you detected rather than assuming.</p>



<p class="wp-block-paragraph">The failure mode here is quiet and expensive. Variable-width Bates numbers sort lexically instead of numerically, so ABC10 lands between ABC1 and ABC2, attachment ranges resolve to the wrong parents, and a family of documents silently splits. Nobody notices until an expert cites a covering email whose attachment, in your system, belongs to a different letter. Pad to fixed width on ingest and reconcile every BegAttach range against the documents you actually loaded.</p>



<h2 class="wp-block-heading">Layer two: normalization, and the OCR tax nobody budgets for</h2>



<p class="wp-block-paragraph">Construction corpora are the worst kind of mixed. Native Office files with clean text, born-digital PDFs, scanned site instructions with a signature and a coffee ring, photographs of whiteboards, and drawings where the only text that matters is inside a title block in the bottom right corner.</p>



<p class="wp-block-paragraph">Route by what the file actually is, not by extension. For a PDF, try text extraction first; if the extracted layer is empty or nearly empty, it is a scan and needs OCR. Tesseract via OCRmyPDF is the cheap path and it is genuinely good on clean typed pages. It degrades badly on handwriting, on stamps, and on anything skewed, which describes most site paperwork. A managed service such as Amazon Textract handles tables and forms better and returns geometry for every detected block, which matters more than accuracy alone.</p>



<p class="wp-block-paragraph">Geometry is the part teams skip and then rebuild six months later. If you store only the extracted string, you can prove a phrase exists somewhere in a document. If you store the page number and the bounding box, you can put a highlight on the exact line in the viewer, and the expert can confirm it in two seconds instead of reading eleven pages. That difference is the difference between a tool people use and a tool people abandon.</p>



<p class="wp-block-paragraph">Keep the OCR text separate from the original in storage, and record which engine and which settings produced it. When a disputed reading of a handwritten date turns up, and it will, you want to be able to say exactly how that character was produced.</p>



<h2 class="wp-block-heading">Layer three: the schedule plane is a separate data model</h2>



<p class="wp-block-paragraph">This is where most teams underestimate the work. Programme data is not documents. It is a graph, versioned, with a calendar attached, and it needs its own tables, its own ingest path and its own validation.</p>



<p class="wp-block-paragraph">Primavera P6 exports as XER, a tab-delimited text format with a table-per-section layout, or as P6 XML. Either way you are pulling out projects, the WBS tree, activities, logic links and calendars. In XER these live in sections including PROJECT, PROJWBS, TASK, TASKPRED and CALENDAR. Parsing it by hand is a bad use of a week; PyP6Xer is an open source Python parser that will get you to objects quickly.</p>



<pre class="wp-block-code"><code>from xerparser.reader import Reader

xer = Reader("update-14.xer")

for project in xer.projects:
    print(project.proj_short_name)
    for activity in project.activities:
        print(activity.task_code, activity.task_name)</code></pre>



<p class="wp-block-paragraph">Two things bite immediately. First, several duration and float fields in XER are stored in hours rather than days, which you can see in the column names themselves. If you load them straight into a &#8220;days&#8221; column because the numbers looked plausible on a five-day calendar, every float figure downstream is wrong by a factor that varies with the calendar. Check your own export before you trust any mapping, including this one.</p>



<p class="wp-block-paragraph">Second, activity IDs are not stable across a project&#8217;s life. Activities get renumbered, split, deleted and re-added between updates. If your schema keys on the activity ID alone, two updates will silently merge activities that were never the same work. Key on a composite of schedule version plus activity ID, and maintain a separate mapping table for identity across versions that a human can correct. That table is boring, it is not automatable, and it is the difference between a windows analysis that holds together and one that quietly compares apples to a different orchard.</p>



<p class="wp-block-paragraph">Store every update as an immutable snapshot with its data date. Never mutate a loaded schedule. The whole point of the layer is that you can reconstruct exactly what the programme said on a given day, which is precisely what a time slice or windows method needs.</p>



<h2 class="wp-block-heading">Layer four: the chronology is the join, and the join is the hard part</h2>



<p class="wp-block-paragraph">Documents on one side, schedule snapshots on the other. The product is the thing in the middle: an event, with a date, a description, a set of supporting document references, and a set of affected activities.</p>



<p class="wp-block-paragraph">Model it explicitly. An event is not a document and it is not an activity. One letter can create three events. One event can be evidenced by nine documents and a photograph. The relationship is many-to-many in both directions and every attempt I have seen to flatten it ends with someone maintaining a spreadsheet on the side.</p>



<pre class="wp-block-code"><code>-- Every assertion resolves to a page and a box on that page
CREATE TABLE evidence_ref (
  id            BIGSERIAL PRIMARY KEY,
  event_id      BIGINT NOT NULL REFERENCES event(id),
  document_id   UUID   NOT NULL REFERENCES document(id),
  page_number   INT    NOT NULL,
  bbox          JSONB,             -- normalized coordinates from OCR
  quoted_text   TEXT   NOT NULL,
  asserted_by   TEXT   NOT NULL,   -- 'analyst' or 'model'
  confirmed_by  TEXT,              -- NULL until a human confirms
  confirmed_at  TIMESTAMPTZ
);</code></pre>



<p class="wp-block-paragraph">Those last three columns are the ones that matter. They encode the difference between something the machine proposed and something a person stands behind. Report generation reads only rows where <code>confirmed_by</code> is set. Everything else is a suggestion queue. This is not a compliance box; it is the mechanism by which an expert can put their name on the output, and it needs to exist in the schema rather than in a workflow convention that erodes under deadline.</p>



<h2 class="wp-block-heading">Layer five: retrieval that finds document numbers, not just meaning</h2>



<p class="wp-block-paragraph">Pure vector search is the wrong default for this corpus, and it fails in a way that looks like success. Embeddings are good at meaning and bad at tokens. Ask for &#8220;RFI 0442&#8221; and a dense retriever will cheerfully hand you eleven chunks about requests for information, none of which is RFI 0442, because the identifier got smeared into its semantic neighbourhood. Construction disputes run on identifiers: RFI numbers, variation order references, drawing revisions, activity IDs, Bates ranges.</p>



<p class="wp-block-paragraph">Run lexical and semantic retrieval in parallel and fuse the rankings. PostgreSQL with the pgvector extension does both without adding infrastructure: an HNSW index over the embedding column, a GIN index over a generated tsvector column, and reciprocal rank fusion to merge the two candidate lists.</p>



<pre class="wp-block-code"><code>CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunk (
  id          BIGSERIAL PRIMARY KEY,
  document_id UUID NOT NULL,
  page_number INT  NOT NULL,
  body        TEXT NOT NULL,
  body_tsv    tsvector GENERATED ALWAYS AS
                (to_tsvector('english', body)) STORED,
  embedding   vector(1536)
);

CREATE INDEX chunk_tsv_idx  ON chunk USING gin (body_tsv);
CREATE INDEX chunk_hnsw_idx ON chunk USING hnsw (embedding vector_cosine_ops);</code></pre>



<p class="wp-block-paragraph">Over-fetch from each side and trim after fusion rather than pulling the final count from each; more candidates give the fusion step something to work with. Add a trigram index if your corpus has OCR noise, because &#8220;RFI 0442&#8221; and &#8220;RFl 0442&#8221; are the same document to a human and different tokens to Postgres.</p>



<p class="wp-block-paragraph">Filter hard on metadata before ranking. Matter, custodian, date range, privilege status. A retrieval layer that can return a privileged document into a shared workspace is a liability, not a feature.</p>



<h2 class="wp-block-heading">Layer six: generation, bounded hard</h2>



<p class="wp-block-paragraph">Language models are useful here and the useful jobs are smaller than the demos suggest: proposing candidate events from a batch of correspondence, drafting neutral one-line summaries, clustering near-duplicate records, suggesting which activities an event might touch. Every one of those is a proposal that lands in a review queue.</p>



<p class="wp-block-paragraph">The reason to be strict is not squeamishness. Stanford&#8217;s RegLab and HAI ran a preregistered evaluation of commercial legal research tools built on retrieval-augmented generation, with expert-scored answers, and found hallucination rates in the range of roughly 17 to 33 percent depending on the product, against vendor marketing that had claimed hallucination-free citations. The vendors disputed the characterization. The finding that should worry you more than the headline number is the failure mode underneath it: a citation that exists, is quoted accurately, and does not stand for what the answer says it stands for. Retrieval reduces fabrication. It does not produce correct reasoning about what a document means, and misgrounding is far harder to spot than an invented reference.</p>



<p class="wp-block-paragraph">So bound it. Generation writes to the proposal table, never to the report. Every generated assertion carries the chunk IDs it was built from, and the UI shows the source page beside the claim by default rather than behind a click. If a proposed event has no evidence reference, it does not enter the queue at all.</p>



<h2 class="wp-block-heading">Tenancy, privilege, and the thing that ends the engagement</h2>



<p class="wp-block-paragraph">Matters do not share data. Ever. Not &#8220;logically separated by a WHERE clause that we always remember to add&#8221;. The cheapest defensible pattern is a database per matter, or at minimum a schema per matter with connection-level scoping, so that a missing predicate produces an empty result rather than another client&#8217;s correspondence.</p>



<p class="wp-block-paragraph">Two more that get discovered late. Privilege state is per document per matter and it changes; model it as a versioned attribute with an audit trail, not a boolean someone flipped. And plan the exit at the start: matters end, retention clocks expire, and someone has to certify destruction. That means knowing every copy you made, including OCR text, thumbnails, embeddings and backups. Tools like the O&amp;O Software utilities have a place at the endpoint end of that job; the platform side is a deletion manifest that enumerates derived artifacts per document, written at ingest rather than reconstructed under pressure.</p>



<p class="wp-block-paragraph">If experts are working from home, from site cabins, or from three countries, the access path is part of the design. A managed VPN such as NordVPN or Surfshark on the client side plus network-level allowlisting on the platform side is a pragmatic combination, and it is worth writing into the engagement documentation so nobody improvises with a personal Dropbox at eleven at night.</p>



<h2 class="wp-block-heading">Where these platforms break in practice</h2>



<ul class="wp-block-list">
<li><strong>Chronology dates drift from document dates.</strong> Usually a timezone or a date-format collision: a scanned form read as DD/MM, an email header parsed as UTC, a schedule data date stored as a naive timestamp. Store the raw string you extracted alongside the parsed value and show both in the review UI.</li>



<li><strong>Float figures look wrong by a constant factor.</strong> Hours versus days in the schedule import. Check against the same activity opened in P6 before you debug anything else.</li>



<li><strong>Retrieval misses a document everyone knows exists.</strong> Either it was never OCR&#8217;d because the PDF had a thin text layer that passed your emptiness check, or the identifier is only in the title block of a drawing. Log extraction character counts per page and alert on suspiciously low ones.</li>



<li><strong>Two activities merged across updates.</strong> Composite keying was not applied, or an activity was renumbered. Diff activity code sets between consecutive snapshots and surface additions and deletions as a report the planner reviews.</li>



<li><strong>Document families split.</strong> Bates padding or an unreconciled attachment range from the load file. Re-run the family reconciliation and count orphaned attachments.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating the schedule as documents. It is a graph with versions, and shoehorning it into the document store costs you every analysis you actually wanted.</li>



<li>Storing extracted text without page and coordinate provenance. Cheap now, a full reprocessing run later.</li>



<li>Vector-only retrieval. Looks great in a demo built from prose, falls over on the first RFI number.</li>



<li>Letting model output reach a deliverable without a recorded human confirmation.</li>



<li>Mutating loaded schedules to &#8220;fix&#8221; them. Fix on the way in, or hold the correction as an annotation. Never edit the snapshot.</li>



<li>Deferring the deletion story to the end of the matter, when the derived artifacts are already scattered.</li>
</ul>



<h2 class="wp-block-heading">Best practices for building a construction dispute intelligence platform</h2>



<ul class="wp-block-list">
<li>Hash at the door, lock the originals, and never process in place.</li>



<li>Make every derived artifact point back to a document ID, a page and a region.</li>



<li>Give schedules their own immutable, version-keyed store with the data date as a first-class field.</li>



<li>Model events separately from both documents and activities, with many-to-many links in each direction.</li>



<li>Run hybrid retrieval and fuse the rankings; filter on matter and privilege before you rank.</li>



<li>Keep generation in a proposal queue with confirmation recorded in the schema.</li>



<li>Isolate matters at the database or schema boundary, not in application code.</li>



<li>Instrument ingest as seriously as you would a production pipeline. Grafana over your extraction counts and OCR queue depth will tell you a corpus is going wrong days before an analyst does.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Do I need a dedicated vector database for this?</h3>



<p class="wp-block-paragraph">Almost certainly not. Matter corpora are large in gigabytes and small in chunk count compared to consumer-scale search. PostgreSQL with pgvector keeps the embeddings in the same transaction as the provenance rows, which means you cannot end up with a chunk whose document record was rolled back. That consistency is worth more here than the last few milliseconds of query latency.</p>



<h3 class="wp-block-heading">Should the platform pick a delay analysis method?</h3>



<p class="wp-block-paragraph">No. That choice belongs to the expert and depends on the contract, the forum, the records available and the timing of the analysis. What the platform should do is make it obvious which methods the record set can support, by reporting what you actually have: how many schedule updates, at what intervals, with what data dates, and how complete the contemporaneous correspondence is around each window.</p>



<h3 class="wp-block-heading">XER or P6 XML for schedule ingest?</h3>



<p class="wp-block-paragraph">Accept both, because you will be sent both. XER is what planners export by habit and it is easy to parse with existing libraries. P6 XML is more explicit about structure and less prone to encoding surprises. Whichever arrives, keep the original file, hashed, alongside the parsed rows.</p>



<h3 class="wp-block-heading">How much of the review can realistically be automated?</h3>



<p class="wp-block-paragraph">The ranking and the grouping, most of it. The judgement, none of it. Automation that reduces a hundred thousand documents to a few thousand worth reading is enormously valuable and uncontroversial. Automation that decides what a letter means is where you inherit risk you cannot price.</p>



<h3 class="wp-block-heading">Can I build this on top of a common data environment we already have?</h3>



<p class="wp-block-paragraph">Partly. A CDE such as Aconex is a good source of records and a poor system of record for a dispute, because access changes, exports are point-in-time, and the retention rules belong to the project rather than to the matter. Pull from it, hash what you pulled, and hold your own copy under your own lock.</p>



<h3 class="wp-block-heading">What does this cost to run?</h3>



<p class="wp-block-paragraph">The shape of the bill is more useful than a number. OCR and embedding are one-time per document and scale with page count, so a scan-heavy corpus costs multiples of a born-digital one. Storage is small and constant. Retrieval is negligible. Generation scales with how much you push through it, which is another argument for narrow, queued use rather than a chat box over the whole corpus.</p>



<h3 class="wp-block-heading">Is a general eDiscovery platform not enough?</h3>



<p class="wp-block-paragraph">For document review, often yes, and they are considerably better at productions and privilege logging than anything you will build in a quarter. What they do not have is the schedule plane. The value of a construction-specific build is the join between correspondence and programme versions, so a sensible architecture frequently uses an established review platform for documents and adds the schedule and chronology layers alongside it.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">A construction dispute intelligence platform earns its place by making provenance cheap. Search, clustering and summarization are commodity capabilities now; anyone can bolt them onto a corpus in a fortnight. What is hard, and what decides whether an expert will rely on the output, is that every date, every event and every sentence in the eventual report resolves in one click to a specific page of a specific produced document, with a record of who confirmed it.</p>



<p class="wp-block-paragraph">Build the provenance spine first. Hash at intake, keep coordinates, version the schedules, model events as their own thing, and keep the model on the proposal side of a human confirmation. The clever features are easy to add afterwards. Retrofitting evidence handling into a system that was designed as a search tool is a rebuild, and it always arrives at the worst possible moment, which is about three weeks before a hearing.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help building or fixing this?</h2>



<p class="wp-block-paragraph">I work on the infrastructure side of exactly this problem: getting messy project records into a system where every claim is traceable. Things I can help with:</p>



<ul class="wp-block-list">
<li>Designing the intake and chain-of-custody layer, including hashing, immutable storage, and load file parsing that survives odd delimiters and inconsistent Bates padding.</li>



<li>Building the schedule ingest path for XER and P6 XML, with version-keyed snapshots and an activity identity map that holds across updates.</li>



<li>Standing up hybrid retrieval on PostgreSQL and pgvector, including page-level and coordinate-level provenance so highlights land on the right line.</li>



<li>Setting up the OCR and normalization pipeline, with routing rules, quality checks, and alerting on pages that extracted suspiciously little text.</li>



<li>Matter isolation, privilege state modelling, audit logging, and a deletion manifest you can act on at the end of the engagement.</li>



<li>Reviewing an existing platform and telling you plainly where the provenance chain breaks.</li>
</ul>



<p class="wp-block-paragraph">If you have something concrete, send it over: an XER export, a sample DAT and OPT pair, a page count breakdown of your corpus, or the schema you are currently working from. It is much easier to say something useful about a real file than about a description of one.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/construction-dispute-intelligence-platform/">Designing a Construction Dispute Intelligence Platform That Survives Cross-Examination</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Building an AI Construction Claims Platform on AWS That Holds Up Under Scrutiny</title>
		<link>https://john-nessime.com/blog/case-studies/ai-construction-claims-platform-aws/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Mon, 03 Aug 2026 14:05:46 +0000</pubDate>
				<category><![CDATA[Case Studies]]></category>
		<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Athena]]></category>
		<category><![CDATA[Amazon Bedrock]]></category>
		<category><![CDATA[Amazon S3]]></category>
		<category><![CDATA[Amazon S3 Vectors]]></category>
		<category><![CDATA[Amazon Textract]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS Glue]]></category>
		<category><![CDATA[Bedrock Guardrails]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Embeddings]]></category>
		<category><![CDATA[Generative AI]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[Metadata Filtering]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[Serverless]]></category>
		<category><![CDATA[SQL]]></category>
		<category><![CDATA[Vector Database]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=130</guid>

					<description><![CDATA[<p>Semantic search finds the most persuasive document, not the earliest one. Here is how to architect an AI construction claims and dispute intelligence platform on AWS so retrieval respects the contractual clock, schedule data stays out of the vector index, every answer resolves to a page, and privileged material never shares a retrieval path with project records.</p>
<p>The post <a href="https://john-nessime.com/blog/case-studies/ai-construction-claims-platform-aws/">Building an AI Construction Claims Platform on AWS That Holds Up Under Scrutiny</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Someone hands you a shared drive and asks a question that sounds trivial: &#8220;Did we give notice of the delay event inside the contractual period, or didn&#8217;t we?&#8221;</p>



<p class="wp-block-paragraph">The answer is in there. It is one email, or one line in a site diary, sitting among forty thousand other files. Nobody can read forty thousand files, so the instinct is to point a language model at the pile and ask it. That instinct is right. The naive implementation of it is where the money goes.</p>



<p class="wp-block-paragraph">Here is the failure mode that bites hardest, and it is invisible until an expert challenges you on it. You build retrieval over the document set, ask about notice of delay, and the system confidently returns a letter that discusses the delay event in great detail. It is a good letter. It is also dated eleven months after the event, written by the claims consultant during preparation of the claim itself. It scored highest precisely because it was written to argue the point. The contemporaneous notice, the thing you actually needed, was four badly typed lines in a routine progress email that mentioned the word &#8220;delay&#8221; once.</p>



<p class="wp-block-paragraph">Semantic similarity has no concept of a deadline. That single gap is the difference between an <strong>AI construction claims platform</strong> that shortens a disclosure exercise and one that quietly manufactures a wrong answer with a citation attached to it.</p>



<p class="wp-block-paragraph">This post covers how to build that platform on AWS: how to lay out ingestion, how to make retrieval respect the contractual clock, why schedule data must never go anywhere near your vector index, how to keep privileged material out of the same retrieval path as project records, and which AWS building blocks are actually the current ones now that several of the obvious candidates have been moved to maintenance mode.</p>



<h2 class="wp-block-heading">What a claims platform actually has to answer</h2>



<p class="wp-block-paragraph">Before any architecture, be honest about the question shapes. They are not all the same problem and they do not all get solved by retrieval.</p>



<ol class="wp-block-list"><li><strong>Chronology.</strong> What happened, in what order, and on what date was it recorded? This is a retrieval and metadata problem.</li><li><strong>Entitlement.</strong> Which clause applies, and what did it require the parties to do? This is retrieval over the contract plus careful prompting.</li><li><strong>Causation.</strong> Which event moved the critical path, and by how much? This is schedule data and date arithmetic. It is not a language problem at all.</li><li><strong>Quantum.</strong> What did the disruption cost? This is cost and resource data, joined to the events above.</li></ol>



<p class="wp-block-paragraph">Treat all four as &#8220;ask the documents&#8221; and you will get fluent nonsense on two of them. The architecture below splits them deliberately.</p>



<h2 class="wp-block-heading">Failure one: retrieval that finds the best match instead of the first one</h2>



<p class="wp-block-paragraph">Two corpora live in every dispute bundle and they look identical to an embedding model.</p>



<ul class="wp-block-list"><li><strong>Contemporaneous records.</strong> Site diaries, progress emails, minutes, early warnings, RFIs, instructions. Written while the project was running, by people with no idea a dispute was coming.</li><li><strong>Claim-era material.</strong> Narratives, expert reports, without-prejudice correspondence, internal analysis. Written afterwards, specifically to be persuasive about the same events.</li></ul>



<p class="wp-block-paragraph">Claim-era material wins on cosine similarity almost every time, because it is denser in exactly the terms you searched for. If your retriever cannot distinguish them, every answer is contaminated by the argument you were trying to test.</p>



<p class="wp-block-paragraph">The fix is metadata, applied at ingestion, and it is cheap to get right and expensive to retrofit. Amazon Bedrock Knowledge Bases reads a sidecar file that sits next to each document in S3, named with the full original filename plus <code>.metadata.json</code>. So <code>letter-0421.pdf</code> gets <code>letter-0421.pdf.metadata.json</code>. The naming convention is the only link between them; there is no separate registration step.</p>



<pre class="wp-block-code"><code>{
  "metadataAttributes": {
    "doc_date": 20240314,
    "corpus": "contemporaneous",
    "doc_type": "site_correspondence",
    "matter_id": "matter-0007",
    "date_source": "email_header",
    "privileged": false
  }
}</code></pre>



<p class="wp-block-paragraph">Look closely at <code>doc_date</code>. It is an integer, not a string, and that is not a style choice. Bedrock Knowledge Bases metadata attributes support STRING, NUMBER, BOOLEAN and STRING_LIST. The range comparison operators, the ones you need to express &#8220;on or before the notice deadline&#8221;, only apply to NUMBER. Store the date as <code>"2024-03-14"</code> and your filter will not throw an error. It will just quietly match nothing, or match everything, depending on how you wrote it. You will find out weeks later when someone asks why a document they can see in the bundle never appears in results.</p>



<p class="wp-block-paragraph">With the date as a sortable integer, a query filter can express the contractual window directly.</p>



<pre class="wp-block-code"><code>{
  "andAll": [
    { "equals":              { "key": "corpus",   "value": "contemporaneous" } },
    { "equals":              { "key": "matter_id","value": "matter-0007" } },
    { "greaterThanOrEquals": { "key": "doc_date", "value": 20240301 } },
    { "lessThanOrEquals":    { "key": "doc_date", "value": 20240329 } }
  ]
}</code></pre>



<p class="wp-block-paragraph">That is the whole trick. You are no longer asking &#8220;what is the most relevant document about this delay&#8221;. You are asking &#8220;what did the parties actually write during the window in which the contract required them to write it&#8221;. Those are different questions and only one of them is worth anything in a dispute.</p>



<h3 class="wp-block-heading">Where the date comes from matters more than the date</h3>



<p class="wp-block-paragraph">Do not use the S3 object timestamp. It records when someone copied a folder, usually years after the fact and identical across ten thousand files. Derive the date from the document itself: the <code>Date:</code> header on an email, the printed date on a letter, the period covered by a diary entry.</p>



<p class="wp-block-paragraph">Sometimes you cannot, because the scanned undated fax exists in every project archive. Record that honestly with a <code>date_source</code> attribute rather than guessing, and treat unknown-date documents as a separate review pile. An extension of time argument built on an inferred date is an argument you will lose.</p>



<h2 class="wp-block-heading">Failure two: treating the programme like a document</h2>



<p class="wp-block-paragraph">This one is worse, because the output looks right.</p>



<p class="wp-block-paragraph">Oracle Primavera P6 exports XER and PMXML files. Asta Powerproject and Microsoft Project have their own formats. XER in particular is a plain text dump of relational tables, so it goes through a text pipeline without complaint. Chunk it, embed it, and you now have vectors representing fragments of a table of activity codes with no relationships attached.</p>



<p class="wp-block-paragraph">Ask that index how much float activity A1200 had at the March data date and you will get a number. It will be well formatted and it will be invented. Total float is the product of a forward and backward pass across the whole logic network under a specific calendar. It cannot be recovered from a retrieved fragment, and a language model asked to produce it will produce something plausible instead of admitting that.</p>



<p class="wp-block-paragraph">Schedule data goes into a structured store, and the model queries it rather than reasoning about it.</p>



<ol class="wp-block-list"><li>Parse each programme file into tables. <code>PyP6Xer</code> handles XER from Python; MPXJ is a Java library that reads XER, PMXML, Asta Powerproject and MSPDI among others, which matters when the bundle contains four scheduling tools.</li><li>Load activities, logic links, calendars, resource assignments and WBS into Amazon Aurora PostgreSQL for interactive work, or into S3 with AWS Glue and Amazon Athena when you have hundreds of updates and want columnar scans.</li><li>Stamp every row with the <em>data date</em> of the update it came from. This is the single most important column in the whole platform. Without it you have a pile of schedules; with it you have a time series of the project&#8217;s own view of itself.</li><li>Run windows analysis, as-planned versus as-built comparison and float erosion in SQL or Python, deterministically, so the same inputs always give the same numbers.</li><li>Expose the results to the model as a tool it can call, or as generated SQL against a defined schema. The model turns a question into a query and narrates the result. It does not do the arithmetic.</li></ol>



<p class="wp-block-paragraph">A rough shape of the query that makes float erosion visible:</p>



<pre class="wp-block-code"><code>SELECT
    a.activity_id,
    a.data_date,
    a.total_float_days,
    a.total_float_days - LAG(a.total_float_days)
        OVER (PARTITION BY a.activity_id ORDER BY a.data_date)
      AS float_change
FROM   schedule_activities a
WHERE  a.project_id = 'PRJ-01'
  AND  a.data_date BETWEEN DATE '2024-01-01' AND DATE '2024-06-30'
ORDER BY a.activity_id, a.data_date;</code></pre>



<p class="wp-block-paragraph">Nothing clever there, and that is the point. Every number is traceable to a row that came from a named XER file. When an opposing expert asks where a figure came from, the answer is a file name and a query, not &#8220;the model said so&#8221;.</p>



<p class="wp-block-paragraph">Be realistic about effort here. Programme parsing and normalisation across inconsistent updates is the hardest part of the build and the part clients always underestimate. Activity IDs get reused, calendars change mid-project, and someone will have re-baselined without telling anyone. Budget for it.</p>



<h2 class="wp-block-heading">Failure three: an answer with no paper trail</h2>



<p class="wp-block-paragraph">In most RAG applications a citation is a nice touch. In dispute work it <em>is</em> the product. An answer that cannot be traced to a page of a disclosed document is not evidence, it is a rumour with good grammar.</p>



<p class="wp-block-paragraph">Design for that from the ingestion layer, not the presentation layer.</p>



<ul class="wp-block-list"><li><strong>Keep page and position.</strong> Amazon Bedrock Data Automation returns confidence scores and bounding box data alongside extracted fields, and Amazon Textract returns geometry per block. Carry both through the pipeline so a citation resolves to a page and a region, not just a file.</li><li><strong>Route low confidence to humans.</strong> Handwritten site diaries and faxed variation orders will produce low-confidence extractions. Those should land in a review queue by default rather than silently entering the index.</li><li><strong>Reject ungrounded answers.</strong> Amazon Bedrock Guardrails includes contextual grounding checks that score whether a response is supported by the retrieved passages. It reduces confident invention. It does not eliminate it, and anyone who tells you otherwise is selling something.</li><li><strong>Keep an immutable evidential copy.</strong> S3 Versioning plus S3 Object Lock on the landing bucket means the file the platform indexed is provably the file that was disclosed.</li></ul>



<p class="wp-block-paragraph">One design rule underpins all of it: the platform shortlists evidence, it does not decide entitlement. Recognised frameworks for this work, the Society of Construction Law Delay and Disruption Protocol and AACE International&#8217;s Recommended Practice 29R-03 on forensic schedule analysis, both assume a named analyst applying a stated method and exercising judgement. A system that outputs &#8220;the contractor is entitled to 42 days&#8221; is not helping. A system that outputs &#8220;here are the eleven contemporaneous documents inside the notice window, here is the float movement across those updates, here is what is missing&#8221; is doing real work.</p>



<h2 class="wp-block-heading">Failure four: one index for privileged and non-privileged material</h2>



<p class="wp-block-paragraph">Dispute bundles contain legal advice, counsel&#8217;s opinions, without-prejudice correspondence and internal settlement analysis. Those must not be retrievable through the same path as project records.</p>



<p class="wp-block-paragraph">The tempting shortcut is a <code>privileged: false</code> metadata filter on every query. Do not rely on that as your boundary. A metadata filter is a query parameter. One missing filter in one code path, one debug endpoint, one caching layer that drops it, and privileged material surfaces in a general search. The blast radius of that mistake is not a bug report.</p>



<p class="wp-block-paragraph">Separate the indexes physically and separate the IAM roles that can reach them. Amazon S3 Vectors makes this practical: you can set a dedicated customer-managed KMS key per vector index, and you get a large number of indexes per vector bucket, so per-matter and per-sensitivity separation does not become an operational burden. Keep the metadata flag as well, because defence in depth is free, but make the identity boundary the one you actually trust.</p>



<p class="wp-block-paragraph">Amazon Macie is worth pointing at the landing bucket to find personal data you did not expect, particularly in HR records and accident reports that get swept into project archives.</p>



<h2 class="wp-block-heading">Choosing the AWS building blocks, including what not to build on</h2>



<p class="wp-block-paragraph">A lot of published architectures for this kind of platform are now pointing at services AWS has stopped developing. Two matter here, and the dates are the point.</p>



<ul class="wp-block-list"><li><strong>Amazon Kendra</strong> entered maintenance mode on 30 June 2026 and stops accepting new customers on 30 July 2026. Existing customers keep support and security fixes but no new capability. AWS directs new enterprise search and RAG work to Amazon Bedrock Knowledge Bases. If a tutorial or a proposal you are reading starts with a Kendra index, it predates that change.</li><li><strong>Amazon Bedrock Agents</strong> moved to maintenance mode in the same round of service availability changes, with Amazon Bedrock AgentCore as the successor for agentic orchestration. Check the current AWS service availability page before you commit an orchestration layer.</li></ul>



<p class="wp-block-paragraph">For the retrieval layer itself, Bedrock Knowledge Bases now comes in two shapes and the choice is a real trade-off rather than a marketing tier.</p>



<h3 class="wp-block-heading">Managed Knowledge Base</h3>



<p class="wp-block-paragraph">AWS manages the vector store, embeddings model, re-ranker and retrieval orchestration as a single primitive, with native connectors for Amazon S3, SharePoint, Confluence, Google Drive, OneDrive and a web crawler, plus automatic parsing strategy selection and a retriever that decomposes multi-step queries. The connectors pull source permissions along with content, which matters when the document set lives in the client&#8217;s SharePoint rather than a bucket you control.</p>



<p class="wp-block-paragraph">Where it wins: you get a working retrieval layer in an afternoon instead of a fortnight, and the parsing tuning that normally eats the first weeks of a build is done for you. For a first matter, or a proof of value before a client commits budget, this is the one I would reach for.</p>



<h3 class="wp-block-heading">Custom Knowledge Base</h3>



<p class="wp-block-paragraph">You bring your own vector store and control chunking, embedding model and index layout.</p>



<p class="wp-block-paragraph">Where it wins: claims work has awkward chunking requirements. A two-page letter split mid-sentence at a page boundary produces a chunk where the notice sentence has lost its date and its addressee. Controlling chunk boundaries around document structure, and controlling which index a document lands in, are both easier when you own the store. Where it doesn&#8217;t: you now own embedding model upgrades, re-indexing, sync failures and capacity, which is real ongoing work for a small team.</p>



<p class="wp-block-paragraph">Start managed, build a retrieval evaluation set of real questions with known correct documents, and only move to custom when that set demonstrates the problem is chunking. Most teams migrate on a hunch and discover the problem was metadata all along.</p>



<h3 class="wp-block-heading">Where the vector storage bill actually comes from</h3>



<p class="wp-block-paragraph">Rates change, so learn the billing mechanism rather than a number. Amazon S3 Vectors charges on three axes: upload volume by logical gigabyte, storage by logical gigabyte, and queries by data processed, where data processed scales with the size of the index being searched. Note that filtering does not reduce the data processed by a query.</p>



<p class="wp-block-paragraph">That shape suits claims work unusually well. A dispute archive is enormous and cold: millions of chunks, queried by a handful of analysts a few hundred times a day, so you pay mostly for storage, which is the cheap axis. Compare that against Amazon OpenSearch Serverless, which prices on provisioned compute units and therefore rewards high query volume against a smaller index, or Aurora PostgreSQL with pgvector when you already need Aurora for the schedule tables and would rather run one system than two.</p>



<p class="wp-block-paragraph">The practical lever is to split indexes per matter. Query cost scales with index size, so one giant index across every dispute you have ever run makes every query more expensive than it needs to be, on top of being a bad idea for confidentiality.</p>



<h2 class="wp-block-heading">A reference pipeline</h2>



<ol class="wp-block-list"><li>Everything lands in S3 under a per-matter prefix, with Versioning and Object Lock enabled on the evidential copy.</li><li>S3 event notifications trigger AWS Step Functions. Use Step Functions rather than a chain of Lambdas so that a failed extraction on page 300 of a 400-page bundle is visible and resumable.</li><li>Classify and split. Scanned bundles arrive as one PDF containing forty separate documents. Splitting them correctly is a prerequisite for dating them correctly.</li><li>Extract text with Amazon Bedrock Data Automation or Amazon Textract, keeping confidence scores and geometry.</li><li>Derive the document date and write the <code>.metadata.json</code> sidecar. Anything undated goes to the review queue.</li><li>Route by type: correspondence to the knowledge base, programme files to the XER parser and the relational store, cost data to its own tables.</li><li>Sync the knowledge base, then run your retrieval evaluation set before anyone uses it. A sync that succeeds is not the same as an index that answers correctly.</li><li>Serve through an API that refuses to return an answer without citations, and log every query with the filters that were applied.</li></ol>



<p class="wp-block-paragraph">Define the whole thing in Terraform or OpenTofu from the start. Matters are per-client and short-lived, and standing one up should be a variable file, not an afternoon in the console. Point Amazon CloudWatch, or Grafana Cloud if you already run Grafana elsewhere, at the Step Functions execution metrics so a silently failing extraction stage does not go unnoticed for a week.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting</h2>



<h3 class="wp-block-heading">Date filters return nothing, and no error</h3>



<p class="wp-block-paragraph">Almost always the date was stored as a string. Range operators need NUMBER. Convert to an integer in <code>YYYYMMDD</code> form and re-sync the affected documents.</p>



<h3 class="wp-block-heading">A document is in the bucket but never appears in results</h3>



<p class="wp-block-paragraph">Check the sidecar filename first. It must be the complete original filename with <code>.metadata.json</code> appended, extension included. <code>report.pdf.metadata.json</code> works; <code>report.metadata.json</code> is a file the ingestion job will happily ignore. After that, check whether a filter in the query path is excluding it.</p>



<h3 class="wp-block-heading">Answers cite the right document but the wrong passage</h3>



<p class="wp-block-paragraph">Chunking split the document somewhere structurally meaningful. Look at the raw chunks for that file. If the notice sentence and its date are in different chunks, no amount of prompt tuning fixes it. That is the signal to take control of chunking.</p>



<h3 class="wp-block-heading">Float figures do not match the client&#8217;s own analysis</h3>



<p class="wp-block-paragraph">Check calendars before you check logic. Different activity calendars, a changed default calendar, or an update where someone applied a progress override will move float without any logic change. Reconcile activity counts between your parsed tables and the source file before trusting anything downstream.</p>



<h3 class="wp-block-heading">Query costs jumped without more usage</h3>



<p class="wp-block-paragraph">An index grew. With storage-side vector search, query cost tracks the size of the index being scanned, so ingesting a large new bundle raises the price of every subsequent query against that index. Split by matter.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list"><li>Using the file&#8217;s storage timestamp as the document date. It records the migration, not the event.</li><li>Indexing claim narratives and contemporaneous records into the same corpus with no way to tell them apart.</li><li>Embedding programme exports because they happen to be text files.</li><li>Treating a metadata filter as a privilege boundary instead of an optimisation.</li><li>Letting the model state entitlement conclusions rather than assembling and citing evidence.</li><li>Building on services that have moved to maintenance mode because the tutorial you followed predates the change.</li><li>Shipping without a retrieval evaluation set, so you have no way to know whether a change made things better or worse.</li><li>One index for every matter, which is both a cost problem and a confidentiality problem.</li></ul>



<h2 class="wp-block-heading">Best practices</h2>



<ul class="wp-block-list"><li>Make the document date a first-class, numeric, filterable attribute, and record where it came from.</li><li>Keep an immutable evidential copy separate from the working copy the pipeline mutates.</li><li>Separate structured schedule and cost data from unstructured documents, and let the model query the former rather than reason about it.</li><li>Build a retrieval evaluation set from real questions with known correct documents before you tune anything.</li><li>Enforce citations at the API layer, so an uncited answer is impossible rather than discouraged.</li><li>Isolate privileged material by index and by IAM role, with metadata as a second layer.</li><li>Log every query with its filters, so you can reconstruct how any given answer was reached.</li><li>Define infrastructure as code so a new matter is a deployment, not a project.</li></ul>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">Can an AI construction claims platform replace a delay expert?</h3>



<p class="wp-block-paragraph">No, and building toward that goal produces something unusable. Established forensic frameworks assume a named analyst applying a stated method whose reasoning can be tested. The platform&#8217;s value is compressing weeks of document review into hours and making the schedule data queryable, so the expert spends their time on judgement rather than searching.</p>



<h3 class="wp-block-heading">Should I use Amazon Kendra for the search layer?</h3>



<p class="wp-block-paragraph">Not for a new build. Kendra entered maintenance mode on 30 June 2026 and closed to new customers on 30 July 2026, with AWS pointing to Bedrock Knowledge Bases for equivalent and more current capability. Existing Kendra deployments continue to be supported, so this is a migration assessment rather than an emergency, but starting there now means starting on a service with no roadmap.</p>



<h3 class="wp-block-heading">How do I stop the model inventing float and delay figures?</h3>



<p class="wp-block-paragraph">Do not give it the chance. Keep schedule data in a relational or columnar store and have the model generate queries against a defined schema, or call a tool that runs a fixed calculation. The arithmetic happens in SQL or Python where it is deterministic and reproducible; the model only turns questions into queries and results into sentences.</p>



<h3 class="wp-block-heading">Which vector store should I choose for a claims archive?</h3>



<p class="wp-block-paragraph">Match the store to your query pattern. Large, cold archives queried by a few analysts favour storage-priced options like Amazon S3 Vectors, where you mostly pay to keep the data. Smaller indexes hit constantly favour compute-priced options like Amazon OpenSearch Serverless. If you already run Aurora PostgreSQL for schedule data, pgvector alongside it is a legitimate way to avoid operating a second system.</p>



<h3 class="wp-block-heading">How do I handle scanned and handwritten site records?</h3>



<p class="wp-block-paragraph">Extract them with confidence scores retained, set a threshold, and route everything below it to human review before indexing. Handwritten diaries are frequently the most probative documents in a delay claim and also the least reliable to read automatically, so the review queue is not an edge case. Plan capacity for it.</p>



<h3 class="wp-block-heading">Where do documents come from if they are not already in S3?</h3>



<p class="wp-block-paragraph">Most project records live in a common data environment such as Procore, Autodesk Construction Cloud, Aconex or a client SharePoint tenancy. Bedrock Managed Knowledge Base has native connectors for SharePoint, Confluence, Google Drive and OneDrive that ingest permissions alongside content. For platforms without a native connector, export to S3 and keep the export manifest as part of the disclosure record.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">An <strong>AI construction claims platform</strong> lives or dies on whether it understands time. Every hard requirement in this build traces back to that: numeric dates so you can filter to a contractual window, a data date on every schedule row so float movement is measurable, a corpus flag so contemporaneous records are not drowned out by material written to argue about them, and citations that resolve to a page so any answer can be checked.</p>



<p class="wp-block-paragraph">Get the temporal metadata right at ingestion and the rest of the architecture is ordinary AWS work. Get it wrong and you have built a very expensive way to retrieve the most persuasive document instead of the true one.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help building this on AWS?</h2>



<p class="wp-block-paragraph">I design and build document and data platforms on AWS, and this kind of system sits squarely in that work. Things I can help with:</p>



<ul class="wp-block-list"><li>Designing the ingestion pipeline: S3 landing zones with Object Lock, Step Functions orchestration, splitting and classifying scanned bundles, and confidence-based routing to human review.</li><li>Getting the temporal metadata model right, including date derivation, sidecar generation and filter design against Amazon Bedrock Knowledge Bases.</li><li>Parsing Primavera P6 XER and PMXML exports into queryable tables in Aurora PostgreSQL or S3 with Glue and Athena, with a data date on every row.</li><li>Choosing and sizing the vector layer across Amazon S3 Vectors, OpenSearch Serverless and pgvector, based on your actual query pattern rather than a benchmark.</li><li>Building index and IAM separation for privileged material, plus KMS key strategy and Macie scanning of landing buckets.</li><li>Setting up retrieval evaluation, citation enforcement, query audit logging and CloudWatch or Grafana dashboards over the pipeline so failures surface early.</li></ul>



<p class="wp-block-paragraph">If you are partway into something like this already, send me a sample metadata sidecar, a Step Functions execution history, or a query that returns the wrong document, and I will tell you what I think is going on.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/case-studies/ai-construction-claims-platform-aws/">Building an AI Construction Claims Platform on AWS That Holds Up Under Scrutiny</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
