<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>System of Record | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/system-of-record/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/system-of-record/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Sat, 26 Sep 2026 06:00:23 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>System of Record | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/system-of-record/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Your CRM Is Not a Database: Where GoHighLevel Should Stop</title>
		<link>https://john-nessime.com/blog/data-engineering/gohighlevel-as-a-database/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sat, 26 Sep 2026 06:00:23 +0000</pubDate>
				<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Enterprise Integration]]></category>
		<category><![CDATA[SaaS Engineering]]></category>
		<category><![CDATA[Architecture Debt]]></category>
		<category><![CDATA[CRM Audit]]></category>
		<category><![CDATA[Custom Objects]]></category>
		<category><![CDATA[Data Modeling]]></category>
		<category><![CDATA[GoHighLevel]]></category>
		<category><![CDATA[Idempotency]]></category>
		<category><![CDATA[Marketing Automation]]></category>
		<category><![CDATA[Rate Limiting]]></category>
		<category><![CDATA[REST API]]></category>
		<category><![CDATA[Reverse ETL]]></category>
		<category><![CDATA[Schema Design]]></category>
		<category><![CDATA[Snapshots]]></category>
		<category><![CDATA[Sub-Account Management]]></category>
		<category><![CDATA[System of Record]]></category>
		<category><![CDATA[Vendor Lock-In]]></category>
		<category><![CDATA[Webhooks]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=649</guid>

					<description><![CDATA[<p>A CRM never pushes back when you add one more field, which is exactly why the schema drifts until someone asks a question that needs a join. Four boundaries for deciding what belongs in GoHighLevel, what Custom Objects actually change, and the CRM-in-front, system-of-record-behind pattern I default to.</p>
<p>The post <a href="https://john-nessime.com/blog/data-engineering/gohighlevel-as-a-database/">Your CRM Is Not a Database: Where GoHighLevel Should Stop</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">&#8220;Can you pull every job we did last quarter, by technician, with the margin on each one?&#8221;</p>



<p class="wp-block-paragraph">The answer was no. Not &#8220;no, that&#8217;s expensive.&#8221; Just no. The jobs weren&#8217;t records. Each one was a couple of tags on a contact, three custom fields added at different times by different people, an opportunity someone had dragged into Won, and a note cut off mid-sentence. There was no job. There was a person that a job had happened to.</p>



<p class="wp-block-paragraph">Nobody made a bad decision to get there. &#8220;Can we just add a field for that?&#8221; is a five-second request and the answer is always yes, because adding a field is easy. That&#8217;s the trap. The CRM never pushes back, so the schema drifts for eighteen months and nobody notices until someone asks a question that needs a join.</p>



<p class="wp-block-paragraph">This post is about where that line sits: not whether to use the platform, but where <strong>GoHighLevel as a database</strong> stops being a shortcut and starts being architecture debt. I&#8217;ll cover the failure that stays invisible, the four boundaries that decide it, what Custom Objects actually change, and the arrangement I default to when the CRM isn&#8217;t the right home.</p>



<h2 class="wp-block-heading">The failure mode nobody sees coming</h2>



<p class="wp-block-paragraph">Databases push back. Declare a column as a date and it refuses to hold &#8220;next Tuesday.&#8221; Declare a foreign key and it refuses to orphan the row. Half the value of a schema isn&#8217;t storage, it&#8217;s the set of things it won&#8217;t let you do.</p>



<p class="wp-block-paragraph">A custom field holds whatever you type. That&#8217;s a feature when a rep jots down a competitor&#8217;s name. It&#8217;s a slow disaster when the field is called <code>contract_value</code> and it contains &#8220;1,250.00&#8221;, &#8220;$1250&#8221;, &#8220;1250 (pending)&#8221; and &#8220;ask Dave&#8221; across four hundred records.</p>



<p class="wp-block-paragraph">Here&#8217;s why it stays invisible: nothing breaks. Workflows keep firing. Dashboards keep rendering, because they count records rather than summing values. The failure surfaces the day someone needs an aggregate, and by then you have four hundred rows of ambiguity and no way to tell which reading was right. You can&#8217;t retroactively decide whether &#8220;1250 (pending)&#8221; was invoiced.</p>



<p class="wp-block-paragraph">History is the other half. A CRM field is a current value. Overwrite it and the old one is gone. Anything touching money, entitlements or consent eventually needs to prove what the state was and when, and the platform discarded that silently, months earlier.</p>



<h2 class="wp-block-heading">What the platform is genuinely good at</h2>



<p class="wp-block-paragraph">An argument against a tool isn&#8217;t credible unless it starts with the tool&#8217;s real case. GoHighLevel is very good at a specific set of things, most of them expensive to build yourself.</p>



<ul class="wp-block-list">
<li><strong>Conversation state across channels.</strong> One thread per person spanning SMS, email and calls, with delivery status attached. Rebuilding that is months of work and a compliance surface you don&#8217;t want.</li>

<li><strong>Sequenced outreach with human interrupts.</strong> Wait steps, conditions, and a reply pulling someone out of a sequence. Easy to describe, tedious to get right.</li>

<li><strong>Tenant isolation by default.</strong> Each sub-account is its own location, and nothing crosses unless you deliberately copy it.</li>

<li><strong>Non-engineers can change it.</strong> A workflow a client can edit on a Tuesday afternoon beats an elegant pipeline only you can touch.</li>
</ul>



<p class="wp-block-paragraph">Every one of those is about people and communication. That&#8217;s the pattern: when the thing you&#8217;re storing is a person, or something that happened to a person, the CRM is the right home. When it has a life independent of any person, you&#8217;re outside the design.</p>



<h2 class="wp-block-heading">Four boundaries that decide it for you</h2>



<p class="wp-block-paragraph">&#8220;Is this a database?&#8221; isn&#8217;t a useful question. These four are decidable, and any one of them crossing is usually enough.</p>



<h3 class="wp-block-heading">1. The question you need to ask requires a join</h3>



<p class="wp-block-paragraph">Not &#8220;show me contacts tagged X.&#8221; I mean revenue by technician by quarter, or every account with more than three open items where the last payment was over sixty days ago: grouping, filtering across related records, and arithmetic on typed values, all at once. Smart lists answer a narrow, predefined set of questions well, but they aren&#8217;t a query engine, and the gap between the two is where most people discover they&#8217;ve built a database by accident.</p>



<h3 class="wp-block-heading">2. More than one system writes the same field</h3>



<p class="wp-block-paragraph">This is the one that quietly costs the most. A stock level or invoice status lives in a CRM field, and both an automation and an external sync can write it. There&#8217;s no transaction and no compare-and-set. Last write wins, and &#8220;last&#8221; is whichever request happened to land second. You see it as an occasional wrong value you can&#8217;t reproduce, which is the worst kind of bug to be handed.</p>



<p class="wp-block-paragraph">The fix isn&#8217;t better retry logic. The field has the wrong owner. Exactly one system should write any given piece of state, and if that system isn&#8217;t the CRM, the CRM holds a copy for display rather than the value itself.</p>



<h3 class="wp-block-heading">3. Someone will need the history</h3>



<p class="wp-block-paragraph">Anything where you may later have to prove a prior state needs append-only storage. The workaround people reach for is a note per change. It doesn&#8217;t hold: notes aren&#8217;t structured, aren&#8217;t queryable, and the export path treats them as an afterthought, which leads straight to the next boundary.</p>



<h3 class="wp-block-heading">4. The exit shape doesn&#8217;t match what you put in</h3>



<p class="wp-block-paragraph">People check this last and should check it first. Contact export lands as CSV, and only the most recent note travels with it, truncated to 255 characters. A year of notes exports as one, clipped. Custom object data isn&#8217;t part of a direct export at all: the documented route is a snapshot from one sub-account to another, which is a copy inside the platform, not a file you keep. Export permissions are role-gated, so a client who believes they own their data may not be able to pull it themselves.</p>



<p class="wp-block-paragraph">None of that makes it a bad choice. It makes it a bad <em>sole</em> home for anything you&#8217;d be in trouble without.</p>



<h2 class="wp-block-heading">What Custom Objects change, and what they don&#8217;t</h2>



<p class="wp-block-paragraph">Custom Objects are a real improvement and worth using. Properties, policies, vehicles, courses: things that aren&#8217;t people, modelled as first-class records with their own fields and associations back to contacts, usable in workflows. For years the only option was squashing everything into a contact, and that was worse. They don&#8217;t turn the CRM into a database, though, and the reasons are specific rather than philosophical.</p>



<ul class="wp-block-list">
<li><strong>Ten objects per sub-account, documented as a hard cap.</strong> That&#8217;s your entire entity budget. Fifteen nouns worth modelling means five go back into custom fields.</li>

<li><strong>Uniqueness is limited and one-way.</strong> Up to ten unique fields per object, and only single line text, multi line text, number and phone types can be unique. It&#8217;s enforced across the sub-account and every entry point including the API, which is genuinely useful. But downgrade a unique field to non-unique and you cannot make it unique again.</li>

<li><strong>Write access is admin-only.</strong> Creating, updating and deleting object records is restricted to location admins; regular users are read-only. Check this before assuming the front desk will maintain records.</li>

<li><strong>Records count against storage quota.</strong> Object records and attached files consume the same account storage as everything else, so high-volume machine-generated data is the wrong shape.</li>

<li><strong>The exit is a snapshot, not an export.</strong> Objects, fields and associations move between sub-accounts via snapshots, subject to the same cap in the target account.</li>
</ul>



<p class="wp-block-paragraph">Read those as design constraints rather than complaints. They tell you the intended scale: dozens of entity types, no. Millions of rows, no. A handful of business nouns with clean identity and moderate volume, yes, and it works well there.</p>



<h2 class="wp-block-heading">The API budget is a design constraint too</h2>



<p class="wp-block-paragraph">If you&#8217;re treating the platform as a datastore behind an integration, the rate limits shape what&#8217;s possible. On the v2 API a marketplace app gets a burst allowance of 100 requests per 10 seconds per resource, and 200,000 requests per day per resource, where a resource means a location or company. Version 1 is end of support, so anything new belongs on v2.</p>



<p class="wp-block-paragraph">Every v2 request carries a version header alongside the bearer token. It&#8217;s easy to forget and produces a confusing failure when missing:</p>



<pre class="wp-block-code"><code>curl -H "Authorization: Bearer $TOKEN" 
     -H "Version: 2021-04-15" 
     -H "Accept: application/json" 
     "$BASE_URL/&lt;resource&gt;?locationId=$LOCATION_ID"</code></pre>



<p class="wp-block-paragraph">More useful than the numbers is that responses carry their own accounting, so you never have to guess where you are in the window: <code>X-RateLimit-Remaining</code> and <code>X-RateLimit-Max</code> cover the burst interval, <code>X-RateLimit-Daily-Remaining</code> and <code>X-RateLimit-Limit-Daily</code> cover the day.</p>



<p class="wp-block-paragraph">Pace against the remaining count rather than reacting to failures, and watch the daily counter separately: burst limits recover in seconds, a daily allowance doesn&#8217;t, and a backfill that exhausts it takes the rest of the day&#8217;s normal traffic with it. The consequence is that polling is expensive and webhooks are cheap. Push where you can; poll only to reconcile.</p>



<h2 class="wp-block-heading">The shape I reach for: CRM in front, system of record behind</h2>



<p class="wp-block-paragraph">When something crosses one of those boundaries, this is the arrangement I default to. It&#8217;s deliberately boring.</p>



<ol class="wp-block-list">
<li><strong>Pick one owner per piece of state and write it down.</strong> Contact details and conversation history belong to the CRM. Money, inventory and anything with an audit requirement belong to the external store. No field gets two owners.</li>

<li><strong>Give every record a stable external ID on both sides.</strong> Do this on day one. Retrofitting identity onto records matched by email is genuinely painful.</li>

<li><strong>Push changes out with webhooks, not polling.</strong> Webhook to a queue, queue to a worker, and make the worker idempotent, because you will receive duplicates and you&#8217;ll need to replay after an outage.</li>

<li><strong>Reconcile on a schedule anyway.</strong> Webhooks get dropped. A nightly sweep comparing counts and updated timestamps, re-pulling only the drift, catches what the push path missed without burning the daily allowance.</li>

<li><strong>Push a flattened summary back for display.</strong> The CRM doesn&#8217;t need the ledger, just the two or three values a human wants on the contact record. Treat those fields as read-only and never let a workflow write them.</li>
</ol>



<p class="wp-block-paragraph">The external side doesn&#8217;t need to be impressive. For a small engagement it&#8217;s usually managed Postgres, or plain Postgres on a modest VPS from somewhere like Contabo or InterServer, sat behind Cloudflare so nothing is directly exposed. The glue is often n8n or Make rather than anything custom. What matters is that constraints exist:</p>



<pre class="wp-block-code"><code>CREATE TABLE job (
  id             bigserial PRIMARY KEY,
  crm_contact_id text        NOT NULL,
  technician_id  bigint      NOT NULL REFERENCES technician(id),
  completed_on   date        NOT NULL,
  value_cents    bigint      NOT NULL CHECK (value_cents &gt;= 0),
  created_at     timestamptz NOT NULL DEFAULT now()
);</code></pre>



<p class="wp-block-paragraph">Nothing clever there. But <code>value_cents</code> can&#8217;t hold &#8220;ask Dave&#8221;, <code>technician_id</code> can&#8217;t point at a technician who doesn&#8217;t exist, and the quarterly margin question becomes one query instead of a conversation about why it can&#8217;t be answered.</p>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<ul class="wp-block-list">
<li><strong>&#8220;We&#8217;re too small for a second system.&#8221;</strong> Sometimes right, and worth taking seriously. But the cost of adding a store later isn&#8217;t the store, it&#8217;s the reconstruction, and some of that history is unrecoverable.</li>

<li><strong>&#8220;We&#8217;ll clean the data up later.&#8221;</strong> Cleanup needs to know what the value should have been. Free-text fields with no history don&#8217;t retain that. This is the one I&#8217;d push back on hardest.</li>

<li><strong>&#8220;Custom Objects solved this.&#8221;</strong> They solved modelling non-person entities, which was the biggest gap. They didn&#8217;t add joins, transactions, versioning or an export path.</li>

<li><strong>&#8220;The client wants everything in one place.&#8221;</strong> They want one place to <em>look</em>, which is a UI requirement, not a storage requirement. A summary pushed back to the contact record satisfies it.</li>

<li><strong>&#8220;We can export whenever we want.&#8221;</strong> Test it before you rely on it. Export the notes. Export a custom object. Do it as the client&#8217;s own user, not as an admin.</li>
</ul>



<h2 class="wp-block-heading">How I&#8217;d decide</h2>



<ol class="wp-block-list">
<li><strong>Name the thing.</strong> If it&#8217;s a noun that isn&#8217;t a person, it wants to be a record, not a field on a contact.</li>

<li><strong>Write out the hardest question anyone will ask of it.</strong> The actual sentence. If it needs grouping, arithmetic, or two entity types at once, the CRM isn&#8217;t the home.</li>

<li><strong>Count the writers.</strong> More than one system writing the same field means the CRM holds a copy, not the value.</li>

<li><strong>Decide whether history matters.</strong> Assume it does for anything involving money or consent.</li>

<li><strong>Do the export before you commit.</strong> Not the docs, the actual file. Whatever doesn&#8217;t come out cleanly is data you don&#8217;t really own.</li>

<li><strong>Check the entity budget.</strong> Ten objects per sub-account. If you&#8217;re modelling toward that ceiling, decide now which nouns get a slot.</li>
</ol>



<p class="wp-block-paragraph">Steps two and five take about twenty minutes together and prevent most of what I&#8217;ve described.</p>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Can I use GoHighLevel as a database for my business data?</h3>



<p class="wp-block-paragraph">For people, conversations, bookings and pipeline state, yes, and it&#8217;s a good one. For anything needing joins, transactions, versioned history or guaranteed export, no. Using <strong>GoHighLevel as a database</strong> for that second category works right up until someone asks a question the platform can&#8217;t express, and by then the data is already ambiguous.</p>



<h3 class="wp-block-heading">How many Custom Objects can a sub-account have?</h3>



<p class="wp-block-paragraph">Ten per location, documented as a hard cap. Each object supports up to ten unique fields, and only single line text, multi line text, number and phone types can be marked unique. Treat those slots as a fixed budget when you plan the model.</p>



<h3 class="wp-block-heading">Can I export Custom Object data out of the platform?</h3>



<p class="wp-block-paragraph">Not through a direct export the way contacts go to CSV. The documented path is a snapshot between sub-accounts, which carries fields and associations but stays inside the platform. The API does support CRUD on objects and records, so a scripted pull is the practical route to a copy you keep.</p>



<h3 class="wp-block-heading">Why do my exported notes look truncated?</h3>



<p class="wp-block-paragraph">Contact export includes only the most recent note, cut to 255 characters. That&#8217;s expected behaviour, not a bug. If notes hold anything you&#8217;d need later, pull them through the API and store them somewhere that keeps all of them.</p>



<h3 class="wp-block-heading">What usually trips the API rate limits?</h3>



<p class="wp-block-paragraph">Backfills, and polling loops that scale with contact count. The limits are 100 requests per 10 seconds and 200,000 per day, both per resource. Read the rate limit headers on each response and pace against them rather than retrying into a wall.</p>



<h3 class="wp-block-heading">Does a sub-account count as a real tenancy boundary?</h3>



<p class="wp-block-paragraph">For operational separation, yes: each is its own location and nothing crosses unless you copy it. The consequence is that cross-client reporting has to happen outside the platform, because no query spans locations. If portfolio-level reporting is a requirement, plan for an external store from the start.</p>



<h3 class="wp-block-heading">Should I migrate off the CRM entirely?</h3>



<p class="wp-block-paragraph">Usually not. A full migration means rebuilding the conversation and outreach layer, which is the part it does best. Moving one category of data out, with a clear owner and a summary pushed back for display, is smaller and reversible.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">The CRM will never tell you that you&#8217;ve gone too far. There&#8217;s no constraint to violate, no error to raise, no migration to fail. That&#8217;s exactly what makes the boundary worth drawing on purpose, before the field gets added rather than after four hundred records have been through it.</p>



<p class="wp-block-paragraph">The test I&#8217;d keep: write down the hardest question anyone will ever ask of this data, then check whether the platform can express it. If it can&#8217;t, that&#8217;s not a reporting gap a better dashboard will close. It&#8217;s the boundary telling you where <strong>GoHighLevel as a database</strong> should stop and a real system of record should start. The identity mapping, the webhooks, the reconciliation sweep: all of that is plumbing once the decision is made.</p>



<h2 class="wp-block-heading">Working on this with me</h2>



<p class="wp-block-paragraph">Most of what I do here is unglamorous and ends with the reporting question getting answered:</p>



<ul class="wp-block-list">
<li>Auditing an existing setup and mapping which custom fields have quietly become load-bearing, with an owner assigned to each.</li>

<li>Drawing the boundary: what stays in the CRM, what moves to Custom Objects, what needs an external system of record.</li>

<li>Building the sync layer: webhooks into a queue, idempotent workers, stable ID mapping both ways, and a reconciliation job that catches dropped events without burning the API allowance.</li>

<li>Standing up the store: schema design with constraints that hold, on managed Postgres or a small VPS, with backups you&#8217;ve tested restoring rather than assumed.</li>

<li>Rescuing free-text fields into typed columns, including the judgement calls about what&#8217;s recoverable.</li>

<li>Handover packaging: a real export, documented, so a client can leave with their data whether or not they keep working with you.</li>
</ul>



<p class="wp-block-paragraph">If you&#8217;re weighing this up, send me something concrete: a list of your custom field names, a sample export, or the query you can&#8217;t currently answer. That&#8217;s usually enough to tell you where the line sits.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/data-engineering/gohighlevel-as-a-database/">Your CRM Is Not a Database: Where GoHighLevel Should Stop</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Primavera P6 vs Unifier vs Aconex vs Textura: Which Oracle Product Do You Actually Need?</title>
		<link>https://john-nessime.com/blog/devops/primavera-p6-vs-unifier-vs-aconex/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Fri, 28 Aug 2026 18:00:00 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[DevOps]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[BIM]]></category>
		<category><![CDATA[Capital Projects]]></category>
		<category><![CDATA[Cloud]]></category>
		<category><![CDATA[Common Data Environment]]></category>
		<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[Data Integration]]></category>
		<category><![CDATA[Enterprise Software]]></category>
		<category><![CDATA[Infrastructure]]></category>
		<category><![CDATA[Oracle]]></category>
		<category><![CDATA[Oracle Aconex]]></category>
		<category><![CDATA[Oracle Textura]]></category>
		<category><![CDATA[Primavera Gateway]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[Primavera Unifier]]></category>
		<category><![CDATA[Project Controls]]></category>
		<category><![CDATA[REST API]]></category>
		<category><![CDATA[Self Hosting]]></category>
		<category><![CDATA[Software Selection]]></category>
		<category><![CDATA[System of Record]]></category>
		<category><![CDATA[Vendor Lock-In]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=252</guid>

					<description><![CDATA[<p>All four Oracle Construction and Engineering products overlap on purpose, and three of them hold cost. The damage comes from owning two and never deciding which one is authoritative. A practical comparison by whose record each product holds, plus a decision procedure and the integration constraints nobody scopes.</p>
<p>The post <a href="https://john-nessime.com/blog/devops/primavera-p6-vs-unifier-vs-aconex/">Primavera P6 vs Unifier vs Aconex vs Textura: Which Oracle Product Do You Actually Need?</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The forecast in the monthly report doesn&#8217;t match the forecast in the cost system. Someone asks which one is right, and the honest answer is that nobody knows, because there are three places holding a number and nobody ever wrote down which one wins.</p>



<p class="wp-block-paragraph">That conversation is usually the first sign that an organisation bought two Oracle products without deciding what each one was for. It is a common outcome, and it is not because the products are bad. It&#8217;s because they are sold as a suite, they overlap on purpose, and the sales conversation is about capability rather than authority.</p>



<p class="wp-block-paragraph">This post is a working comparison of the four Oracle Construction and Engineering products people most often confuse: <strong>Primavera P6 vs Unifier vs Aconex</strong>, plus Textura. One profile each, what it genuinely owns, where it wins, where it doesn&#8217;t, and then the part nobody covers: how to decide which system holds the authoritative number when three of them offer a cost module.</p>



<h2 class="wp-block-heading">The question that actually sorts them</h2>



<p class="wp-block-paragraph">Feature lists will not separate these products. All four manage documents. Three of them track cost. Two of them hold a schedule. If you compare capability columns you will conclude that you need all four, which is exactly the conclusion the suite is designed to produce.</p>



<p class="wp-block-paragraph">The question that does separate them is: <em>whose record is this, and who else needs to write to it?</em></p>



<p class="wp-block-paragraph">Two of these products are single-organisation systems. You run them, you own the data, and outside parties get in only if you give them a seat and they accept your governance. One of them is deliberately multi-organisation, where every company on the project holds its own workspace and decides what it shares. And one is a transactional network that touches money leaving your bank account, which makes it a finance system wearing a construction badge.</p>



<p class="wp-block-paragraph">Get that axis right and the rest of the decision mostly falls out.</p>



<h2 class="wp-block-heading">Primavera P6: the schedule, and only the schedule</h2>



<p class="wp-block-paragraph">P6 is a critical path method scheduling engine with resource and cost loading on top. It exists to answer questions about time: what drives the finish date, what happens if this activity slips, what does the resource histogram look like if we accelerate.</p>



<p class="wp-block-paragraph">Two things confuse people here. First, P6 comes in two shapes. P6 Professional is the desktop client that planners actually live in. P6 EPPM is the enterprise web layer with the shared database, portfolio views and web services. Most organisations run both against the same database, and most planners will fight you to keep the desktop client.</p>



<p class="wp-block-paragraph">Second, Oracle Primavera Cloud is a separate product, not P6 in a browser. Oracle says so on its own product page, which is unusually direct for vendor copy. Primavera Cloud shares a lineage with P6 and is positioned as the cloud destination, but the data model is not identical and things planners rely on in P6 do not all have equivalents. If someone tells you a move to Primavera Cloud is a hosting change, they have not tried it.</p>



<h3 class="wp-block-heading">Where P6 wins</h3>



<ul class="wp-block-list">
<li>Contractual programme scheduling, where the schedule is a deliverable and gets audited</li>

<li>Forensic delay work, because the XER and PMXML formats are what every expert and every third-party analysis tool expects</li>

<li>Resource-loaded planning across a portfolio with a shared resource pool</li>

<li>Anywhere the client&#8217;s contract names it, which is more places than you would expect</li>
</ul>



<h3 class="wp-block-heading">Where P6 doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Cost control. P6 has cost fields and they are useful for cash flow shape, but a resource-loaded schedule is not a commitment ledger. It has no concept of a purchase order lifecycle, a variation approval or a payment certificate</li>

<li>Correspondence and document control. There is nothing here for transmittals, RFIs or drawing registers</li>

<li>Collaboration across companies. Subcontractors do not get P6 logins, and you would not want them writing to your programme if they did</li>
</ul>



<h2 class="wp-block-heading">Primavera Unifier: a business process engine that happens to know about projects</h2>



<p class="wp-block-paragraph">Unifier is the one people describe worst, usually as &#8220;the cost one&#8221;. That undersells it and also sets bad expectations.</p>



<p class="wp-block-paragraph">Unifier is a configurable workflow platform for capital programmes. You define business processes: a change order, a funding request, an invoice, a payment application, a site instruction. Each one has a form, a workflow with approval steps and routing rules, and a defined effect on the cost sheet. Oracle ships preconfigured processes and a no-code designer called uDesigner for changing them or building your own. Alongside the business processes sit the managers, including the Cost Manager, Schedule Manager and Document Manager.</p>



<p class="wp-block-paragraph">It&#8217;s sold as base products, principally Project Controls and Facilities and Asset Management, which matters because Unifier&#8217;s remit runs past handover into operating the asset.</p>



<p class="wp-block-paragraph">The honest trade-off with Unifier is this: its configurability is the reason to buy it and the reason implementations run long. A platform that can model any approval workflow will not tell you what your approval workflow should be. If your organisation has not written its cost control process down, Unifier will faithfully automate the argument you are having instead of the process you wanted.</p>



<h3 class="wp-block-heading">Where Unifier wins</h3>



<ul class="wp-block-list">
<li>Owner-side capital programme control, where budget, funding, commitments, forecast and actuals need to roll up across many projects</li>

<li>Anywhere approvals need an audit trail with defined routing, delegation and thresholds</li>

<li>Public sector and utility programmes with fund tracking, where money has strings attached and the strings must be reportable</li>

<li>Organisations that want one governed process applied to every project rather than per-project improvisation</li>
</ul>



<h3 class="wp-block-heading">Where Unifier doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Supply chain collaboration. It&#8217;s your instance, your governance, your seats. External parties can be given access, but they are guests in your system and they behave like it</li>

<li>Drawing and model control at scale. The Document Manager is real, but it is not a common data environment and it is not built for design coordination</li>

<li>Fast starts. Configuration is a project in its own right, and treating it as an install is the most common way to burn a first year</li>
</ul>



<h2 class="wp-block-heading">Aconex: the project record, owned by nobody in particular</h2>



<p class="wp-block-paragraph">Aconex is a common data environment for correspondence, documents, workflows and models across every organisation on a project. Document control, transmittals, RFIs, submittals, drawing registers, model coordination, field inspections.</p>



<p class="wp-block-paragraph">Its defining property is not a feature. It is the data ownership model. Every organisation on the project gets its own workspace and controls what it shares and when. That neutrality is the entire commercial argument, and it is the thing you cannot replicate by inviting subcontractors into your instance of something else.</p>



<p class="wp-block-paragraph">This matters more than it sounds. A designer who believes the main contractor can read their internal drafts will keep their real work somewhere else, and your &#8220;single source of truth&#8221; quietly becomes a partial one. The reason Aconex accumulates a complete project record is that nobody is afraid of it. Oracle&#8217;s own materials lean on that neutrality hard, and in this case the marketing claim and the engineering consequence are the same thing.</p>



<p class="wp-block-paragraph">On the model side, Aconex Model Coordination is design-neutral and built on open BIM standards including IFC and BCF, so it sits alongside the authoring tools rather than replacing them.</p>



<h3 class="wp-block-heading">Where Aconex wins</h3>



<ul class="wp-block-list">
<li>Multi-party projects where the correspondence record will be read by lawyers later</li>

<li>Design-build and joint venture delivery, where no single party should hold everyone&#8217;s data</li>

<li>Document and model volume, which Oracle&#8217;s datasheets describe as unbounded on data and participants</li>

<li>Handover, because the audit trail and the linked document set are the deliverable</li>
</ul>



<h3 class="wp-block-heading">Where Aconex doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>Portfolio-level capital planning. Aconex is organised around projects, not around a programme of investment decisions</li>

<li>Asset lifecycle after handover. The record ends where the project ends</li>

<li>Deep enterprise cost governance. Connected Cost exists and is genuinely useful for project cost, but if your organisation needs funding, capital planning and multi-year rollup, that is Unifier&#8217;s shape</li>
</ul>



<h2 class="wp-block-heading">Textura: money leaving the building</h2>



<p class="wp-block-paragraph">Textura came into Oracle by acquisition and still feels different from the rest of the suite, because it is solving a different class of problem.</p>



<p class="wp-block-paragraph">Textura Payment Management runs the construction payment cycle: initiating a draw, subcontractor pay applications, markups and approvals, owner billing, collecting lien waivers and compliance documents, enforcing compliance holds, payment authorisation and disbursement. It&#8217;s a network product. Your subcontractors transact in it, and the value comes from them being there.</p>



<p class="wp-block-paragraph">The compliance side is the part that decides whether it&#8217;s worth buying. If your risk is a subcontractor&#8217;s insurance lapsing quietly and nobody noticing until a claim, an automated payment hold tied to a missing document is a control you cannot easily build with a spreadsheet and good intentions.</p>



<h3 class="wp-block-heading">Where Textura wins</h3>



<ul class="wp-block-list">
<li>General contractors with a wide subcontractor base and a monthly draw cycle that eats accounts payable alive</li>

<li>Jurisdictions with real lien exposure, where waiver collection is a legal control rather than paperwork</li>

<li>Compliance enforcement at the point of payment, which is the only moment anyone reliably pays attention</li>

<li>Audit trails on who approved what, signed what and released what</li>
</ul>



<h3 class="wp-block-heading">Where Textura doesn&#8217;t</h3>



<ul class="wp-block-list">
<li>It is not a cost control system. It handles the payment leg, not the budget, forecast or commitment ledger</li>

<li>It does not replace your ERP. It integrates with it, and that integration is part of the implementation cost</li>

<li>Adoption is a supply chain problem, not a software problem. If your subcontractors will not onboard, you own a slower process than the one you replaced</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The overlap trap, and why the cost module is where it bites</h2>



<p class="wp-block-paragraph">Here&#8217;s the failure mode that is invisible at procurement and expensive eighteen months later.</p>



<p class="wp-block-paragraph">Three of these products can hold cost. P6 has cost-loaded activities. Unifier has a full Cost Manager. Aconex has Connected Cost. Nobody sets out to run three cost systems, but it happens gradually: the schedule gets cost-loaded because the client asked for an S-curve, the finance team runs the commitment ledger in Unifier, and the project team starts logging variations in Aconex because that&#8217;s where the correspondence already is.</p>



<p class="wp-block-paragraph">Now you have three forecasts. They disagree. Not because anyone is wrong, but because they are answering different questions with different cut-off dates and different accrual rules. And the reconciliation lands in a spreadsheet maintained by one person, which is the outcome every one of these products was bought to prevent.</p>



<p class="wp-block-paragraph">The fix is a decision, not a tool. Before anything is configured, write down one sentence per data class: which system is authoritative, and what the others are allowed to hold.</p>



<ul class="wp-block-list">
<li><strong>Schedule dates and logic:</strong> P6 is authoritative. Everything else receives dates, nothing else edits them</li>

<li><strong>Budget, commitments and forecast:</strong> pick one. If you own Unifier, it is Unifier. If you don&#8217;t, Connected Cost is a reasonable home, but say so out loud</li>

<li><strong>Correspondence and controlled documents:</strong> Aconex if it&#8217;s on the project, and if it is, do not run a parallel drawing register anywhere else</li>

<li><strong>Subcontractor payment and compliance status:</strong> Textura, with the resulting actuals flowing to whichever system owns cost</li>

<li><strong>Everything the authoritative system does not own:</strong> read-only, refreshed by integration, and clearly labelled in reports as a copy</li>
</ul>



<p class="wp-block-paragraph">That list takes an afternoon and saves a year. It&#8217;s also the single most common thing missing from Oracle Primavera implementations I get asked to look at.</p>



<h2 class="wp-block-heading">Integration is the real cost, and it has a shape</h2>



<p class="wp-block-paragraph">Once you own two of these, you own an integration. Budget for it as a workstream, not a checkbox.</p>



<p class="wp-block-paragraph">Oracle offers more than one path. <strong>Primavera Gateway</strong> is the established broker for P6 and Unifier, with built-in providers per application and a canonical intermediate format so mappings survive version changes. In the P6 to Unifier direction, Gateway maps the P6 Project ID to Unifier&#8217;s Project Number, and what flows depends on how the schedule is built: a duration-based schedule sends durations, a resource-loaded one sends durations and units. Oracle also has a lighter integration platform for exchanging data between Unifier, P6, Aconex and Primavera Cloud, so check which one your version and deployment model actually supports before designing anything.</p>



<p class="wp-block-paragraph">Aconex Connected Cost takes a different route, talking to P6 over its web services API. Two constraints from that integration are worth knowing before you draw an architecture diagram:</p>



<ul class="wp-block-list">
<li>You can configure both a P6 and a Primavera Cloud integration for the organisation, but an individual project can only link to one of them at a time</li>

<li>Self-hosted or third-party-hosted P6 servers have to be approved by Oracle and reachable over the internet before the integration can be established. Oracle-hosted P6 skips that approval step</li>
</ul>



<p class="wp-block-paragraph">That second one has teeth if you are running P6 EPPM on your own infrastructure. &#8220;Internet accessible&#8221; and &#8220;exposed&#8221; are not the same thing, and the gap between them is where your security review lives. If you&#8217;re standing up middleware or a reverse proxy in front of an on-premise P6 or Unifier deployment, that box needs to be treated as production: a dedicated host from a provider like Contabo or InterServer rather than a corner of an existing server, TLS terminated properly, access restricted to known networks, and remote administrative access over a business VPN such as NordLayer or Surfshark rather than a port forward someone opened for a consultant and forgot about.</p>



<p class="wp-block-paragraph">Back up the database host properly too, and test a restore. On-premise P6 and Unifier estates are frequently the last thing anyone thinks about until they need it, and imaging tools from vendors like O&amp;O Software earn their keep on exactly this kind of long-lived, rarely-touched server.</p>



<h2 class="wp-block-heading">How I&#8217;d decide</h2>



<p class="wp-block-paragraph">Work through this in order and stop at the first answer that fits. Buying more than you need is far more common than buying too little.</p>



<ol class="wp-block-list">
<li><strong>Name the pain in one sentence.</strong> &#8220;We cannot defend our programme in a delay claim&#8221; points at P6. &#8220;We cannot tell the board what the capital programme will cost&#8221; points at Unifier. &#8220;We cannot find the approved revision&#8221; points at Aconex. &#8220;Subcontractor payments take six weeks and compliance is a filing cabinet&#8221; points at Textura</li>

<li><strong>Count the organisations that must write to the system.</strong> One, and you are choosing between P6 and Unifier. Many, and you are choosing between Aconex and Textura</li>

<li><strong>Ask whether the problem outlives the project.</strong> Assets, maintenance and multi-year capital planning are Unifier&#8217;s territory. Anything that ends at handover is not</li>

<li><strong>Check what the contract already mandates.</strong> Plenty of clients name a scheduling tool or a project CDE in the contract documents. That decision is made, so spend your effort elsewhere</li>

<li><strong>Ask the reseller how each product meters.</strong> Per named user, per project, per transaction or per value processed changes the architecture, not just the invoice. A per-user product pushes you towards fewer, more senior users, and that changes who does data entry</li>

<li><strong>Only then decide on a second product,</strong> and write the system-of-record sentence for every data class the two of them share before signing anything</li>
</ol>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<ul class="wp-block-list">
<li><strong>&#8220;Unifier can do document control, so we don&#8217;t need Aconex.&#8221;</strong> Technically true, practically false on multi-party projects. The obstacle is not features, it is that other companies will not put their real correspondence in your instance</li>

<li><strong>&#8220;Aconex has cost, so we don&#8217;t need Unifier.&#8221;</strong> Fine for project-level cost. It falls down when you need portfolio funding, capital planning and asset lifecycle</li>

<li><strong>&#8220;Primavera Cloud is just P6 in the browser.&#8221;</strong> Oracle itself states that Primavera Cloud is not P6. Treat any migration as a data model exercise with a gap analysis, not a lift and shift</li>

<li><strong>&#8220;We&#8217;ll integrate it later.&#8221;</strong> Later means after users have created data under two incompatible breakdown structures. Agree the project and cost breakdown codes on day one, even if the integration itself waits</li>

<li><strong>&#8220;The suite integrates out of the box.&#8221;</strong> Connectors exist and they are good. Mapping your codes, your workflow states and your calendars is still work, and it is work that needs someone who understands both sides</li>
</ul>



<h2 class="wp-block-heading">Practices worth adopting early</h2>



<ul class="wp-block-list">
<li>Standardise the project ID before anything else. Gateway maps P6&#8217;s Project ID to Unifier&#8217;s Project Number, and a free-text naming convention will hurt on the day you connect them</li>

<li>Keep the schedule in one place and let it flow outward. Bidirectional date editing across two systems is a support ticket generator</li>

<li>Pilot Unifier configuration on one real process end to end before configuring twenty. uDesigner makes it easy to build the wrong thing quickly</li>

<li>Onboard the supply chain before go-live on Aconex or Textura. Network products fail on adoption, not on uptime</li>

<li>Export your own data on a schedule. XER and PMXML for schedules, and regular extracts from the cloud products, so you are not negotiating from zero at renewal</li>

<li>Write the system-of-record statement into the implementation charter, where people will actually see it</li>
</ul>



<h2 class="wp-block-heading">FAQ</h2>



<h3 class="wp-block-heading">What is the difference between Primavera P6 and Unifier?</h3>



<p class="wp-block-paragraph">P6 is a critical path scheduling tool that answers questions about time and resources. Unifier is a configurable business process and cost platform that answers questions about money, approvals and governance across a capital programme. They overlap slightly because P6 can hold cost data and Unifier has a Schedule Manager, but neither is a substitute for the other.</p>



<h3 class="wp-block-heading">Do I need Aconex if I already have Unifier?</h3>



<p class="wp-block-paragraph">If the project involves several independent organisations exchanging drawings and correspondence, usually yes. Unifier&#8217;s Document Manager works well for your own records, but external parties are guests in your instance. Aconex gives every organisation its own workspace and control over what it shares, which is what makes the project record complete rather than partial.</p>



<h3 class="wp-block-heading">Is Oracle Primavera Cloud replacing P6?</h3>



<p class="wp-block-paragraph">Oracle positions Primavera Cloud as the cloud platform and publishes migration guidance from P6, while continuing to sell and support P6 EPPM. Oracle also states plainly that Primavera Cloud is not P6. Plan on the two coexisting for now, and treat any migration as a data model exercise with a proper gap analysis rather than a hosting change.</p>



<h3 class="wp-block-heading">Can Textura replace our accounting system?</h3>



<p class="wp-block-paragraph">No. Textura handles the construction payment cycle: pay applications, approvals, lien waivers, compliance holds and disbursement. It integrates with ERP systems rather than replacing them, and that integration is a real line item in the implementation.</p>



<h3 class="wp-block-heading">Which product should a general contractor start with?</h3>



<p class="wp-block-paragraph">It depends on which pain is loudest. If schedules are being disputed, P6. If the project record is scattered across email, Aconex. If the monthly draw cycle and subcontractor compliance are consuming the finance team, Textura. Contractors reach for Unifier less often than owners do, because Unifier&#8217;s strengths are programme-level funding and governance.</p>



<h3 class="wp-block-heading">How hard is it to integrate P6 with Unifier?</h3>



<p class="wp-block-paragraph">The mechanism is well trodden through Primavera Gateway, which ships providers for both. The hard part is not the connector, it&#8217;s agreement: matching project identifiers, deciding which system owns dates, and reconciling breakdown structures that were designed independently. Expect the mapping work to take longer than the technical setup.</p>



<h3 class="wp-block-heading">What data should I be exporting for portability?</h3>



<p class="wp-block-paragraph">For schedules, take regular XER and PMXML exports, since those formats are what independent analysis tools and delay experts read. For the cloud products, set up scheduled extracts of documents, transaction records and audit trails. The point is not distrust of the vendor, it&#8217;s that a project record you cannot read without an active subscription is a liability during a dispute.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">The <strong>Primavera P6 vs Unifier vs Aconex</strong> comparison, with Textura alongside them, is not really a features question. Each of these products is good at the thing it was built for, and Oracle is not trying to trick anyone by giving three of them a cost module.</p>



<p class="wp-block-paragraph">The damage comes from owning two of them and never deciding which one is authoritative. Sort the products by whose record they hold and how many organisations write to them, pick the smallest set that covers your loudest pain, and write down the system of record for every data class before a single workflow gets configured.</p>



<p class="wp-block-paragraph">Do that and the monthly report matches the cost system. Skip it and you&#8217;ll meet the spreadsheet.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help untangling this?</h2>



<p class="wp-block-paragraph">Most of the work in this area is not clicking through screens, it&#8217;s the plumbing and the decisions around it. That part I can help with:</p>



<ul class="wp-block-list">
<li>Writing the system-of-record map for your data classes before you commit to a second Oracle product, so cost and dates have one owner each</li>

<li>Designing and building integrations between P6, Unifier, Aconex, Textura and your ERP, including the field mapping and error handling nobody scopes</li>

<li>Hardening on-premise P6 EPPM or Unifier deployments that need to be reachable for a cloud integration, with proper TLS, network restrictions and audited access</li>

<li>Automating schedule and cost data extraction into a warehouse or BI layer, so reporting stops depending on someone manually exporting on a Friday</li>

<li>Building repeatable backup, restore-testing and disaster recovery for self-hosted Primavera estates</li>

<li>Second-opinion reviews on an implementation or integration design that already exists and is not behaving</li>
</ul>



<p class="wp-block-paragraph">If you&#8217;re stuck on something concrete, send it over: an integration mapping spec, a Gateway error, a network diagram for an on-premise P6 that needs to talk to a cloud service. Easier to give you something useful when I can see the actual thing.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/devops/primavera-p6-vs-unifier-vs-aconex/">Primavera P6 vs Unifier vs Aconex vs Textura: Which Oracle Product Do You Actually Need?</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>Designing a Construction Dispute Intelligence Platform That Survives Cross-Examination</title>
		<link>https://john-nessime.com/blog/technical-guides/construction-dispute-intelligence-platform/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Thu, 13 Aug 2026 20:19:52 +0000</pubDate>
				<category><![CDATA[Construction Technology]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Architecture]]></category>
		<category><![CDATA[As-Built Schedule]]></category>
		<category><![CDATA[Audit Logging]]></category>
		<category><![CDATA[Bates Numbering]]></category>
		<category><![CDATA[Capital Projects]]></category>
		<category><![CDATA[Chain of Custody]]></category>
		<category><![CDATA[Common Data Environment]]></category>
		<category><![CDATA[Data Lineage]]></category>
		<category><![CDATA[Delay Analysis]]></category>
		<category><![CDATA[Document Processing]]></category>
		<category><![CDATA[Embeddings]]></category>
		<category><![CDATA[Extension of Time]]></category>
		<category><![CDATA[Forensic Schedule Analysis]]></category>
		<category><![CDATA[Human In The Loop]]></category>
		<category><![CDATA[Hybrid Search]]></category>
		<category><![CDATA[Intelligent Document Processing]]></category>
		<category><![CDATA[Legal Tech]]></category>
		<category><![CDATA[Litigation Hold]]></category>
		<category><![CDATA[Load Files]]></category>
		<category><![CDATA[Metadata Filtering]]></category>
		<category><![CDATA[OCR]]></category>
		<category><![CDATA[Oracle Aconex]]></category>
		<category><![CDATA[pgvector]]></category>
		<category><![CDATA[PostgreSQL]]></category>
		<category><![CDATA[Primavera P6]]></category>
		<category><![CDATA[Project Controls]]></category>
		<category><![CDATA[Provenance]]></category>
		<category><![CDATA[RAG]]></category>
		<category><![CDATA[Reciprocal Rank Fusion]]></category>
		<category><![CDATA[S3 Object Lock]]></category>
		<category><![CDATA[Schema Design]]></category>
		<category><![CDATA[System of Record]]></category>
		<category><![CDATA[Tenant Isolation]]></category>
		<category><![CDATA[Vector Database]]></category>
		<category><![CDATA[XER Files]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=336</guid>

					<description><![CDATA[<p>A layer-by-layer architecture for a construction dispute intelligence platform: hashed intake and immutable originals, OCR with page coordinates, a versioned Primavera P6 schedule plane, an event model that joins documents to activities, hybrid retrieval on pgvector, and a generation layer bounded by human confirmation. Written around what each layer prevents.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/construction-dispute-intelligence-platform/">Designing a Construction Dispute Intelligence Platform That Survives Cross-Examination</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The question that kills these systems is always the same one, and it always comes from the expert, not the engineer: &#8220;Where did this date come from?&#8221;</p>



<p class="wp-block-paragraph">You have a screen showing a beautiful chronology. Two hundred events, colour-coded by discipline, cross-referenced against the programme. The delay expert points at row 47, the one that says the piling rig arrived late, and asks which document that came from, who produced it, what page, and whether the other side has seen it. If the answer involves anyone opening a folder and searching, the platform has already failed. Not technically. Evidentially. A chronology nobody can trace back to a produced document is a nice picture, and a nice picture is worth nothing in an adjudication.</p>



<p class="wp-block-paragraph">That is the constraint that shapes everything else. A construction dispute intelligence platform is not a search engine with a construction theme on it. It is a provenance system that happens to do search. Get that ordering wrong and you build something impressive that no expert will sign their name to.</p>



<p class="wp-block-paragraph">This post walks through the architecture layer by layer: intake and chain of custody, normalization, the schedule data plane, the chronology model that joins the two, retrieval, and the tightly bounded generation layer on top. For each one I&#8217;ll say what it prevents, because that is the only reason any of these layers exist.</p>



<h2 class="wp-block-heading">What a construction dispute intelligence platform actually has to do</h2>



<p class="wp-block-paragraph">Strip away the vocabulary and the job is narrow. Take a pile of heterogeneous records from a capital project, some of them structured and some of them scanned faxes, and let a small number of expensive humans answer three questions quickly and defensibly:</p>



<ul class="wp-block-list">
<li>What happened, in what order, and who said so at the time?</li>



<li>What did the programme say before and after each of those things happened?</li>



<li>Which contemporaneous record supports every single assertion in the report?</li>
</ul>



<p class="wp-block-paragraph">The third one is not a nice-to-have. Both of the industry references people actually argue from lean on it. The Society of Construction Law Delay and Disruption Protocol raised record keeping to a core principle in its second edition and stopped expressing a preference between retrospective delay analysis methods, on the reasoning that the records you have largely determine the method available to you. AACE International&#8217;s Recommended Practice 29R-03 codifies forensic schedule analysis into a taxonomy of Method Implementation Protocols, and its minimum implementation protocols are essentially statements about which source schedules you must be able to produce. Both documents are, underneath, about evidence handling. So is your architecture.</p>



<h2 class="wp-block-heading">Layer one: intake that a forensic expert can defend</h2>



<p class="wp-block-paragraph">Everything upstream of this layer is somebody else&#8217;s mess. Site photos on a phone, a shared drive nobody pruned since mobilization, an Aconex export, a hard drive that arrived in an envelope. Your job at intake is to make an irreversible statement about what you received and when, before anyone touches it.</p>



<p class="wp-block-paragraph">Hash every file on arrival and store the hash next to the object, not in a spreadsheet. SHA-256 over the raw bytes, computed before any conversion, deduplication or renaming. The hash becomes the document&#8217;s identity for the rest of its life.</p>



<pre class="wp-block-code"><code># Build an arrival manifest before anything else runs
find ./ACME-INTAKE-001 -type f -print0 
  | xargs -0 sha256sum &gt; manifest-ACME-INTAKE-001.txt

# Re-verify later, after the files have moved through processing
sha256sum -c manifest-ACME-INTAKE-001.txt</code></pre>



<p class="wp-block-paragraph">Then put the originals somewhere they cannot be quietly edited. On AWS, S3 Object Lock in compliance mode with a retention period is the blunt instrument: once set, the object version cannot be overwritten or deleted by anyone, including the root account, until retention expires. Governance mode is the softer variant, where a specifically permissioned principal can lift the lock, which is fine for internal working copies and wrong for originals. If you are self-hosting on a Contabo or InterServer box because the client will not put project data in a hyperscaler, the equivalent discipline is a write-once staging volume plus an append-only manifest log, and you accept that you are relying on process rather than on the storage layer refusing the write.</p>



<h3 class="wp-block-heading">The manifest is the product</h3>



<p class="wp-block-paragraph">If the matter is already in formal disclosure, you will be handed productions rather than raw folders, and those arrive as load files. A Concordance DAT carries one row per document with fields like BegBates, EndBates, Custodian, BegAttach, EndAttach and an MD5 hash. An Opticon OPT carries one row per page, mapping Bates numbers to image paths. The delimiters are deliberately weird characters that real text rarely contains, and different tools disagree about the exact code points, so parse defensively and log what you detected rather than assuming.</p>



<p class="wp-block-paragraph">The failure mode here is quiet and expensive. Variable-width Bates numbers sort lexically instead of numerically, so ABC10 lands between ABC1 and ABC2, attachment ranges resolve to the wrong parents, and a family of documents silently splits. Nobody notices until an expert cites a covering email whose attachment, in your system, belongs to a different letter. Pad to fixed width on ingest and reconcile every BegAttach range against the documents you actually loaded.</p>



<h2 class="wp-block-heading">Layer two: normalization, and the OCR tax nobody budgets for</h2>



<p class="wp-block-paragraph">Construction corpora are the worst kind of mixed. Native Office files with clean text, born-digital PDFs, scanned site instructions with a signature and a coffee ring, photographs of whiteboards, and drawings where the only text that matters is inside a title block in the bottom right corner.</p>



<p class="wp-block-paragraph">Route by what the file actually is, not by extension. For a PDF, try text extraction first; if the extracted layer is empty or nearly empty, it is a scan and needs OCR. Tesseract via OCRmyPDF is the cheap path and it is genuinely good on clean typed pages. It degrades badly on handwriting, on stamps, and on anything skewed, which describes most site paperwork. A managed service such as Amazon Textract handles tables and forms better and returns geometry for every detected block, which matters more than accuracy alone.</p>



<p class="wp-block-paragraph">Geometry is the part teams skip and then rebuild six months later. If you store only the extracted string, you can prove a phrase exists somewhere in a document. If you store the page number and the bounding box, you can put a highlight on the exact line in the viewer, and the expert can confirm it in two seconds instead of reading eleven pages. That difference is the difference between a tool people use and a tool people abandon.</p>



<p class="wp-block-paragraph">Keep the OCR text separate from the original in storage, and record which engine and which settings produced it. When a disputed reading of a handwritten date turns up, and it will, you want to be able to say exactly how that character was produced.</p>



<h2 class="wp-block-heading">Layer three: the schedule plane is a separate data model</h2>



<p class="wp-block-paragraph">This is where most teams underestimate the work. Programme data is not documents. It is a graph, versioned, with a calendar attached, and it needs its own tables, its own ingest path and its own validation.</p>



<p class="wp-block-paragraph">Primavera P6 exports as XER, a tab-delimited text format with a table-per-section layout, or as P6 XML. Either way you are pulling out projects, the WBS tree, activities, logic links and calendars. In XER these live in sections including PROJECT, PROJWBS, TASK, TASKPRED and CALENDAR. Parsing it by hand is a bad use of a week; PyP6Xer is an open source Python parser that will get you to objects quickly.</p>



<pre class="wp-block-code"><code>from xerparser.reader import Reader

xer = Reader("update-14.xer")

for project in xer.projects:
    print(project.proj_short_name)
    for activity in project.activities:
        print(activity.task_code, activity.task_name)</code></pre>



<p class="wp-block-paragraph">Two things bite immediately. First, several duration and float fields in XER are stored in hours rather than days, which you can see in the column names themselves. If you load them straight into a &#8220;days&#8221; column because the numbers looked plausible on a five-day calendar, every float figure downstream is wrong by a factor that varies with the calendar. Check your own export before you trust any mapping, including this one.</p>



<p class="wp-block-paragraph">Second, activity IDs are not stable across a project&#8217;s life. Activities get renumbered, split, deleted and re-added between updates. If your schema keys on the activity ID alone, two updates will silently merge activities that were never the same work. Key on a composite of schedule version plus activity ID, and maintain a separate mapping table for identity across versions that a human can correct. That table is boring, it is not automatable, and it is the difference between a windows analysis that holds together and one that quietly compares apples to a different orchard.</p>



<p class="wp-block-paragraph">Store every update as an immutable snapshot with its data date. Never mutate a loaded schedule. The whole point of the layer is that you can reconstruct exactly what the programme said on a given day, which is precisely what a time slice or windows method needs.</p>



<h2 class="wp-block-heading">Layer four: the chronology is the join, and the join is the hard part</h2>



<p class="wp-block-paragraph">Documents on one side, schedule snapshots on the other. The product is the thing in the middle: an event, with a date, a description, a set of supporting document references, and a set of affected activities.</p>



<p class="wp-block-paragraph">Model it explicitly. An event is not a document and it is not an activity. One letter can create three events. One event can be evidenced by nine documents and a photograph. The relationship is many-to-many in both directions and every attempt I have seen to flatten it ends with someone maintaining a spreadsheet on the side.</p>



<pre class="wp-block-code"><code>-- Every assertion resolves to a page and a box on that page
CREATE TABLE evidence_ref (
  id            BIGSERIAL PRIMARY KEY,
  event_id      BIGINT NOT NULL REFERENCES event(id),
  document_id   UUID   NOT NULL REFERENCES document(id),
  page_number   INT    NOT NULL,
  bbox          JSONB,             -- normalized coordinates from OCR
  quoted_text   TEXT   NOT NULL,
  asserted_by   TEXT   NOT NULL,   -- 'analyst' or 'model'
  confirmed_by  TEXT,              -- NULL until a human confirms
  confirmed_at  TIMESTAMPTZ
);</code></pre>



<p class="wp-block-paragraph">Those last three columns are the ones that matter. They encode the difference between something the machine proposed and something a person stands behind. Report generation reads only rows where <code>confirmed_by</code> is set. Everything else is a suggestion queue. This is not a compliance box; it is the mechanism by which an expert can put their name on the output, and it needs to exist in the schema rather than in a workflow convention that erodes under deadline.</p>



<h2 class="wp-block-heading">Layer five: retrieval that finds document numbers, not just meaning</h2>



<p class="wp-block-paragraph">Pure vector search is the wrong default for this corpus, and it fails in a way that looks like success. Embeddings are good at meaning and bad at tokens. Ask for &#8220;RFI 0442&#8221; and a dense retriever will cheerfully hand you eleven chunks about requests for information, none of which is RFI 0442, because the identifier got smeared into its semantic neighbourhood. Construction disputes run on identifiers: RFI numbers, variation order references, drawing revisions, activity IDs, Bates ranges.</p>



<p class="wp-block-paragraph">Run lexical and semantic retrieval in parallel and fuse the rankings. PostgreSQL with the pgvector extension does both without adding infrastructure: an HNSW index over the embedding column, a GIN index over a generated tsvector column, and reciprocal rank fusion to merge the two candidate lists.</p>



<pre class="wp-block-code"><code>CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunk (
  id          BIGSERIAL PRIMARY KEY,
  document_id UUID NOT NULL,
  page_number INT  NOT NULL,
  body        TEXT NOT NULL,
  body_tsv    tsvector GENERATED ALWAYS AS
                (to_tsvector('english', body)) STORED,
  embedding   vector(1536)
);

CREATE INDEX chunk_tsv_idx  ON chunk USING gin (body_tsv);
CREATE INDEX chunk_hnsw_idx ON chunk USING hnsw (embedding vector_cosine_ops);</code></pre>



<p class="wp-block-paragraph">Over-fetch from each side and trim after fusion rather than pulling the final count from each; more candidates give the fusion step something to work with. Add a trigram index if your corpus has OCR noise, because &#8220;RFI 0442&#8221; and &#8220;RFl 0442&#8221; are the same document to a human and different tokens to Postgres.</p>



<p class="wp-block-paragraph">Filter hard on metadata before ranking. Matter, custodian, date range, privilege status. A retrieval layer that can return a privileged document into a shared workspace is a liability, not a feature.</p>



<h2 class="wp-block-heading">Layer six: generation, bounded hard</h2>



<p class="wp-block-paragraph">Language models are useful here and the useful jobs are smaller than the demos suggest: proposing candidate events from a batch of correspondence, drafting neutral one-line summaries, clustering near-duplicate records, suggesting which activities an event might touch. Every one of those is a proposal that lands in a review queue.</p>



<p class="wp-block-paragraph">The reason to be strict is not squeamishness. Stanford&#8217;s RegLab and HAI ran a preregistered evaluation of commercial legal research tools built on retrieval-augmented generation, with expert-scored answers, and found hallucination rates in the range of roughly 17 to 33 percent depending on the product, against vendor marketing that had claimed hallucination-free citations. The vendors disputed the characterization. The finding that should worry you more than the headline number is the failure mode underneath it: a citation that exists, is quoted accurately, and does not stand for what the answer says it stands for. Retrieval reduces fabrication. It does not produce correct reasoning about what a document means, and misgrounding is far harder to spot than an invented reference.</p>



<p class="wp-block-paragraph">So bound it. Generation writes to the proposal table, never to the report. Every generated assertion carries the chunk IDs it was built from, and the UI shows the source page beside the claim by default rather than behind a click. If a proposed event has no evidence reference, it does not enter the queue at all.</p>



<h2 class="wp-block-heading">Tenancy, privilege, and the thing that ends the engagement</h2>



<p class="wp-block-paragraph">Matters do not share data. Ever. Not &#8220;logically separated by a WHERE clause that we always remember to add&#8221;. The cheapest defensible pattern is a database per matter, or at minimum a schema per matter with connection-level scoping, so that a missing predicate produces an empty result rather than another client&#8217;s correspondence.</p>



<p class="wp-block-paragraph">Two more that get discovered late. Privilege state is per document per matter and it changes; model it as a versioned attribute with an audit trail, not a boolean someone flipped. And plan the exit at the start: matters end, retention clocks expire, and someone has to certify destruction. That means knowing every copy you made, including OCR text, thumbnails, embeddings and backups. Tools like the O&amp;O Software utilities have a place at the endpoint end of that job; the platform side is a deletion manifest that enumerates derived artifacts per document, written at ingest rather than reconstructed under pressure.</p>



<p class="wp-block-paragraph">If experts are working from home, from site cabins, or from three countries, the access path is part of the design. A managed VPN such as NordVPN or Surfshark on the client side plus network-level allowlisting on the platform side is a pragmatic combination, and it is worth writing into the engagement documentation so nobody improvises with a personal Dropbox at eleven at night.</p>



<h2 class="wp-block-heading">Where these platforms break in practice</h2>



<ul class="wp-block-list">
<li><strong>Chronology dates drift from document dates.</strong> Usually a timezone or a date-format collision: a scanned form read as DD/MM, an email header parsed as UTC, a schedule data date stored as a naive timestamp. Store the raw string you extracted alongside the parsed value and show both in the review UI.</li>



<li><strong>Float figures look wrong by a constant factor.</strong> Hours versus days in the schedule import. Check against the same activity opened in P6 before you debug anything else.</li>



<li><strong>Retrieval misses a document everyone knows exists.</strong> Either it was never OCR&#8217;d because the PDF had a thin text layer that passed your emptiness check, or the identifier is only in the title block of a drawing. Log extraction character counts per page and alert on suspiciously low ones.</li>



<li><strong>Two activities merged across updates.</strong> Composite keying was not applied, or an activity was renumbered. Diff activity code sets between consecutive snapshots and surface additions and deletions as a report the planner reviews.</li>



<li><strong>Document families split.</strong> Bates padding or an unreconciled attachment range from the load file. Re-run the family reconciliation and count orphaned attachments.</li>
</ul>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating the schedule as documents. It is a graph with versions, and shoehorning it into the document store costs you every analysis you actually wanted.</li>



<li>Storing extracted text without page and coordinate provenance. Cheap now, a full reprocessing run later.</li>



<li>Vector-only retrieval. Looks great in a demo built from prose, falls over on the first RFI number.</li>



<li>Letting model output reach a deliverable without a recorded human confirmation.</li>



<li>Mutating loaded schedules to &#8220;fix&#8221; them. Fix on the way in, or hold the correction as an annotation. Never edit the snapshot.</li>



<li>Deferring the deletion story to the end of the matter, when the derived artifacts are already scattered.</li>
</ul>



<h2 class="wp-block-heading">Best practices for building a construction dispute intelligence platform</h2>



<ul class="wp-block-list">
<li>Hash at the door, lock the originals, and never process in place.</li>



<li>Make every derived artifact point back to a document ID, a page and a region.</li>



<li>Give schedules their own immutable, version-keyed store with the data date as a first-class field.</li>



<li>Model events separately from both documents and activities, with many-to-many links in each direction.</li>



<li>Run hybrid retrieval and fuse the rankings; filter on matter and privilege before you rank.</li>



<li>Keep generation in a proposal queue with confirmation recorded in the schema.</li>



<li>Isolate matters at the database or schema boundary, not in application code.</li>



<li>Instrument ingest as seriously as you would a production pipeline. Grafana over your extraction counts and OCR queue depth will tell you a corpus is going wrong days before an analyst does.</li>
</ul>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Do I need a dedicated vector database for this?</h3>



<p class="wp-block-paragraph">Almost certainly not. Matter corpora are large in gigabytes and small in chunk count compared to consumer-scale search. PostgreSQL with pgvector keeps the embeddings in the same transaction as the provenance rows, which means you cannot end up with a chunk whose document record was rolled back. That consistency is worth more here than the last few milliseconds of query latency.</p>



<h3 class="wp-block-heading">Should the platform pick a delay analysis method?</h3>



<p class="wp-block-paragraph">No. That choice belongs to the expert and depends on the contract, the forum, the records available and the timing of the analysis. What the platform should do is make it obvious which methods the record set can support, by reporting what you actually have: how many schedule updates, at what intervals, with what data dates, and how complete the contemporaneous correspondence is around each window.</p>



<h3 class="wp-block-heading">XER or P6 XML for schedule ingest?</h3>



<p class="wp-block-paragraph">Accept both, because you will be sent both. XER is what planners export by habit and it is easy to parse with existing libraries. P6 XML is more explicit about structure and less prone to encoding surprises. Whichever arrives, keep the original file, hashed, alongside the parsed rows.</p>



<h3 class="wp-block-heading">How much of the review can realistically be automated?</h3>



<p class="wp-block-paragraph">The ranking and the grouping, most of it. The judgement, none of it. Automation that reduces a hundred thousand documents to a few thousand worth reading is enormously valuable and uncontroversial. Automation that decides what a letter means is where you inherit risk you cannot price.</p>



<h3 class="wp-block-heading">Can I build this on top of a common data environment we already have?</h3>



<p class="wp-block-paragraph">Partly. A CDE such as Aconex is a good source of records and a poor system of record for a dispute, because access changes, exports are point-in-time, and the retention rules belong to the project rather than to the matter. Pull from it, hash what you pulled, and hold your own copy under your own lock.</p>



<h3 class="wp-block-heading">What does this cost to run?</h3>



<p class="wp-block-paragraph">The shape of the bill is more useful than a number. OCR and embedding are one-time per document and scale with page count, so a scan-heavy corpus costs multiples of a born-digital one. Storage is small and constant. Retrieval is negligible. Generation scales with how much you push through it, which is another argument for narrow, queued use rather than a chat box over the whole corpus.</p>



<h3 class="wp-block-heading">Is a general eDiscovery platform not enough?</h3>



<p class="wp-block-paragraph">For document review, often yes, and they are considerably better at productions and privilege logging than anything you will build in a quarter. What they do not have is the schedule plane. The value of a construction-specific build is the join between correspondence and programme versions, so a sensible architecture frequently uses an established review platform for documents and adds the schedule and chronology layers alongside it.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">A construction dispute intelligence platform earns its place by making provenance cheap. Search, clustering and summarization are commodity capabilities now; anyone can bolt them onto a corpus in a fortnight. What is hard, and what decides whether an expert will rely on the output, is that every date, every event and every sentence in the eventual report resolves in one click to a specific page of a specific produced document, with a record of who confirmed it.</p>



<p class="wp-block-paragraph">Build the provenance spine first. Hash at intake, keep coordinates, version the schedules, model events as their own thing, and keep the model on the proposal side of a human confirmation. The clever features are easy to add afterwards. Retrofitting evidence handling into a system that was designed as a search tool is a rebuild, and it always arrives at the worst possible moment, which is about three weeks before a hearing.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help building or fixing this?</h2>



<p class="wp-block-paragraph">I work on the infrastructure side of exactly this problem: getting messy project records into a system where every claim is traceable. Things I can help with:</p>



<ul class="wp-block-list">
<li>Designing the intake and chain-of-custody layer, including hashing, immutable storage, and load file parsing that survives odd delimiters and inconsistent Bates padding.</li>



<li>Building the schedule ingest path for XER and P6 XML, with version-keyed snapshots and an activity identity map that holds across updates.</li>



<li>Standing up hybrid retrieval on PostgreSQL and pgvector, including page-level and coordinate-level provenance so highlights land on the right line.</li>



<li>Setting up the OCR and normalization pipeline, with routing rules, quality checks, and alerting on pages that extracted suspiciously little text.</li>



<li>Matter isolation, privilege state modelling, audit logging, and a deletion manifest you can act on at the end of the engagement.</li>



<li>Reviewing an existing platform and telling you plainly where the provenance chain breaks.</li>
</ul>



<p class="wp-block-paragraph">If you have something concrete, send it over: an XER export, a sample DAT and OPT pair, a page count breakdown of your corpus, or the schema you are currently working from. It is much easier to say something useful about a real file than about a description of one.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/construction-dispute-intelligence-platform/">Designing a Construction Dispute Intelligence Platform That Survives Cross-Examination</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
