<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Migration Assessment | John Nessime</title>
	<atom:link href="https://john-nessime.com/blog/tag/migration-assessment/feed/" rel="self" type="application/rss+xml" />
	<link>https://john-nessime.com/blog/tag/migration-assessment/</link>
	<description>Cloud, DevOps, Data &#38; AI — Built, Tested, Explained</description>
	<lastBuildDate>Sun, 08 Nov 2026 07:05:47 +0000</lastBuildDate>
	<language>en-US</language>
	<sy:updatePeriod>
	hourly	</sy:updatePeriod>
	<sy:updateFrequency>
	1	</sy:updateFrequency>
	<generator>https://wordpress.org/?v=7.1.4</generator>

<image>
	<url>https://john-nessime.com/blog/wp-content/uploads/2026/07/cropped-jn-32x32.png</url>
	<title>Migration Assessment | John Nessime</title>
	<link>https://john-nessime.com/blog/tag/migration-assessment/</link>
	<width>32</width>
	<height>32</height>
</image> 
	<item>
		<title>Oracle to Aurora PostgreSQL: A Migration Plan That Survives Contact</title>
		<link>https://john-nessime.com/blog/technical-guides/oracle-to-aurora-postgresql-migration-plan/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sun, 08 Nov 2026 07:05:47 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Databases]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Aurora]]></category>
		<category><![CDATA[AWS]]></category>
		<category><![CDATA[AWS DMS]]></category>
		<category><![CDATA[AWS SCT]]></category>
		<category><![CDATA[Change Data Capture]]></category>
		<category><![CDATA[Cutover Planning]]></category>
		<category><![CDATA[Database Migration]]></category>
		<category><![CDATA[Migration Assessment]]></category>
		<category><![CDATA[ora2pg]]></category>
		<category><![CDATA[Oracle]]></category>
		<category><![CDATA[orafce]]></category>
		<category><![CDATA[PL/SQL]]></category>
		<category><![CDATA[PostgreSQL]]></category>
		<category><![CDATA[Query Plan Management]]></category>
		<category><![CDATA[Rollback Plan]]></category>
		<category><![CDATA[Schema Conversion]]></category>
		<category><![CDATA[Supplemental Logging]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=498</guid>

					<description><![CDATA[<p>The assessment report says most objects convert automatically, and that number becomes a date on a roadmap. It measures objects, not work. Here are the six decisions that actually set the timeline of an Oracle to Aurora PostgreSQL migration, a phase sequence gated on artefacts rather than dates, and the places estimates quietly go soft.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/oracle-to-aurora-postgresql-migration-plan/">Oracle to Aurora PostgreSQL: A Migration Plan That Survives Contact</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">The date was on a slide before anyone had opened the converter.</p>



<p class="wp-block-paragraph">That is usually how it happens. A director asks how long it takes to get off Oracle, somebody runs the assessment, the report says most objects convert automatically, and that number becomes a quarter on a roadmap. Nobody lied. The number is real. It just does not measure the thing the schedule depends on.</p>



<p class="wp-block-paragraph">Here is the part that bites, and it stays invisible until about the second month of testing: the conversion percentage counts objects, not work. The objects that fail to convert are the safe ones. They get flagged, scoped and estimated. The objects that convert cleanly are where the risk lives, because a converted function compiles, runs, returns a value, and can still be quietly wrong in a way nobody notices until a finance user queries a total.</p>



<p class="wp-block-paragraph">This post is about planning around that. It covers the six decisions that set the timeline, a phase sequence that reflects how the work really lands, and the specific places an Oracle to Aurora PostgreSQL migration plan goes soft. It is not a console walkthrough. AWS documents the clicking well enough.</p>



<h2 class="wp-block-heading">Start from what the assessment report does not tell you</h2>



<p class="wp-block-paragraph">There are two supported ways to convert an Oracle schema for a PostgreSQL target. DMS Schema Conversion runs in the AWS console and CLI as a managed feature. AWS SCT is the older Java desktop application you install yourself. They are two front ends over the same conversion engine, and AWS now points most OLTP conversions at the managed one. SCT still earns its place when you want a local tool, want to script the conversion into a pipeline, or hit a path the managed feature does not cover.</p>



<p class="wp-block-paragraph">One planning input that used to exist no longer does. AWS ended support for DMS Fleet Advisor on 20 May 2026. If your last migration playbook opened with &#8220;run Fleet Advisor across the estate to find candidates,&#8221; that step is gone and the discovery work goes back to you, your CMDB, or a script that walks the data dictionary. Worth knowing before you copy an old plan forward.</p>



<p class="wp-block-paragraph">DMS Schema Conversion also has a generative AI option that proposes conversions for code objects it could not handle on its own, and it genuinely shrinks the manual pile. Be honest in the plan about what it changes though: it moves effort from writing PL/pgSQL to reviewing PL/pgSQL. Review is cheaper than authoring. It is not free, and it still needs someone who knows what the original code was meant to do.</p>



<p class="wp-block-paragraph">What no assessment report measures:</p>



<ul class="wp-block-list">
<li>Behavioural divergence in the objects that converted successfully</li>



<li>Query performance on the target, which is a different optimiser making different choices</li>



<li>SQL that lives outside the database, in application code, report definitions and shell scripts</li>



<li>Whether your source tables can support change data capture at all</li>



<li>How long it takes to get a production-shaped dataset into a test environment</li>
</ul>



<h2 class="wp-block-heading">Decision one: what moves, what waits, and what should never move</h2>



<p class="wp-block-paragraph">Not every Oracle schema is a migration candidate, and finding that out in month four is expensive.</p>



<p class="wp-block-paragraph">Vendor applications are the clearest case. If the vendor certifies Oracle and only Oracle, you are not planning a database migration, you are planning an application replacement, and that is a different project with a different budget. Decide that first and take those schemas off the board.</p>



<p class="wp-block-paragraph">The second filter is mechanical. DMS needs a primary key or unique constraint on a table to replicate changes to it. The premigration assessment flags tables that lack one, along with LOB columns that are not nullable on the target and Oracle-specific structures that complicate capture: index-organized tables, function-based indexes, global temporary tables, anything leaning on ROWID. Run it early, not as a pre-flight check the week before the load.</p>



<p class="wp-block-paragraph">You can find the primary key problem yourself in a minute, and it is worth doing before anyone commits to a date:</p>



<pre class="wp-block-code"><code>-- Tables in scope with no primary key and no unique constraint.
-- These cannot participate in DMS change data capture as-is.
SELECT t.owner, t.table_name
FROM   all_tables t
WHERE  t.owner IN ('HR', 'SALES', 'BILLING')
AND    NOT EXISTS (
         SELECT 1
         FROM   all_constraints c
         WHERE  c.owner = t.owner
         AND    c.table_name = t.table_name
         AND    c.constraint_type IN ('P', 'U'))
ORDER  BY t.owner, t.table_name;</code></pre>



<p class="wp-block-paragraph">Every row that comes back is a decision. Add a key to the source table, which is a production change on a table that may be very large. Migrate it full load only and freeze writes to it during cutover. Or leave it behind. All three are acceptable. Discovering the list after you have promised a cutover weekend is not.</p>



<h2 class="wp-block-heading">Decision two: how much Oracle you keep inside PostgreSQL</h2>



<p class="wp-block-paragraph">PostgreSQL has an extension called orafce that implements a subset of Oracle functions and packages. It is available on Aurora PostgreSQL and RDS for PostgreSQL, and the conversion tooling knows about it. There are also conversion settings that emulate Oracle behaviour directly, such as standing in for the ROWID pseudocolumn with a bigint or character column, and emulating format parameters that PostgreSQL&#8217;s own <code>TO_CHAR</code> and <code>TO_DATE</code> do not accept.</p>



<p class="wp-block-paragraph">The case for turning it all on is strong. Compatibility shims delete whole categories of rewrite. Oracle&#8217;s <code>SUBSTR</code> behaves differently from PostgreSQL&#8217;s on certain inputs, and orafce gives you the Oracle behaviour without auditing every call site. If the goal is to be off Oracle licensing by a fixed date, that trade is often correct.</p>



<p class="wp-block-paragraph">The cost is that every shim is one more thing your team has to remember is not stock PostgreSQL. Two years later somebody writes a query on a laptop against community Postgres, it behaves differently in production, and nobody can explain why. You did not leave Oracle. You brought a piece of it along and stopped labelling it.</p>



<p class="wp-block-paragraph">How I would decide: turn on compatibility where it removes an entire class of rewrite you would otherwise do by hand, and turn it off where it preserves an Oracle idiom you were specifically trying to escape. Package-level session state is the obvious one to let go of. String semantics are the obvious one to keep.</p>



<p class="wp-block-paragraph">The single divergence with the widest blast radius does not have a clean shim, and it deserves its own line item on the plan. Oracle treats an empty string as NULL. PostgreSQL does not.</p>



<pre class="wp-block-code"><code>-- Oracle: these two return the same rows.
SELECT * FROM customers WHERE notes IS NULL;
SELECT * FROM customers WHERE notes = '';

-- PostgreSQL: the second returns nothing.
-- '' is a zero-length string here, and NULL is absence of a value.
SELECT * FROM customers WHERE notes IS NULL;
SELECT * FROM customers WHERE notes = '';</code></pre>



<p class="wp-block-paragraph">That affects WHERE clauses, concatenation, unique constraints and any application code that inserts <code>''</code> expecting it to land as NULL. It is not a bug you fix once. It is an audit you schedule.</p>



<h2 class="wp-block-heading">Decision three: who rewrites the PL/SQL and how you prove it behaves</h2>



<p class="wp-block-paragraph">This is the decision that dominates the schedule, and it is usually the one with the least detail on the plan.</p>



<p class="wp-block-paragraph">Converted PL/pgSQL compiles. That is the trap. Compiling proves the syntax is legal, not that the logic survived. The cleanest example is <code>SELECT INTO</code>, which is used in nearly every Oracle procedure ever written.</p>



<pre class="wp-block-code"><code>-- PL/pgSQL: if no row matches, v_name is simply left NULL.
-- If several match, you get one of them. No exception either way.
SELECT name INTO v_name FROM customers WHERE id = p_id;

-- With STRICT, you get NO_DATA_FOUND and TOO_MANY_ROWS
-- the way Oracle raises them implicitly.
SELECT name INTO STRICT v_name FROM customers WHERE id = p_id;</code></pre>



<p class="wp-block-paragraph">An Oracle procedure that relied on <code>NO_DATA_FOUND</code> to branch now takes the happy path with a NULL in hand. It will not error. It will post a record with a missing field, skip a validation, or return zero where it should have raised. Multiply that by a few hundred procedures.</p>



<p class="wp-block-paragraph">Transaction handling is the other place assumptions break. Oracle commits DDL implicitly; PostgreSQL runs DDL inside transactions and lets you roll it back. Autonomous transactions, which Oracle code often uses to write an audit row that survives a rollback, have no direct equivalent. Both are fine once you know about them, and both silently change behaviour if you do not.</p>



<p class="wp-block-paragraph">The only test that settles this is differential: same inputs, both engines, compare outputs. Not &#8220;does the procedure run.&#8221; Capture real inputs from production before the rewrite starts, because reconstructing them from memory afterwards is how you end up testing the happy path only.</p>



<p class="wp-block-paragraph">The staffing reality underneath all of this: the people who know what those packages are supposed to do are usually Oracle developers who have maintained them for a decade, and the people fluent in PostgreSQL are usually somebody else. That gap is the real cost of the migration, and no tool closes it.</p>



<h2 class="wp-block-heading">Decision four: what you are asking the source database to give up</h2>



<p class="wp-block-paragraph">Change data capture is not something you switch on in DMS. It is a change to production Oracle that has to happen weeks before anyone migrates anything, and it needs its own milestone and its own change window.</p>



<p class="wp-block-paragraph">The source has to be in ARCHIVELOG mode. It needs supplemental logging, minimally at the database level, plus identification key logging so that updates carry enough of the row to be replayed. AWS notes that retaining archive logs for around 24 hours is usually enough when a task starts capturing immediately, and longer if you plan to start from a point in the past.</p>



<pre class="wp-block-code"><code>-- Minimal supplemental logging at the database level.
ALTER DATABASE ADD SUPPLEMENTAL LOG DATA;

-- All-column logging for a single replicated table, if you
-- would rather not carry the overhead database-wide.
ALTER TABLE hr.employees ADD SUPPLEMENTAL LOG DATA (ALL) COLUMNS;</code></pre>



<p class="wp-block-paragraph">You can also let DMS add table-level supplemental logging itself with the <code>addSupplementalLogging=Y</code> extra connection attribute, which requires the DMS user to hold ALTER on the replicated tables. That is convenient and it is also a DBA conversation, because it means a migration tool is issuing DDL against production.</p>



<p class="wp-block-paragraph">Two planning consequences fall out of this. Supplemental logging increases redo volume, which lands on backup windows and archive storage on a system that was sized before anyone mentioned Aurora. And LogMiner, which DMS uses to read redo by default, does not support connecting to a pluggable database. If the source is multitenant, you are on Binary Reader, which reads the log files directly, needs different privileges, and behaves differently around ASM and around backups that clear archive logs early.</p>



<p class="wp-block-paragraph">None of that is hard. All of it takes calendar time you have to ask another team for.</p>



<h2 class="wp-block-heading">Decision five: what &#8220;fast enough&#8221; means, decided before cutover</h2>



<p class="wp-block-paragraph">Aurora PostgreSQL&#8217;s planner is not Oracle&#8217;s optimiser. A query that ran in single-digit milliseconds for years can pick a different join order and take minutes, and it will do that on the queries you did not think to test.</p>



<p class="wp-block-paragraph">The first thing to get right is trivial and gets missed constantly. After a DMS full load, the target tables have no statistics, and plans built on no statistics are close to random. Run <code>ANALYZE</code> before you measure anything, or you will spend a week tuning a problem that does not exist.</p>



<pre class="wp-block-code"><code>-- After the full load completes, before any performance testing.
ANALYZE VERBOSE;

-- Query plan management is enabled at the cluster level first.
SHOW rds.enable_plan_management;</code></pre>



<p class="wp-block-paragraph">Aurora has a feature Oracle people will recognise: query plan management, packaged as the <code>apg_plan_mgmt</code> extension. It captures plans as the optimiser generates them, stores them in <code>apg_plan_mgmt.dba_plans</code>, and lets you approve, reject or force specific ones. New plans arrive as unapproved, and <code>apg_plan_mgmt.evolve_plan_baselines</code> compares them on real execution time so you can promote the good ones. Set <code>apg_plan_mgmt.capture_plan_baselines</code> to collect, and <code>apg_plan_mgmt.use_plan_baselines</code> to enforce.</p>



<p class="wp-block-paragraph">It is the closest thing to Oracle&#8217;s plan stability and it is genuinely useful, but be clear about what it does. It stabilises plans you have already decided are acceptable. It does not find the slow queries for you. For that you need a workload replay against the target and a baseline captured from Oracle before you started.</p>



<p class="wp-block-paragraph">Whatever you already use for database monitoring works here, whether that is CloudWatch and Performance Insights, Grafana against Prometheus exporters, Percona Monitoring and Management, or Datadog. The requirement is not a particular tool. It is that you took the Oracle baseline before you touched anything, because you cannot go back and capture it later.</p>



<h2 class="wp-block-heading">Decision six: the cutover and the way back</h2>



<p class="wp-block-paragraph">Decide the rollback story during planning, because one of the two options takes weeks to build.</p>



<p class="wp-block-paragraph">A one-way door is cheaper and perfectly legitimate for a system where a few hours of degraded service is survivable. You cut over, freeze the source, and fix forward. It just requires saying out loud that there is no going back once the first write lands on Aurora.</p>



<p class="wp-block-paragraph">Reverse replication is the alternative: a second DMS task running Aurora back to Oracle so the old database stays current and you can repoint the application. It is the only real rollback once production writes are on the new engine, and it is also a full migration in the opposite direction with its own type mappings, key requirements and testing. Budget it as a work stream, not a checkbox. The failure mode I see most often is a plan that assumes reverse replication and funds a one-way door.</p>



<h2 class="wp-block-heading">A realistic Oracle to Aurora PostgreSQL migration plan, phase by phase</h2>



<p class="wp-block-paragraph">The ordering matters more than the durations, which vary enormously with how much PL/SQL you have. Gate each phase on an artefact rather than a date.</p>



<ol class="wp-block-list">
<li><strong>Inventory and disposition.</strong> Every schema gets a decision: migrate, replace, retire, or leave. Gate: a signed list.</li>



<li><strong>Target shape and compatibility.</strong> Aurora or RDS, which shims are on, which Oracle idioms you are deliberately keeping. Gate: a written decision with reasons, because someone will ask in month five.</li>



<li><strong>Convert and apply to a scratch cluster.</strong> First contact with reality. Gate: the schema applies without manual patching.</li>



<li><strong>PL/SQL rewrite and differential testing.</strong> The longest phase by a wide margin. It runs in parallel with everything below it. Gate: input and output pairs matching across both engines.</li>



<li><strong>Source enablement.</strong> ARCHIVELOG, supplemental logging, DMS user privileges, log retention. This is a production change request. Gate: capture proven working on a single table.</li>



<li><strong>Timed full load rehearsal.</strong> You need the real duration, not an estimate, because it sets the shape of the cutover window. Gate: a measured number.</li>



<li><strong>CDC steady state and validation.</strong> Let it run for days, not hours. Gate: lag stable under production write volume.</li>



<li><strong>Performance parity.</strong> Replay the workload, fix what is slow, capture plans. Gate: the slow list is empty or explicitly accepted.</li>



<li><strong>Cutover rehearsal, twice.</strong> Once to find the gaps, once to prove you fixed them. Gate: a runbook someone else could execute.</li>



<li><strong>Cutover.</strong> Boring, if the nine steps above were not.</li>
</ol>



<p class="wp-block-paragraph">Phase four is the one that decides your date. Everything else is reasonably predictable once you have measured it. Rewriting and proving business logic is not, because the estimate depends on code nobody has read in years.</p>



<h2 class="wp-block-heading">Where these plans actually slip</h2>



<ul class="wp-block-list">
<li><strong>SQL outside the database.</strong> Query strings embedded in application code, report definitions in a BI tool, ad hoc scripts on a scheduler. None of it appears in a schema assessment and all of it has to be found. Routinely the biggest underestimate on the plan.</li>



<li><strong>Adding keys to source tables.</strong> Putting a primary key on a very large production table so CDC can replicate it is its own change, its own maintenance window and its own risk.</li>



<li><strong>Test data.</strong> Getting production-shaped volume into a non-production environment is slow, and if the data is regulated you have to build masking first. A project inside the project.</li>



<li><strong>Running two databases at once.</strong> For the whole parallel period you pay for Oracle, for Aurora, and for the replication capacity between them. Model that before it becomes a surprise in a finance review.</li>



<li><strong>Waiting on other teams.</strong> The supplemental logging change window, the network path, the security review of the DMS user&#8217;s privileges. Each one is short and none of them start when you ask.</li>



<li><strong>Environment sprawl.</strong> The conversion tooling, the replay harness and the comparison scripts all want somewhere persistent to live, and a laptop is not it. A modest dedicated box from a provider like InterServer or Contabo, or a small always-on EC2 instance, saves a lot of &#8220;it worked on mine&#8221; over a long project.</li>
</ul>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<h3 class="wp-block-heading">&#8220;The tool converted almost everything, so we&#8217;re almost done&#8221;</h3>



<p class="wp-block-paragraph">Object counts and effort are not proportional. A thousand trivially converted tables and one 4,000-line billing package are not a 1000-to-1 ratio of work. The leftovers are also the objects most tangled up in business rules, which is exactly why the tool could not convert them.</p>



<h3 class="wp-block-heading">&#8220;We&#8217;ll tune the slow queries after go-live&#8221;</h3>



<p class="wp-block-paragraph">You can, and sometimes that is the right call. Just understand the trade: after the first production write lands on Aurora, the rollback window is closed unless you built reverse replication. Deferring performance work means accepting whatever you find with no way back.</p>



<h3 class="wp-block-heading">&#8220;We can skip a rehearsal because the load is repeatable&#8221;</h3>



<p class="wp-block-paragraph">The full load is repeatable. The cutover is not. The rehearsal tests the sequence around the load: stopping application writes, draining lag to zero, resetting sequences, re-enabling triggers and foreign keys, repointing connection strings, confirming the first write landed. That sequence is where the time goes and where the mistakes happen.</p>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">How long does an Oracle to Aurora PostgreSQL migration take?</h3>



<p class="wp-block-paragraph">The honest answer is that the data movement is the predictable part and the code is not. A schema with few stored procedures and a well-understood application can move in weeks. A system with hundreds of thousands of lines of PL/SQL encoding two decades of business rules is a multi-quarter programme. Estimate from the volume of procedural code and the availability of people who understand it, not from the object count in the assessment report.</p>



<h3 class="wp-block-heading">Should I target Aurora PostgreSQL or RDS for PostgreSQL?</h3>



<p class="wp-block-paragraph">Aurora separates compute from a shared storage layer, which gives faster replica creation and failover, and it carries AWS-specific extensions including query plan management. RDS for PostgreSQL is closer to community PostgreSQL and simpler to reason about. For a workload coming off Oracle with plan-stability concerns and high availability requirements, Aurora usually fits better. For a small, steady workload where portability matters more than failover speed, RDS is a reasonable answer and easier to leave later.</p>



<h3 class="wp-block-heading">Do I still need AWS SCT, or is DMS Schema Conversion enough?</h3>



<p class="wp-block-paragraph">For a standard Oracle to PostgreSQL OLTP conversion, the managed DMS Schema Conversion feature is the default path and what AWS points you at. Reach for the SCT desktop application when you want the conversion scripted into a repeatable pipeline, need a feature the managed version does not expose, or hit a conversion path it does not cover.</p>



<h3 class="wp-block-heading">What replaced DMS Fleet Advisor?</h3>



<p class="wp-block-paragraph">Nothing directly. Support ended on 20 May 2026, and AWS did not ship a like-for-like successor inside DMS. Fleet-level discovery now comes from whatever you already have: an existing CMDB, AWS Application Discovery Service for the server layer, or scripts that query each database&#8217;s data dictionary for object counts and feature usage. If you are migrating one or two databases rather than an estate, you were probably not using it anyway.</p>



<h3 class="wp-block-heading">Can I migrate from Oracle to Aurora with no downtime at all?</h3>



<p class="wp-block-paragraph">Near-zero rather than zero. Full load plus CDC keeps the target current while the source stays live, so the outage shrinks to the moment you stop writes, let lag reach zero and repoint the application. That is typically minutes. What stretches it is sequence resets, re-enabling constraints and triggers, and connection pools holding the old endpoint longer than expected.</p>



<h3 class="wp-block-heading">Is ora2pg a real alternative to the AWS tooling?</h3>



<p class="wp-block-paragraph">Yes, and it is worth running even if you plan to use AWS tools. ora2pg is a mature open source converter that handles schema and PL/SQL and produces its own migration assessment. Teams that run both tend to find each catches things the other missed, and the second assessment costs an afternoon. Where AWS tooling wins is integration with DMS for the data movement itself.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">A conversion percentage measures objects. Your schedule is set by behaviour.</p>



<p class="wp-block-paragraph">The code that failed to convert is already on someone&#8217;s list. The code that converted cleanly is not, and that is where the empty strings, the missing exceptions and the different plans are hiding. An Oracle to Aurora PostgreSQL migration plan that budgets for proving behaviour, rather than for converting objects, is the one that lands on its date.</p>



<p class="wp-block-paragraph">If you take one action from this post, run the primary key query above against your real schemas this week. It takes a minute and it will tell you more about your cutover shape than the assessment report will.</p>



<h2 class="wp-block-heading">Working on an Oracle to Aurora PostgreSQL migration?</h2>



<p class="wp-block-paragraph">This is the kind of work I do as a freelance DevOps and cloud engineer. Specifically, on migrations like this one:</p>



<ul class="wp-block-list">
<li>Turning an assessment report into a phased plan with realistic gates, including the schemas that should not move at all</li>



<li>Setting up DMS full load and CDC properly: supplemental logging, LogMiner versus Binary Reader, LOB handling, and premigration assessments run early rather than late</li>



<li>Building the differential test harness that proves converted PL/pgSQL behaves like the original, instead of just compiling</li>



<li>Performance parity work on Aurora PostgreSQL: statistics, plan capture with query plan management, and finding the queries the optimiser now handles differently</li>



<li>Writing and rehearsing the cutover runbook, including the sequence resets and constraint re-enablement that get forgotten</li>



<li>Deciding whether reverse replication is worth building for your risk profile, and building it if it is</li>
</ul>



<p class="wp-block-paragraph">If you are in the middle of one of these, send me something concrete: your assessment report summary, a DMS task log, or the output of the primary key query above. I would rather look at your actual situation than talk in general terms.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/oracle-to-aurora-postgresql-migration-plan/">Oracle to Aurora PostgreSQL: A Migration Plan That Survives Contact</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>AWS Schema Conversion Tool for Oracle: What Converts and What Quietly Doesn&#8217;t</title>
		<link>https://john-nessime.com/blog/technical-guides/aws-schema-conversion-tool-oracle/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sat, 03 Oct 2026 06:43:14 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Databases]]></category>
		<category><![CDATA[Technical Guides]]></category>
		<category><![CDATA[Amazon Aurora]]></category>
		<category><![CDATA[Amazon RDS]]></category>
		<category><![CDATA[AWS DMS]]></category>
		<category><![CDATA[AWS SCT]]></category>
		<category><![CDATA[Cutover Planning]]></category>
		<category><![CDATA[Data Type Mapping]]></category>
		<category><![CDATA[Database Migration]]></category>
		<category><![CDATA[Migration Assessment]]></category>
		<category><![CDATA[Oracle]]></category>
		<category><![CDATA[PL/SQL]]></category>
		<category><![CDATA[PostgreSQL]]></category>
		<category><![CDATA[Schema Conversion]]></category>
		<category><![CDATA[SQL]]></category>
		<category><![CDATA[Vendor Lock-In]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=388</guid>

					<description><![CDATA[<p>The assessment report grades syntax, not behavior. A four-tier breakdown of what AWS SCT converts cleanly, what it converts while silently changing meaning, what it only handles through the extension pack, what it refuses outright, and the whole category it never looks at.</p>
<p>The post <a href="https://john-nessime.com/blog/technical-guides/aws-schema-conversion-tool-oracle/">AWS Schema Conversion Tool for Oracle: What Converts and What Quietly Doesn&#8217;t</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">Open the assessment report on a mid-sized Oracle schema and the top-line number looks fantastic. Ninety-something percent of objects convert automatically. The red items are counted, categorized, and given effort estimates. It reads like a project plan you could take to a steering committee.</p>



<p class="wp-block-paragraph">The part that hurts you later is not on that page.</p>



<p class="wp-block-paragraph">The AWS Schema Conversion Tool for Oracle grades <em>syntax</em>. It tells you whether it could emit valid PostgreSQL for an object. It does not tell you whether that object will behave the same way at runtime. A trigger can convert with a green tick and fire in a different order. A column can convert with a green tick and silently drop the time component off every date in the table. A procedure can convert with a green tick and run twenty times slower because every string function call now goes through a wrapper.</p>



<p class="wp-block-paragraph">This post sorts the tool&#8217;s output into four buckets: what converts cleanly, what converts while quietly changing meaning, what converts only through the extension pack, and what it refuses outright. Then the category nobody budgets for, which is everything it never looks at.</p>



<h2 class="wp-block-heading">What the AWS Schema Conversion Tool for Oracle actually does</h2>



<p class="wp-block-paragraph">It is a Java desktop application. It connects over JDBC to source and target, reads the Oracle data dictionary, parses the DDL and PL/SQL it finds, and generates equivalent PostgreSQL objects and PL/pgSQL. The converted code stays inside the project until you explicitly apply it to the target or save it out as SQL files.</p>



<p class="wp-block-paragraph">Three things follow from that description, and they matter more than any feature list.</p>



<ul class="wp-block-list">
<li>It only sees what is stored in the database. Code that lives in your application, your job scheduler, or a reporting tool is invisible to it unless you point the separate application SQL conversion feature at those files yourself.</li>

<li>It only sees what the connecting user can see. Connect with a thin account and half the schema will not appear in the report at all.</li>

<li>It moves structure and code. It does not move rows. That is AWS DMS, and it is a separate exercise with its own failure modes.</li>
</ul>



<p class="wp-block-paragraph">There is also a command line interface, which is the part most teams discover too late. It uses scenario files rather than flag-style arguments, and it needs Amazon Corretto 11 installed alongside the tool.</p>



<pre class="wp-block-code"><code># Interactive mode, run from the AWS SCT app folder.
# End each command with a / on its own line.
java -jar AWSSchemaConversionToolBatch.jar -type interactive

# Script mode, running a saved .scts scenario end to end.
java -jar /opt/aws-schema-conversion-tool/lib/app/AWSSchemaConversionToolBatch.jar 
  -type scts -script /home/sct/project/convert.scts</code></pre>



<p class="wp-block-paragraph">Script mode turns a one-off conversion into something you can rerun, which matters because on a real migration you will convert the schema many times as you clean up the source. Re-clicking through the GUI each time is how mistakes creep in.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Tier one: converts cleanly, and you can trust it</h2>



<p class="wp-block-paragraph">Credit where it is due. For the bulk of an ordinary OLTP schema the tool is good, and hand-writing this would be miserable.</p>



<ul class="wp-block-list">
<li>Table definitions: names, columns, ordinary data types, NOT NULL, defaults, primary keys, unique constraints, foreign keys.</li>

<li>Standard b-tree indexes, including most single-column functional indexes.</li>

<li>Ordinary views. The basic CREATE VIEW syntax is close to identical between the two engines.</li>

<li>Sequences and identity columns, despite the syntax looking nothing alike.</li>

<li>ANSI-standard SQL, joins, subqueries, and almost all window and analytic functions.</li>

<li>PL/SQL flow control: loops, blocks, conditionals, exception structure.</li>

<li>The four standard transaction isolation levels, plus BEGIN, COMMIT, and ROLLBACK.</li>
</ul>



<p class="wp-block-paragraph">If your schema is mostly tables, views, and reporting SQL with modest procedural code, the automation figure in the report is roughly honest and you should believe it.</p>



<h2 class="wp-block-heading">Tier two: converts green, behaves differently</h2>



<p class="wp-block-paragraph">This is the tier that costs money, because none of it shows up as a red action item. The object converted. The DDL applied. The application connected. Something is wrong three weeks later and nobody thinks to look back at the schema conversion.</p>



<h3 class="wp-block-heading">Oracle DATE is not PostgreSQL date</h3>



<p class="wp-block-paragraph">Oracle&#8217;s DATE type carries a time component down to the second. PostgreSQL&#8217;s <code>date</code> is a calendar day and nothing more. Map an Oracle DATE to a PostgreSQL <code>date</code> and every timestamp in that column is truncated, permanently, without an error. The correct target is <code>timestamp</code>. Check the conversion settings and check the generated DDL before you apply, because this one is not recoverable after the data lands.</p>



<h3 class="wp-block-heading">NUMBER without precision becomes numeric, and numeric is slow</h3>



<p class="wp-block-paragraph">An unqualified Oracle NUMBER is an extremely flexible container, so the default mapping to PostgreSQL <code>numeric</code> is technically correct. It is also arbitrary-precision arithmetic implemented in software. Put that on a surrogate key that participates in every join and index in your application and you have bought yourself a performance problem that no amount of index tuning will fix.</p>



<p class="wp-block-paragraph">Find the candidates before conversion, not after:</p>



<pre class="wp-block-code"><code>-- Oracle: unqualified NUMBER columns, the ones that default to numeric.
SELECT table_name, column_name, data_precision, data_scale
FROM   user_tab_columns
WHERE  data_type = 'NUMBER'
AND    data_precision IS NULL
ORDER  BY table_name, column_name;</code></pre>



<p class="wp-block-paragraph">For each one, look at the actual minimum, maximum, and scale of the data, then decide whether it is really an <code>integer</code>, a <code>bigint</code>, or a genuine <code>numeric(p,s)</code>. Surrogate keys and foreign keys are almost always integers pretending to be NUMBER. Money columns usually are not.</p>



<h3 class="wp-block-heading">Empty string is NULL in Oracle and is not in PostgreSQL</h3>



<p class="wp-block-paragraph">This is the single most reliable source of post-cutover bugs, and it is not a conversion defect. Oracle treats an empty string as NULL. PostgreSQL treats it as a zero-length string, which is a real value. Every <code>IS NULL</code> check in your procedural code and your application was written under Oracle&#8217;s rule.</p>



<p class="wp-block-paragraph">Concatenation flips too. In Oracle, concatenating a NULL is a no-op. In PostgreSQL, concatenating a NULL with the <code>||</code> operator yields NULL for the whole expression. A report that used to print a name with a missing middle initial now prints nothing at all.</p>



<p class="wp-block-paragraph">After the data lands, count the divergence before you let users near it:</p>



<pre class="wp-block-code"><code>-- PostgreSQL: how many rows are empty string rather than NULL?
SELECT count(*) FILTER (WHERE middle_name = '')   AS empty_string,
       count(*) FILTER (WHERE middle_name IS NULL) AS real_null
FROM   app.customer;</code></pre>



<p class="wp-block-paragraph">If <code>empty_string</code> is non-zero anywhere, you have a decision to make: normalize the data on load, or audit every NULL check that touches those columns. Normalizing is usually cheaper and is far easier to verify.</p>



<h3 class="wp-block-heading">The warnings the tool does raise, that people scroll past</h3>



<p class="wp-block-paragraph">Some of these semantic differences do appear in the action list, just not in red. They are worth reading individually rather than in aggregate.</p>



<ul class="wp-block-list">
<li>GREATEST and LEAST are flagged as potentially producing different results from the source, because of how each engine handles NULL arguments and type coercion.</li>

<li>Timestamp handling inside converted partitioned tables is flagged as potentially divergent.</li>

<li>Converted code is flagged as possibly incorrect where bind variable names, user-defined functions, or dynamic SQL are involved. That is the tool telling you it guessed.</li>

<li>Trigger firing order is not carried over. Oracle&#8217;s FOLLOWS and PRECEDES clauses have no equivalent, and PostgreSQL fires triggers on the same event in alphabetical order by trigger name. If your Oracle triggers depended on sequencing, you now depend on naming.</li>

<li>An empty exception block in the converted output is called out explicitly. Oracle code that swallowed errors will now swallow them differently.</li>
</ul>



<p class="wp-block-paragraph">One more that is not an action code at all: Oracle PL/SQL routinely issues COMMIT in the middle of a procedure. A PostgreSQL function cannot do that. Either the logic becomes a procedure invoked with CALL, or the transaction boundary moves out into the calling application. That is a design decision, not a find-and-replace.</p>



<h2 class="wp-block-heading">Tier three: converts, but only through the extension pack</h2>



<p class="wp-block-paragraph">When there is no native PostgreSQL equivalent for something Oracle provides, the tool installs an extension pack: an additional schema on the target, named <code>aws_oracle_ext</code> for Oracle sources, containing wrapper functions, tables, and supporting objects that emulate the source behavior. For some capabilities like job scheduling and sending mail, the pack also deploys AWS Lambda functions.</p>



<p class="wp-block-paragraph">So a call to <code>INSTR</code> in your PL/SQL comes out the other side as <code>aws_oracle_ext.INSTR</code>. Package variables, which have no PostgreSQL equivalent because PL/pgSQL has no packages, get routed through getter and setter functions in the pack.</p>



<p class="wp-block-paragraph">The case for accepting this is real. If the goal is off Oracle licensing by a fixed date, the pack converts code that would otherwise need a developer per object. That is the difference between a migration that ships and one that stalls in refactoring, and nobody gets a bonus for elegant PL/pgSQL on a project that missed its window.</p>



<p class="wp-block-paragraph">The case against is equally real:</p>



<ul class="wp-block-list">
<li>Wrapper functions sit between your code and the engine. In hot loops, that overhead is measurable, and the package-variable emulation in particular is not free.</li>

<li>You have swapped one dependency for another. The pack is AWS-specific code living in your database, which is an awkward thing to explain if the stated goal was ending vendor lock-in.</li>

<li>Every time the pack is applied, its components are overwritten. Anything you hand-tuned inside that schema is gone.</li>

<li>Upgrades and target-version changes now have an extra moving part to validate.</li>
</ul>



<p class="wp-block-paragraph">There is a middle path. The tool can be configured to use the open-source <code>orafce</code> extension, which implements a subset of Oracle functions and packages natively in PostgreSQL, in place of some pack wrappers. My rule of thumb: use the pack to get the migration over the line, then treat the size of the <code>aws_oracle_ext</code> call graph as a debt register you burn down deliberately, starting with whatever appears in your slowest queries. Rewriting collections as native arrays or JSON, and package variables as parameters, buys back most of the performance.</p>



<h2 class="wp-block-heading">Tier four: it stops and hands you the work</h2>



<p class="wp-block-paragraph">These are the red items: the honest part of the report, and paradoxically the least dangerous, because you can see them and plan around them. Not exhaustive, but this is what turns up on real Oracle schemas.</p>



<ul class="wp-block-list">
<li><strong>Database links.</strong> No automation at all. The mechanism has to be rebuilt, usually on <code>postgres_fdw</code>, and foreign data wrappers do not support calling user-defined functions on the remote side. If your architecture leans on links, this is a workstream, not a task.</li>

<li><strong>Autonomous transactions.</strong> PostgreSQL has no direct equivalent. Logging patterns that relied on committing independently of the parent transaction need redesign, often with <code>dblink</code> back to the same instance or an out-of-band queue.</li>

<li><strong>ROWID and UROWID.</strong> No equivalent, and any code that reads ROWID from a table is flagged separately. Row-identity tricks in PL/SQL are a rewrite.</li>

<li><strong>Dynamic SQL and DBMS_SQL.</strong> Anything assembled at runtime the parser cannot resolve statically comes back untouched or with a health warning.</li>

<li><strong>Synonyms.</strong> Not supported, and neither is their usage, so every reference through a synonym has to be resolved by hand or mimicked with views and search paths.</li>

<li><strong>Global temporary tables, index-organized tables, external tables, clustered tables, object tables, nested tables, virtual columns, BFILE.</strong> Storage and type features with no target equivalent.</li>

<li><strong>Bitmap indexes, domain indexes, multi-column functional indexes.</strong> The first two have no counterpart; the third is a documented conversion gap.</li>

<li><strong>Associative arrays, VARRAY of VARRAY, multi-dimensional arrays, object type methods, member functions in user-defined types.</strong> Collection-heavy PL/SQL is where the automation percentage collapses.</li>

<li><strong>Compound triggers, system triggers, REFERENCING clauses, conditional predicates.</strong> The trigger model differs enough that anything beyond a plain row trigger needs attention.</li>

<li><strong>Java stored routines and table functions.</strong> Not converted.</li>

<li><strong>Query hints.</strong> Partially converted, and you should delete them all first anyway. Hints tuned for Oracle&#8217;s optimizer are noise to a different planner. Add them back only after schema, index, and query work have failed.</li>

<li><strong>Materialized view incremental refresh and DML on materialized views.</strong> The view itself may convert; the refresh strategy will not.</li>
</ul>



<h3 class="wp-block-heading">Some red items are more conservative than your actual target</h3>



<p class="wp-block-paragraph">Here is a detail that saves real budget. The action code index and the migration playbook were written against a baseline target, and PostgreSQL has moved since. MERGE is listed as unsupported, but PostgreSQL has had a native MERGE statement since version 15. FOR UPDATE SKIP LOCKED is listed as unsupported, and it has been in PostgreSQL for a long time. GROUPING SETS, CUBE, and ROLLUP are listed as unsupported and are also long-standing PostgreSQL features.</p>



<p class="wp-block-paragraph">What this means practically: an action code tells you the converter will not emit the construct for you. It is not a statement about your target engine. Before you scope a rewrite for anything on that list, check the feature against the PostgreSQL version you are actually deploying. Some of those &#8220;rewrites&#8221; are a copy-paste with a syntax tweak.</p>



<h2 class="wp-block-heading">What it never looks at</h2>



<p class="wp-block-paragraph">This is the category that wrecks timelines, because it is not in the report at all. Not green, not red. Out of scope, therefore invisible.</p>



<ul class="wp-block-list">
<li><strong>Application code.</strong> There is a separate application SQL conversion feature that scans source files for embedded SQL, but it is opt-in, file-based, and you have to aim it. Nothing happens automatically.</li>

<li><strong>Scheduled work.</strong> DBMS_SCHEDULER jobs, cron entries calling SQL*Plus, shell wrappers, ETL controllers. The schema converts; the thing that runs it every night does not.</li>

<li><strong>Grants, roles, profiles, and proxy authentication.</strong> Your security model is a separate design exercise on the target.</li>

<li><strong>Oracle features with no schema footprint.</strong> Data Guard, RAC, ASM, RMAN, Oracle Text indexes, fine-grained access control policies. Each is its own replacement decision.</li>

<li><strong>Optimizer statistics and execution plans.</strong> Nothing about performance carries over. You are starting from zero on the target, and that is the right expectation to set with stakeholders early.</li>

<li><strong>Everything downstream.</strong> BI dashboards, reporting layers, and integration jobs that embed Oracle-flavored SQL will break on the day you cut over, and none of them appear in the assessment.</li>
</ul>



<p class="wp-block-paragraph">Inventory these before conversion, not after. A one-page list of every consumer of the schema is worth more than a week of tuning.</p>



<h2 class="wp-block-heading">Desktop tool or DMS Schema Conversion?</h2>



<p class="wp-block-paragraph">DMS Schema Conversion is the managed, browser-based version built on the same engine. Instead of a laptop install you create an instance profile, register source and target data providers with credentials in Secrets Manager, and run the project from the console, with metadata staged in S3.</p>



<p class="wp-block-paragraph">AWS now points standard OLTP conversions at the managed option, and generative AI assisted conversion is available there on supported paths. The desktop tool remains available and still covers ground the managed feature does not.</p>



<ul class="wp-block-list">
<li><strong>Pick the desktop tool</strong> when you need scripting through the CLI, want to edit the SQL for source and target inside the project, need migration rules and tree filters, are converting a data warehouse target, or want the application SQL and ETL conversion features.</li>

<li><strong>Pick DMS Schema Conversion</strong> when your source is reachable from a VPC, you would rather not manage a workstation with database credentials on it, your conversion path and region are supported, and the generative AI assist is useful to you.</li>
</ul>



<p class="wp-block-paragraph">Check supported paths, versions, and regions before committing either way; they differ between the two. On the desktop side, note the supported operating systems are Windows, Fedora, and Ubuntu, all 64-bit. If your analysts are on Macs, plan for a Linux jump box. A small VPS from a provider like Contabo or InterServer, or an EC2 instance inside the VPC, is the usual answer, and it has the side benefit of keeping the JDBC connection close to the source rather than dragging a dictionary read across a home broadband link.</p>



<p class="wp-block-paragraph">If the Oracle source is on-premises and the conversion box is not, you need a network path that a DBA will sign off on. That is normally a site-to-site VPN or Direct Connect. For an individual consultant working against a client&#8217;s staging environment, a commercial VPN such as NordVPN or Surfshark is sometimes what is on offer, but treat it as a stopgap: it gets you connectivity, not an audit trail.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Troubleshooting</h2>



<h3 class="wp-block-heading">The CLI jar refuses to open</h3>



<p class="wp-block-paragraph">A common failure is an unexpected error opening <code>AWSSchemaConversionToolBatch.jar</code>. Recent OpenJDK builds cap the size of jar signature files and the SCT jar exceeds the default. Raising that limit on the java invocation is the widely reported workaround:</p>



<pre class="wp-block-code"><code>java -Djdk.jar.maxSignatureFileSize=20000000 
  -jar AWSSchemaConversionToolBatch.jar -type interactive</code></pre>



<p class="wp-block-paragraph">Confirm you are on the Corretto 11 runtime the tool expects before chasing anything more exotic.</p>



<h3 class="wp-block-heading">It hangs on a large schema, or will not connect at all</h3>



<p class="wp-block-paragraph">Thousands of objects in one schema puts the JVM under memory pressure; Global settings exposes JVM options, so raise the heap and retry on a machine with real memory. Separately, JDBC driver paths live in Global settings under Drivers, per engine. A driver sitting on disk but not registered there fails in a way that looks like a network problem and is not.</p>



<h3 class="wp-block-heading">&#8220;The package body doesn&#8217;t include source code&#8221;</h3>



<p class="wp-block-paragraph">The source is wrapped. Obfuscated PL/SQL cannot be parsed, so it cannot be converted. Either find the original in version control or accept a from-scratch rewrite. Finding this on a critical package late in the project is a bad day, so check for wrapped objects on day one.</p>



<h3 class="wp-block-heading">Objects will not apply, or the report is missing things</h3>



<p class="wp-block-paragraph">Failures on apply are usually dependency order: an object references a parent that was not converted or sits outside the schema mapping, and the action list says so explicitly. Convert the whole dependency closure, not just the objects you care about. A short report is usually privileges, since the tool reads through the connecting account&#8217;s visibility. Compare its object counts against a direct dictionary count before trusting any percentage in it.</p>



<h2 class="wp-block-heading">Common mistakes</h2>



<ul class="wp-block-list">
<li>Treating the automation percentage as a completion estimate. It measures objects, not effort, and one gnarly package can outweigh four hundred clean tables.</li>

<li>Applying to the target before reviewing the generated DDL. Save to SQL, read it, diff it on the next run.</li>

<li>Accepting default type mappings wholesale. DATE and unqualified NUMBER are the two that reliably hurt.</li>

<li>Carrying Oracle hints across. Start with none and earn each one back.</li>

<li>Scoping rewrites straight off the action codes without checking your target PostgreSQL version first.</li>

<li>Leaving project files and scenario scripts containing database credentials sitting on a laptop after the engagement. They are plain files; when the work is done, wipe them properly with a tool like O&amp;O SafeErase rather than dragging them to the trash.</li>
</ul>



<h2 class="wp-block-heading">Best practices</h2>



<ol class="wp-block-list">
<li>Connect with an account that can see the whole schema, then verify object counts against the dictionary before reading any percentage.</li>

<li>Profile your NUMBER and DATE columns first and set the type mappings deliberately. Do this before the first conversion, not after the first load.</li>

<li>Script the conversion with the CLI so every run is identical and reviewable.</li>

<li>Sort red action items by object criticality, not by count. Twenty flags on a nightly batch job matter more than two hundred on a deprecated reporting view.</li>

<li>Write the semantic test suite before cutover: empty string versus NULL, date boundaries, numeric rounding, trigger ordering, transaction behavior in converted routines.</li>

<li>Instrument the target from day one. Query-level metrics in Amazon CloudWatch, or a Grafana stack against pg_stat_statements, will show you which extension pack wrappers are actually costing you before users complain.</li>
</ol>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Is the AWS Schema Conversion Tool free?</h3>



<p class="wp-block-paragraph">The tool itself is a free download from AWS. What you pay for is everything around it: the compute you run it on, the target database, DMS replication instances for the data move, and the engineering time on the objects it hands back to you. That last item is the whole budget on most migrations.</p>



<h3 class="wp-block-heading">Does it migrate data as well as schema?</h3>



<p class="wp-block-paragraph">No. It converts structure and code. Moving rows is AWS DMS, which handles both a full load and ongoing change capture. They are designed to be used together, and each has its own set of behaviors worth understanding separately.</p>



<h3 class="wp-block-heading">Can I skip the extension pack entirely?</h3>



<p class="wp-block-paragraph">You can, but you are then hand-writing PostgreSQL equivalents for every Oracle built-in the pack would have covered. Configuring the tool to prefer <code>orafce</code> where possible is the usual compromise: fewer AWS-specific wrappers, open-source implementation, still automated. Full independence is achievable and is a refactoring project, not a setting.</p>



<h3 class="wp-block-heading">Why did my report say 95% automatic and the project still take months?</h3>



<p class="wp-block-paragraph">Because the percentage counts objects, ignores runtime semantics entirely, and excludes application code, scheduled jobs, security, and downstream consumers. Tier two and the out-of-scope list in this post are where the months go.</p>



<h3 class="wp-block-heading">Should I clean up the Oracle schema before or after conversion?</h3>



<p class="wp-block-paragraph">Before, every time. Dropping dead objects, unwrapping code you own, and removing synonyms on the source reduces the surface the tool has to reason about and shrinks the red list. It is also the only cleanup you can do while you still have a working system to test against.</p>



<h3 class="wp-block-heading">How do I verify the conversion actually worked?</h3>



<p class="wp-block-paragraph">Row counts and DDL comparison are the floor, not the ceiling. Run the same business queries against both engines and compare results, with deliberate coverage of NULL handling, date boundaries, sorting and collation, and numeric rounding. Anything that only shows up under concurrency needs a load test, not a smoke test.</p>



<h2 class="wp-block-heading">The one thing worth remembering</h2>



<p class="wp-block-paragraph">The AWS Schema Conversion Tool for Oracle is a good parser and an honest one about the things it cannot parse. The red list is trustworthy. The trouble is that a green tick means &#8220;I emitted valid PostgreSQL,&#8221; not &#8220;this behaves the way it did.&#8221; Dates lose their time, unqualified NUMBER lands as slow arithmetic, empty strings stop being NULL, triggers reorder themselves alphabetically, and none of that appears as a problem in the report.</p>



<p class="wp-block-paragraph">Use it for what it is good at, which is the tedium you should never do by hand. Then spend your review time on the objects that passed, because that is where the expensive surprises live. Treat the assessment report as an inventory, never as a schedule.</p>



<h2 class="wp-block-heading">Need a second pair of eyes on an Oracle to PostgreSQL conversion?</h2>



<p class="wp-block-paragraph">Most of the value in this work is in the review, not the click. Things I help with:</p>



<ul class="wp-block-list">
<li>Reading an assessment report properly: separating object counts from real effort, and re-scoping red items against the PostgreSQL version you are actually deploying.</li>

<li>Auditing type mappings before you apply, with a column-level profile of NUMBER and DATE usage and a concrete mapping decision for each.</li>

<li>Building the semantic test suite that catches empty-string and NULL divergence, trigger ordering, and rounding differences before users find them.</li>

<li>Scripting the conversion through the SCT CLI so it is repeatable, reviewable, and lives in version control alongside the generated DDL.</li>

<li>Planning the extension pack burn-down: which <code>aws_oracle_ext</code> calls are costing you, and what the native PostgreSQL replacement looks like.</li>

<li>Mapping the out-of-scope surface, meaning scheduled jobs, grants, downstream reports and integrations, so cutover day has no surprises in it.</li>
</ul>



<p class="wp-block-paragraph">Send me an assessment report, a converted DDL file, or the action item list, and I will tell you what I would actually worry about in it.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>

<p>The post <a href="https://john-nessime.com/blog/technical-guides/aws-schema-conversion-tool-oracle/">AWS Schema Conversion Tool for Oracle: What Converts and What Quietly Doesn&#8217;t</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
		<item>
		<title>SageMaker Lakehouse vs Glue + Athena: Is Migration Worth It?</title>
		<link>https://john-nessime.com/blog/cloud-computing/sagemaker-lakehouse-migration/</link>
		
		<dc:creator><![CDATA[John Nessime]]></dc:creator>
		<pubDate>Sun, 27 Sep 2026 06:01:58 +0000</pubDate>
				<category><![CDATA[Cloud Computing]]></category>
		<category><![CDATA[Data Engineering]]></category>
		<category><![CDATA[FinOps]]></category>
		<category><![CDATA[Amazon Athena]]></category>
		<category><![CDATA[Amazon Redshift]]></category>
		<category><![CDATA[Apache Iceberg]]></category>
		<category><![CDATA[AWS Glue]]></category>
		<category><![CDATA[Cost Allocation Tags]]></category>
		<category><![CDATA[Cost Optimization]]></category>
		<category><![CDATA[Cross-Account Access]]></category>
		<category><![CDATA[Data Catalog]]></category>
		<category><![CDATA[Data Governance]]></category>
		<category><![CDATA[Federated Catalogs]]></category>
		<category><![CDATA[Hybrid Access Mode]]></category>
		<category><![CDATA[Iceberg REST Catalog]]></category>
		<category><![CDATA[Lake Formation]]></category>
		<category><![CDATA[Lakehouse]]></category>
		<category><![CDATA[Migration Assessment]]></category>
		<category><![CDATA[Operational Complexity]]></category>
		<category><![CDATA[S3 Tables]]></category>
		<category><![CDATA[SageMaker Unified Studio]]></category>
		<category><![CDATA[Total Cost of Ownership]]></category>
		<category><![CDATA[Zero-ETL]]></category>
		<guid isPermaLink="false">https://john-nessime.com/blog/?p=595</guid>

					<description><![CDATA[<p>SageMaker Lakehouse is built on the Glue Data Catalog and Lake Formation, so a migration barely moves data. It moves enforcement. Here is the failure mode that bites, the cost levers that actually change, and a decision procedure for working out whether the move is worth it at all.</p>
<p>The post <a href="https://john-nessime.com/blog/cloud-computing/sagemaker-lakehouse-migration/">SageMaker Lakehouse vs Glue + Athena: Is Migration Worth It?</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></description>
										<content:encoded><![CDATA[
<p class="wp-block-paragraph">A Glue job that had run every morning for two years failed on a Monday. Access denied on <code>GetTable</code>, thrown by Lake Formation. Nothing about the job had changed. Nothing about the IAM role had changed. What had changed was that someone spent Friday afternoon &#8220;moving us onto the lakehouse&#8221; and, as part of that, revoked a permission from a virtual group most people have never heard of.</p>



<p class="wp-block-paragraph">That is the shape of the risk. When people ask whether a SageMaker Lakehouse migration is worth it, they picture data moving. Almost none of it does. What moves is the permission model, and that is the part nobody tests.</p>



<p class="wp-block-paragraph">This post covers what the migration actually changes underneath, the failure mode that bites hardest, where the lakehouse earns its place, where staying on Glue plus Athena is the better call, how the cost levers shift, and a decision procedure you can run in an afternoon.</p>



<h2 class="wp-block-heading">First, understand what you are actually migrating</h2>



<p class="wp-block-paragraph">The framing &#8220;SageMaker Lakehouse versus Glue and Athena&#8221; is misleading, and it drives bad decisions.</p>



<p class="wp-block-paragraph">SageMaker Lakehouse is built on the AWS Glue Data Catalog and AWS Lake Formation. It is not a replacement metastore. It is a reorganisation of the catalog you already have, plus connectors, plus an Apache Iceberg REST interface, plus a console to look at all of it through.</p>



<p class="wp-block-paragraph">The unit it introduces is the <em>catalog</em>: a logical container of databases, tables and views, nestable to mirror the shape of the source system. A <strong>managed catalog</strong> holds data you own in S3 or Redshift Managed Storage. A <strong>federated catalog</strong> points at something external such as Redshift, DynamoDB, PostgreSQL, Oracle, BigQuery or Snowflake, and lets you query it in place.</p>



<p class="wp-block-paragraph">Your existing databases and tables do not disappear. They sit under the account catalog and keep working. AWS states in its own FAQ that you do not have to migrate, and that Athena, EMR, Glue and MWAA continue to work as they do today. That is an unusual thing for a vendor to publish about a flagship product, and it is the most useful sentence in the documentation set.</p>



<p class="wp-block-paragraph">So the question is not &#8220;should we move our data&#8221;. It is &#8220;should we move our governance&#8221;.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The failure mode that actually bites</h2>



<p class="wp-block-paragraph">Lake Formation grants <code>Super</code> permission to a virtual group called <code>IAMAllowedPrincipals</code> on every existing Data Catalog database and table by default. That group effectively means &#8220;anyone whose IAM policy already lets them in&#8221;. It exists purely for backward compatibility. While it holds <code>Super</code> on a resource, access is decided by IAM and S3 policies exactly as it was before Lake Formation existed. Your Lake Formation grants are recorded, but they are not what gets enforced.</p>



<p class="wp-block-paragraph">So the first time someone writes a careful column-level policy and tests it, the policy appears not to work. The analyst can still see the column. The natural next move is to revoke <code>IAMAllowedPrincipals</code> so the grants take effect.</p>



<p class="wp-block-paragraph">That revoke is the moment everything relying on IAM-only access stops working. Glue jobs. Notebooks. Scheduled dbt runs. The QuickSight dataset nobody has looked at since the person who built it left. None of them had a Lake Formation grant, because none of them ever needed one.</p>



<p class="wp-block-paragraph">It is invisible in advance because nothing warns you. Lake Formation cannot know which principals are silently depending on the compatibility group. There is no dry run.</p>



<h3 class="wp-block-heading">Hybrid access mode is the way through</h3>



<p class="wp-block-paragraph">Hybrid access mode exists exactly for this. It supports two permission pathways to the same catalog objects at once. Principals you explicitly opt in are governed by Lake Formation grants. Everyone else keeps reaching the data through their IAM and S3 policies, untouched.</p>



<p class="wp-block-paragraph">The detail worth internalising: when an S3 location is registered in hybrid access mode, Lake Formation enforces only <code>CREATE_TABLE</code>, <code>CREATE_PARTITION</code> and <code>UPDATE_TABLE</code> by default. The write path gets governed first. The read path stays on IAM until you opt a principal in. That asymmetry is what lets you migrate one team at a time instead of one account at a time.</p>



<p class="wp-block-paragraph">The safe sequence: register the location in hybrid mode, opt in one low-stakes principal, verify its queries respect the fine-grained grants, verify a non-opted-in principal still works, repeat. Only after every consumer is opted in and verified do you consider revoking the compatibility group. Plenty of teams never need that last step.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Where the lakehouse genuinely wins</h2>



<ul class="wp-block-list">
<li><strong>One permission surface across engines.</strong> If governance today is &#8220;Lake Formation for the lake, Redshift GRANTs for the warehouse, IAM for DynamoDB&#8221;, you maintain three access models that disagree with each other. Catalogs let you express column and row-level policy once and have Athena, Redshift, EMR and Spark honour it.</li>



<li><strong>Tag-based access control across federated sources.</strong> LF-TBAC extends to federated catalogs covering S3 Tables, Redshift and external sources. If you hand-maintain named-resource grants across dozens of databases, tag expressions are a real operational win.</li>



<li><strong>Querying across boundaries without copying.</strong> A federated catalog lets one statement touch a Redshift table and an S3 Iceberg table, with no ETL job whose only purpose is moving rows so they can sit next to each other. Every pipeline you delete is a pipeline that cannot break at 3am.</li>



<li><strong>The Iceberg REST interface.</strong> Exposing the catalog through Iceberg REST APIs lets engines outside AWS read your tables over a standard protocol. If you run Trino, Spark or DuckDB elsewhere, this is what makes the catalog genuinely portable rather than nominally open.</li>



<li><strong>Cross-account sharing without replication.</strong> For data mesh setups, sharing a catalog instead of shipping a copy per consumer removes a whole category of staleness bug. Cross-account grants on federated catalog objects require the cross-account data sharing version set to 4 or higher.</li>
</ul>



<p class="wp-block-paragraph">Notice what these share. Every one is a governance or interoperability problem. None is a query performance problem, and none is a cost problem.</p>



<h2 class="wp-block-heading">Where Glue plus Athena is still the right answer</h2>



<ul class="wp-block-list">
<li><strong>Single account, S3 only, one team.</strong> Catalogs solve a problem you do not have. You would be adding a governance layer to govern one group.</li>



<li><strong>Your access control is coarse and that is correct.</strong> Not every dataset needs column masking. If bucket-level IAM boundaries match your actual risk model, Lake Formation enforcement adds a failure mode without adding safety.</li>



<li><strong>You have heavy IAM-only automation.</strong> Every Glue job, Lambda, Airflow DAG and third-party tool touching the catalog is a principal you must eventually opt in and re-verify. That inventory is the real cost of migration, and it is always larger than the first estimate.</li>



<li><strong>Your bottleneck is query cost, not governance.</strong> Partitioning, file sizes, compaction and columnar layout drive Athena spend. A lakehouse migration touches none of them. Fix the small-files problem first; it pays back faster and it is reversible.</li>



<li><strong>You depend on a specific engine behaviour today.</strong> Fine-grained access control support varies by engine and table format. Confirm your exact combination is supported rather than assuming parity across Athena, Redshift, EMR and Glue Spark.</li>
</ul>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">The cost levers that move, and the ones that don&#8217;t</h2>



<p class="wp-block-paragraph">Pricing pages change, so here are the billing mechanisms rather than numbers.</p>



<p class="wp-block-paragraph"><strong>Unchanged:</strong> metadata storage and API requests follow existing Glue Data Catalog pricing, including free tier allowances. Reorganising into catalogs does not put you on a new metadata meter. Athena still bills on data scanned. S3 storage and request charges are what they were.</p>



<p class="wp-block-paragraph"><strong>New line items:</strong></p>



<ul class="wp-block-list">
<li><strong>Lambda invocations behind federated catalogs.</strong> When a query hits a federated table, Lake Formation vends credentials that invoke a Lambda function defined in the Glue connection to retrieve metadata from the source. That is compute you did not previously pay for, and it is latency on the metadata path. Chatty BI tools that re-resolve schemas on every refresh are the ones to watch.</li>



<li><strong>Service-managed Redshift Serverless compute.</strong> Queries against a federated catalog pointing at an existing Redshift warehouse execute through a service-managed Redshift Serverless workgroup, which stages results before returning them. That compute is billed, and it does not appear where you expect it.</li>



<li><strong>Redshift Managed Storage.</strong> Managed catalogs backed by RMS rather than S3 put you on Redshift storage and Redshift Serverless compute economics, which behave nothing like S3 plus Athena.</li>
</ul>



<p class="wp-block-paragraph">The service-managed workgroup is the one that catches people, because the spend lands under Redshift Serverless with no obvious link to the analyst who ran the query. AWS publishes a generated cost allocation tag for exactly this, which you must activate in the billing console before it produces anything:</p>



<pre class="wp-block-code"><code># Cost allocation tag on lakehouse-managed Redshift Serverless compute.
# Activate under Billing -&gt; Cost allocation tags before it appears in reports.

Key:   aws:redshift-serverless:LakehouseManagedWorkgroup
Value: "True"</code></pre>



<p class="wp-block-paragraph">Activate it on day one, not after the first surprising invoice. Tags only apply to usage recorded after activation, so a late start leaves a blind period you cannot reconstruct. If you already run a cost visibility layer such as Vantage, CloudZero or Datadog&#8217;s cloud cost views, build a view isolating this tag before you create your first federated catalog. If you do not, the AWS Cost and Usage Report grouped by that tag key is enough to start.</p>



<h2 class="wp-block-heading">Is a SageMaker Lakehouse migration worth it? A decision procedure</h2>



<p class="wp-block-paragraph">Run these in order. Stop at the first one that gives a clear answer.</p>



<ol class="wp-block-list">
<li><strong>Count your non-S3 governed sources.</strong> Redshift clusters, DynamoDB tables, operational databases and external warehouses that analysts need and that each carry their own access model today. Zero or one means the federated catalog story does not apply to you yet. Three or more and the unified permission surface starts paying for itself.</li>



<li><strong>Inventory every principal touching the Data Catalog.</strong> Pull CloudTrail for Glue and Lake Formation API calls across a full billing cycle, long enough to catch monthly jobs. Every distinct role in that list is work. This number, not your data volume, is the migration estimate.</li>



<li><strong>Ask whether column or row-level control has actually been requested.</strong> If the answer is a compliance requirement with a name attached, proceed. If it is &#8220;it would be nice&#8221;, you are buying a permission model to solve a hypothetical.</li>



<li><strong>Check engine coverage against your real workloads.</strong> Confirm fine-grained access control support for each production engine in current documentation. One unsupported critical path makes hybrid mode permanent rather than transitional, which is a legitimate outcome but should be a decision, not a discovery.</li>



<li><strong>Price the federated path against a copy.</strong> For your highest-frequency cross-source query, estimate metadata Lambda invocations plus any service-managed compute, then compare against a nightly job that lands a copy in S3. Federation wins on freshness and on one less pipeline. It does not automatically win on cost.</li>



<li><strong>Decide the destination, not just the direction.</strong> &#8220;Hybrid access mode indefinitely, Lake Formation enforcement on the two regulated datasets&#8221; is a good end state. Write it down first, because migrations without a finish line become permanent half-migrations.</li>
</ol>



<h3 class="wp-block-heading">Three things to run before you commit</h3>



<p class="wp-block-paragraph">Enumerate what Athena can see today. This is your before picture, and it exposes a tooling gap worth knowing up front:</p>



<pre class="wp-block-code"><code>aws athena list-data-catalogs

# Returns catalogs registered directly with Athena, for example:
# {
#   "DataCatalogsSummary": [
#     { "CatalogName": "AwsDataCatalog", "Type": "GLUE" },
#     { "CatalogName": "cw_logs_catalog", "Type": "LAMBDA" }
#   ]
# }</code></pre>



<p class="wp-block-paragraph">S3 Tables catalogs use a <code>s3tablescatalog/bucket-name</code> naming convention and are not returned by this call, even though you can query them by naming the catalog explicitly. The console shows them through mechanisms the public API does not expose. Any inventory or drift-detection tooling built on <code>list-data-catalogs</code> will quietly under-report, so enumerate S3 Tables catalogs separately.</p>



<p class="wp-block-paragraph">Next, confirm the role you intend to use as data lake administrator can create catalogs at all. If it is not already a data lake admin, it needs an explicit grant:</p>



<pre class="wp-block-code"><code>aws lakeformation grant-permissions 
  --cli-input-json 
  '{
     "Principal": {
       "DataLakePrincipalIdentifier": "arn:aws:iam::123456789012:role/Admin"
     },
     "Resource": { "Catalog": {} },
     "Permissions": [ "CREATE_CATALOG", "DESCRIBE" ]
   }'</code></pre>



<p class="wp-block-paragraph">This avoids the common false start where catalog creation works in the console for an admin user and fails for the automation role you actually meant to use.</p>



<p class="wp-block-paragraph">Finally, if you read lakehouse tables from a Glue Spark job, fine-grained access control is not on by default. The job parameters that enable it:</p>



<pre class="wp-block-code"><code>--datalake-formats                          iceberg
--enable-lakeformation-fine-grained-access  true</code></pre>



<p class="wp-block-paragraph">Without the second one, the job reads through its IAM role and your column-level grants are not applied. That is a governance gap that looks exactly like a working pipeline, which is the worst kind.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Arguments that don&#8217;t survive contact</h2>



<ul class="wp-block-list">
<li><strong>&#8220;It&#8217;s the same Glue Data Catalog, so migrating is low risk.&#8221;</strong> True about the metadata, false about enforcement. The catalog is the same object. The thing deciding whether a query returns rows is not.</li>



<li><strong>&#8220;We&#8217;ll migrate to avoid falling behind.&#8221;</strong> AWS states plainly that you can keep using Athena, EMR, Glue and MWAA as you do today. Nothing here is a deprecation. Migrating on an invented schedule is how you end up owning a permission model nobody needed.</li>



<li><strong>&#8220;Federated catalogs mean we can retire our ETL.&#8221;</strong> Some of it, sometimes. Federation removes copies that existed only for co-location. It does not remove transformation, deduplication, late-arriving data handling or historical snapshots. Those jobs stay.</li>



<li><strong>&#8220;Iceberg REST support means no lock-in.&#8221;</strong> The table format is portable and that is real. The Lake Formation grants, LF-Tags, hybrid mode opt-ins and catalog hierarchy are not, and they become the hardest part of your platform to reproduce elsewhere. Portability moved down a layer; it did not appear.</li>



<li><strong>&#8220;We can test this in a sandbox.&#8221;</strong> You can test the mechanics. You cannot test the thing that actually breaks, which is the set of production principals depending on IAM-only access. That inventory does not exist in a sandbox.</li>
</ul>



<h2 class="wp-block-heading">How I&#8217;d decide</h2>



<p class="wp-block-paragraph">S3-only, one account, small team: I would not migrate. The same effort spent on partitioning and compaction returns more, and it keeps returning.</p>



<p class="wp-block-paragraph">Multiple governed sources, several consuming teams, a named compliance requirement for fine-grained control: migrate, and treat hybrid access mode as the destination rather than a waypoint. Enforce Lake Formation on the datasets that need it, leave the rest on IAM. The pressure to reach a &#8220;clean&#8221; fully-enforced state is aesthetic, not operational.</p>



<p class="wp-block-paragraph">Either way, start with a federated catalog over one source that is currently painful to reach. Prove the query path and the cost, and let the rest follow evidence rather than a plan drawn before anyone had numbers.</p>



<h2 class="wp-block-heading">Frequently asked questions</h2>



<h3 class="wp-block-heading">Does SageMaker Lakehouse replace AWS Glue and Athena?</h3>



<p class="wp-block-paragraph">No. It is built on the Glue Data Catalog and Lake Formation, and Athena remains one of the engines you query it with. AWS documentation states you can continue using Athena, EMR, Glue and MWAA without migrating. It is a governance and connectivity layer, not a replacement stack.</p>



<h3 class="wp-block-heading">Will a SageMaker Lakehouse migration break my existing Athena queries?</h3>



<p class="wp-block-paragraph">Reorganising into catalogs does not break queries on its own. What breaks them is revoking <code>Super</code> from <code>IAMAllowedPrincipals</code> before every consuming principal has a Lake Formation grant. Use hybrid access mode, opt principals in one at a time, and existing queries keep working throughout.</p>



<h3 class="wp-block-heading">Do I have to move data into Redshift Managed Storage?</h3>



<p class="wp-block-paragraph">No. Managed catalogs can be backed by S3, including S3 Tables, and existing S3 data lakes integrate directly. RMS is a storage choice with different cost and performance behaviour, not a requirement of the architecture.</p>



<h3 class="wp-block-heading">How much extra does it cost?</h3>



<p class="wp-block-paragraph">The catalog layer itself is not a new meter: metadata storage and API requests follow existing Glue Data Catalog pricing including free tier allowances. New costs come from what you build on it, mainly Lambda invocations for federated metadata retrieval, service-managed Redshift Serverless compute for federated Redshift queries, and RMS storage if you choose it.</p>



<h3 class="wp-block-heading">Is SageMaker Unified Studio required?</h3>



<p class="wp-block-paragraph">No. Unified Studio is the integrated console experience. The catalogs are addressable from Athena, Redshift, EMR, Glue and Iceberg-compatible engines directly. Adopting the catalog structure without adopting the studio is a valid and common choice.</p>



<h3 class="wp-block-heading">What is the smallest useful first step?</h3>



<p class="wp-block-paragraph">Create one federated catalog over a single non-S3 source that analysts currently reach through an awkward workaround. Grant read access to one team. Measure query latency and the resulting Lambda and compute charges. That gives you real numbers on your own workload, and it is trivially reversible.</p>



<h2 class="wp-block-heading">Conclusion</h2>



<p class="wp-block-paragraph">A SageMaker Lakehouse migration is not a data migration. The Glue Data Catalog underneath is the same catalog, the tables are the same tables, Athena is still Athena. What changes is which system decides whether a query returns rows.</p>



<p class="wp-block-paragraph">That makes the call simpler than the marketing implies. Several governed sources, multiple consuming teams and a real fine-grained control requirement, and the unified permission surface is worth the work. Without them, you are taking on a permission model and a Lambda-backed metadata path in exchange for a nicer console.</p>



<p class="wp-block-paragraph">Whichever way you go: register in hybrid access mode, opt principals in one at a time, and do not revoke <code>IAMAllowedPrincipals</code> until you can name every principal that will notice.</p>



<hr class="wp-block-separator has-alpha-channel-opacity"/>



<h2 class="wp-block-heading">Need help deciding, or unpicking a migration already underway?</h2>



<p class="wp-block-paragraph">An outside read on the actual inventory usually saves more time than any amount of architecture discussion. Things I help with:</p>



<ul class="wp-block-list">
<li>Building the principal inventory from CloudTrail so you know exactly who depends on IAM-only Data Catalog access before anything is revoked</li>



<li>Designing a hybrid access mode rollout that opts teams in incrementally, with a verification step and a rollback position at every stage</li>



<li>Modelling federated catalog costs against the copy-the-data alternative for your query patterns, including the service-managed compute nobody budgets for</li>



<li>Diagnosing Lake Formation access denied errors across Athena, Glue Spark and Redshift, including the fine-grained access flags that silently bypass your grants</li>



<li>Cost allocation tagging so lakehouse spend is attributable per catalog and per team from the first query rather than retrofitted later</li>



<li>Honest second opinions on whether a migration is worth doing at all, including the answer that it isn&#8217;t</li>
</ul>



<p class="wp-block-paragraph">Send me a failing query, a Lake Formation grant listing or a CloudTrail export and I&#8217;ll tell you what I&#8217;d do with it.</p>



<div class="wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex">
<div class="wp-block-button"><a class="wp-block-button__link wp-element-button" href="https://www.upwork.com/freelancers/~01f15a912ad84a6620" target="_blank" rel="noreferrer noopener">Work with me on Upwork</a></div>
</div>

<p>The post <a href="https://john-nessime.com/blog/cloud-computing/sagemaker-lakehouse-migration/">SageMaker Lakehouse vs Glue + Athena: Is Migration Worth It?</a> appeared first on <a href="https://john-nessime.com/blog">John Nessime</a>.</p>
]]></content:encoded>
					
		
		
			</item>
	</channel>
</rss>
