{"id":683,"date":"2026-11-14T09:01:31","date_gmt":"2026-11-14T07:01:31","guid":{"rendered":"https:\/\/john-nessime.com\/blog\/?p=683"},"modified":"2026-11-14T09:01:31","modified_gmt":"2026-11-14T07:01:31","slug":"s3-tables-vs-iceberg-on-s3","status":"publish","type":"post","link":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/","title":{"rendered":"S3 Tables vs Iceberg on S3: When Managed Compaction Is Worth Paying For"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">The dashboard that used to load in eight seconds now takes ninety. Nobody changed the query, nobody changed the schema, and the table is barely bigger than it was last quarter. Someone opens a ticket titled &#8220;Athena slow?&#8221; and it sits there, because there is no error to point at.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What changed is the file count. A streaming job has been writing small Parquet files every few minutes for months, and each one is another entry the query planner opens before it can plan anything. The data is fine. The layout rotted.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That is the problem underneath the whole S3 Tables vs Iceberg on S3 argument. It is not a storage question. It is a question about who owns table maintenance, and what happens when the answer is nobody. This post covers what S3 Tables actually runs for you, where each option wins, why the access path matters more than most comparisons admit, how the two bills are shaped, and a decision procedure you can apply to a specific table.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The failure that doesn&#8217;t announce itself<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Iceberg on general purpose S3 works beautifully on day one. You write Parquet, commit a snapshot, Athena reads it. Nothing about that first week tells you it needs upkeep. Three things degrade quietly afterward.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Small files accumulate.<\/strong> Planning cost scales with how many files the planner considers, not with how much data you actually scan.<\/li>\n\n\n\n<li><strong>Snapshots pile up.<\/strong> Every write creates a table version, and old snapshots keep their data files alive. Storage grows even when the row count is flat.<\/li>\n\n\n\n<li><strong>Orphan files never get cleaned.<\/strong> A killed write can leave files no snapshot references. They are invisible to every query and they bill every month forever.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">None of that produces an alert. No threshold crossing, no failed job, no red panel. It is a slope, not a cliff, which is why teams hit the &#8220;Athena slow?&#8221; ticket eighteen months in with nothing to blame.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Amazon S3 Tables exists to take those three jobs off you. That is a real pitch. The question is what you hand over to accept it.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What S3 Tables actually runs for you<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">S3 Tables introduces a distinct bucket type, the table bucket, which holds tables as subresources rather than a prefix full of objects you manage. Inside it, S3 runs three maintenance operations, all on by default. They map one to one onto the standard Apache Iceberg maintenance routines, which tells you exactly what the do-it-yourself equivalent costs in effort.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Compaction<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Compaction merges small objects into larger ones and applies row-level deletes while it does. It is configured per table, with a target file size defaulting to 512MB and settable as low as 64MB. It reads Parquet, Avro and ORC, and writes Parquet by default.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Two limits matter more than the defaults. Compaction does not support the <code>Fixed<\/code> data type, and it does not support brotli or lz4 compression. If your writer uses lz4 because someone benchmarked it years ago, you are paying for a managed service that cannot do the job you bought it for. Check the writer config before you migrate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Also worth internalizing: compaction runs on an automated schedule you do not control. There is no &#8220;compact now, I have a board report in an hour&#8221; button. Between a burst of small writes and the next pass, the table sits in exactly the state you were trying to avoid.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Snapshot management, and the trap inside it<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Snapshot management expires old table versions. Two per-table values govern it: a minimum number of snapshots to keep, and a maximum snapshot age. Both conditions must be satisfied before anything is removed, so the minimum count wins over the age setting on a quiet table.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Here is the part that bites. Snapshot management does not honor retention set as Iceberg table properties. Configure a branch or tag based retention policy, or a retention in the table metadata longer than what you set through the maintenance API, and snapshot management is <em>disabled<\/em> for that table.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Not overridden. Disabled. S3 stops expiring snapshots, you keep paying storage for every version, and nothing tells you. Someone runs an innocuous <code>ALTER TABLE ... SET TBLPROPERTIES<\/code> to tag a release for reproducibility, and months later the storage line has doubled on a table nobody thought was growing. If you adopt S3 Tables, make &#8220;nobody sets snapshot retention in table properties&#8221; a written rule and audit for it.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Unreferenced file removal<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">This is orphan cleanup. It deletes data and metadata files no longer referenced by Iceberg metadata once they pass a retention window, configured as a number of days for unreferenced files and a number of days for non-current files.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The structural detail is scope. Compaction and snapshot management are per table. Unreferenced file removal is configured <em>at the table bucket level only<\/em>. You cannot give a compliance-sensitive table a one day orphan window and a scratch table ten days in the same bucket. Different retention behavior means different table buckets, and that is a design-time decision.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Editing any of this needs the maintenance configuration permissions, and automated runs surface in CloudTrail, which is your only real visibility into whether maintenance is happening.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>s3tables:GetTableMaintenanceConfiguration\ns3tables:PutTableMaintenanceConfiguration<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Grant those to whoever owns the table, not to the pipeline role. A pipeline should never be able to switch off its own cleanup.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">S3 Tables: where it wins, where it doesn&#8217;t<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Where it wins.<\/strong> A high-frequency writer with no dedicated data platform engineer. Change data capture, clickstream, IoT telemetry, log tables: anything producing a steady drip of small files that nobody will schedule a compaction job for. AWS also positions table buckets as supporting substantially higher transaction rates than general purpose buckets, which is a vendor claim worth testing against your own write pattern rather than taking from a marketing page.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">It wins again when you want maintenance to be a property of the storage rather than a job in someone&#8217;s scheduler. A compaction job in Airflow can be deleted, break silently, or lose its owner when a contract ends. Maintenance that lives in the bucket does not.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Where it doesn&#8217;t.<\/strong> Table buckets are not general purpose buckets, and the gap is wider than the name suggests.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Public access policies and presigned URLs are not supported. If any workflow hands out object URLs, that pattern is gone.<\/li>\n\n\n\n<li>Table bucket and table access policies are capped at 20 KB, which sounds generous until you are writing per-tenant conditions.<\/li>\n\n\n\n<li>Replication has edges: replicas are read-only and compaction does not run on them, tables with tags or branches cannot be replicated, and a table upgraded from Iceberg V2 to V3 cannot be replicated afterward.<\/li>\n\n\n\n<li>Your lifecycle rules, inventory reports, bucket-level scripts and storage class habits do not transfer directly.<\/li>\n\n\n\n<li>Compaction timing is not yours. Where layout freshness matters within the hour, the scheduled model is a genuine constraint.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Iceberg on general purpose S3: where it wins, where it doesn&#8217;t<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Give this option its real case, because the &#8220;unmanaged&#8221; framing is out of date. Iceberg tables registered in the AWS Glue Data Catalog can have automatic table optimization enabled, covering compaction, snapshot retention and orphan file management. The optimizer watches partitions and starts compaction once file counts and sizes cross a threshold. The choice is no longer managed versus manual. It is closer to managed at the storage layer versus managed at the catalog layer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Where it wins.<\/strong><\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>You control timing.<\/strong> Compact before a heavy reporting window or after a backfill, because you call the procedure.<\/li>\n\n\n\n<li><strong>One bucket type, one mental model.<\/strong> Lifecycle rules, storage classes, bucket policies and VPC endpoint patterns keep working.<\/li>\n\n\n\n<li><strong>Cheaper base storage rate.<\/strong> Table bucket storage carries a premium, and at volume that is not a rounding error.<\/li>\n\n\n\n<li><strong>Exit is cheap.<\/strong> Iceberg and Parquet in a bucket you own. Moving catalogs is a metadata operation, not a data migration.<\/li>\n\n\n\n<li><strong>Mixed workloads.<\/strong> Raw drops, partner exports and Iceberg tables can share one storage architecture.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Where it doesn&#8217;t.<\/strong> You own the failure mode. If nobody enables the optimizer or schedules the procedures, nothing runs and nothing complains. The work is not complicated, which is exactly the problem: it is easy enough that everyone assumes someone else did it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The manual path is three Iceberg Spark procedures on a schedule you own:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>-- consolidate small data files\nCALL catalog.system.rewrite_data_files(table =&gt; 'db.events')\n\n-- drop old table versions\nCALL catalog.system.expire_snapshots(table =&gt; 'db.events')\n\n-- delete files no snapshot references\nCALL catalog.system.remove_orphan_files(table =&gt; 'db.events')<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Each accepts arguments for targets and retention that you should set deliberately rather than accept by default. Treat <code>remove_orphan_files<\/code> with care: it compares storage against metadata, and a short retention window can delete files a concurrent write is still committing.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The other cost is compute. Those procedures run on Spark, so an EMR cluster or a Glue job, with its own IAM, network path, failure modes and bill. That is the real thing you buy out of with S3 Tables, and it is a bigger line than people estimate.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The access path is a bigger difference than the maintenance<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This is the part that generates support tickets, and most comparisons skip it. There are two REST catalog paths to an S3 table, with different authorization models.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Direct, through the S3 Tables Iceberg REST endpoint.<\/strong> Any Iceberg-compatible client connects. Authorization is IAM only, via identity-based policies or resource-based policies on the table and table bucket. AWS suggests this for self-managed or partner catalogs, or basic read and write against a single table bucket.<\/li>\n\n\n\n<li><strong>Through the AWS Glue Data Catalog integration.<\/strong> Tables register as Data Catalog objects, visible to Athena, Redshift, EMR, Glue ETL and Firehose, with third-party clients reaching them through the Glue Iceberg REST endpoint. Authorization is IAM <em>and<\/em> Lake Formation. Both checks must pass.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">That second point is where teams lose an afternoon. A PyIceberg client with a correct IAM policy still gets denied, because integrating with the Data Catalog registered the tables as Lake Formation resources and no Lake Formation grant exists. The IAM policy is not wrong. It is not sufficient. For external engines to receive credentials, Lake Formation also has to permit full table access for external engines, and a data lake administrator has to apply the grants.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">If you already run Lake Formation for fine-grained access control, this is a feature and column-level permissions come along for free. If you do not, you have added a governance service, an administrator role and a second permissions model to a project that was supposed to be about file sizes. Price that in honestly.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How the two bills are shaped<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Skip the per-gigabyte rates. They change, and the decision does not live there. Look at the shape instead.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>S3 Tables bills on four dimensions:<\/strong> storage, requests, a per-object monitoring fee for every object in the table bucket, and compaction charged both per object and per byte processed. The monitoring fee is the one people miss, because it scales with object <em>count<\/em>. The same workload that makes compaction expensive makes monitoring expensive, and both are driven by how many small files your writer produces.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Iceberg on general purpose S3 bills on:<\/strong> storage, requests, and the compute you run maintenance on. That compute is lumpy and easy to underestimate, but it is a knob you control. A weekly compaction window on spot capacity costs very differently from continuous optimization.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One lever moves both models: <strong>batch your writes<\/strong>. Committing every thirty seconds instead of every five minutes multiplies object count by ten, and object count feeds the monitoring fee, the compaction fee, the planning time and the Spark hours alike. Fixing the writer is almost always cheaper than choosing storage around a bad writer.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Whichever you pick, make the cost visible. Cost allocation tags on the table bucket, a CloudWatch or Grafana panel on object counts, and CloudTrail on maintenance events will teach you more in a month than any spreadsheet estimate teaches you today.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">A decision procedure for S3 Tables vs Iceberg on S3<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Work through these for a specific table, not for your platform as a whole. Different tables can legitimately land in different places.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Count the commits per hour.<\/strong> A handful of commits a day from a batch job writing large files is not a small file problem, and managed compaction solves nothing. Stay on general purpose S3.<\/li>\n\n\n\n<li><strong>Check the writer&#8217;s compression and types.<\/strong> lz4 or brotli, or a <code>Fixed<\/code> column, and S3 Tables compaction will not process those files. Fix the writer or stay put.<\/li>\n\n\n\n<li><strong>Ask who owns maintenance today, by name.<\/strong> If it is a person who will still be here next year and a job that alerts on failure, do it yourself. If the answer is a shrug, that shrug is what you are paying to replace.<\/li>\n\n\n\n<li><strong>Check for presigned URLs, public policies, lifecycle rules or large access policies on this data.<\/strong> Any yes rules out a table bucket for that table.<\/li>\n\n\n\n<li><strong>Decide the governance model before the storage model.<\/strong> Want Lake Formation fine-grained control? The Glue Data Catalog path is a real benefit. Want IAM-only simplicity? Use direct access and keep Lake Formation out of it.<\/li>\n\n\n\n<li><strong>Confirm nothing sets snapshot retention in table properties.<\/strong> If your pipeline or dbt project does, change it or accept that managed snapshot expiry will be silently off.<\/li>\n\n\n\n<li><strong>Try the middle option first.<\/strong> Enable Glue Data Catalog table optimization on the existing table. Reversible, no data movement, and it answers &#8220;is automated maintenance enough&#8221; without committing you to a new bucket type.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">That last step is the one I reach for first in almost every case. It delivers the outcome you actually want, maintenance that runs without a human, at the cost of a checkbox rather than a migration.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Where teams get this wrong<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Treating it as a storage migration.<\/strong> It is an operations and governance decision that happens to involve storage.<\/li>\n\n\n\n<li><strong>Migrating the wrong table.<\/strong> Teams pick the biggest one. The right candidate is the highest commit frequency, which is often small.<\/li>\n\n\n\n<li><strong>Assuming enabled means running.<\/strong> Table properties can disable snapshot expiry and unsupported compression can skip compaction. Verify in CloudTrail that operations actually fire.<\/li>\n\n\n\n<li><strong>Forgetting concurrency.<\/strong> Iceberg uses optimistic concurrency, so compaction and your writes can conflict. Compaction retries on failure; your pipeline needs retry logic too, on both sides of this comparison.<\/li>\n\n\n\n<li><strong>Skipping the exit plan.<\/strong> The data stays open, the bucket type does not. Write down how you would move back before you move in.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Is S3 Tables just Iceberg with extra steps?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. It is standard Apache Iceberg with maintenance run by the service and a purpose-built bucket type underneath. The table format is identical, so engines and data stay portable. What differs is who executes maintenance and which bucket-level features you give up to get it.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can I get managed maintenance without moving to a table bucket?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes. AWS Glue Data Catalog table optimization brings automatic compaction, snapshot retention and orphan file management to Iceberg tables in general purpose buckets. For most teams asking this question, that is the answer they actually wanted.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Does compaction break time travel?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Compaction writes a new snapshot, so history survives. Snapshot management is what removes time travel, because expiring a snapshot deletes its metadata and can delete the associated data files. If you rely on travelling to a specific point, set the minimum snapshot count and maximum snapshot age to cover that window deliberately.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Why does my PyIceberg client get access denied on S3 Tables?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Usually because you are going through the Glue Data Catalog integration, where both IAM and Lake Formation must authorize the request and only the IAM half is configured. Either add the Lake Formation grants and enable external engine access, or switch to the S3 Tables Iceberg REST endpoint for direct IAM-only access.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Is S3 Tables more expensive than regular S3?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">The storage rate carries a premium, plus a per-object monitoring fee and compaction charges. Against that, you remove the compute you were spending on maintenance. Whether the total is higher depends almost entirely on object count, so model it from your file counts rather than your terabytes.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can I disable compaction and control it myself?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes, compaction can be disabled per table through the maintenance configuration API, which also stops the associated charges. Reasonable if you already have a tuned compaction job, though it undercuts most of the reason to be on a table bucket.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Which should a small team pick?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A small team with one high-churn streaming table and no data platform engineer is the clearest fit for S3 Tables. A small team running a nightly batch load into a few large tables should stay on general purpose S3 and turn on Glue table optimization if maintenance is the worry.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The one thing worth remembering<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">S3 Tables vs Iceberg on S3 is not really a performance comparison, and not really a cost comparison either. Both store the same Parquet under the same open table format, and either performs well when the layout is maintained.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What you are choosing is where responsibility for compaction, snapshot expiry and orphan cleanup lives, and how confident you are that it will still be happening in two years when the person who set it up has moved on. Low confidence and a high commit rate means managed Iceberg earns its premium. A real owner, a real schedule and large batch writes means you are paying for a problem you do not have.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Either way, check the two silent failures before you ship: table properties that disable snapshot expiry, and a writer whose compression codec compaction cannot read.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Need help deciding, or fixing a table that already rotted?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This is the kind of work I do for teams running Iceberg on AWS:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Auditing Iceberg tables for small file growth, snapshot bloat and orphan files, with a before-and-after on planning time and storage<\/li>\n\n\n\n<li>Modelling S3 Tables against your real object counts and commit frequency, so the decision comes from your numbers rather than a pricing page example<\/li>\n\n\n\n<li>Setting up AWS Glue Data Catalog table optimization as the low-risk middle path, including the IAM and network configuration it needs<\/li>\n\n\n\n<li>Untangling Lake Formation and IAM when external Iceberg clients get denied through the Glue REST endpoint<\/li>\n\n\n\n<li>Tuning the writer side, where the actual fix usually lives: commit batching, target file sizes, partition layout and compression choices<\/li>\n\n\n\n<li>Building monitoring that makes maintenance visible, so degradation shows up on a dashboard instead of in a &#8220;why is Athena slow&#8221; ticket<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Send me a table&#8217;s manifest summary, a Glue optimizer config, or the planning stage of a slow query, and I will tell you what I would change first.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.upwork.com\/freelancers\/~01f15a912ad84a6620\" target=\"_blank\" rel=\"noreferrer noopener\">Work with me on Upwork<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Managed Iceberg is not a storage upgrade, it is a decision about who owns compaction, snapshot expiry and orphan cleanup. A practical comparison of S3 Tables against Iceberg in general purpose S3 buckets, covering the maintenance you actually get, the Lake Formation trap in the access path, how the two bills are shaped, and a seven-step decision procedure you can run against a single table.<\/p>\n","protected":false},"author":1,"featured_media":1228,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25,498,674],"tags":[163,161,164,162,187,160,855,364,237,584,959,582,579,941,583,958,251],"class_list":["post-683","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-computing","category-data-engineering","category-finops","tag-amazon-athena","tag-amazon-s3","tag-apache-iceberg","tag-aws-glue","tag-cost-optimization","tag-data-lake","tag-iceberg-rest-catalog","tag-lake-formation","tag-lakehouse","tag-optimistic-concurrency","tag-orphan-files","tag-s3-tables","tag-small-files","tag-snapshots","tag-table-compaction","tag-table-maintenance","tag-vendor-lock-in","entry","has-media"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.8 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>S3 Tables vs Iceberg on S3: When Managed Wins<\/title>\n<meta name=\"description\" content=\"S3 Tables vs Iceberg on S3, compared on maintenance, catalog access, cost shape and exit cost, plus a decision procedure for picking one per table.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"S3 Tables vs Iceberg on S3: When Managed Wins\" \/>\n<meta property=\"og:description\" content=\"S3 Tables vs Iceberg on S3, compared on maintenance, catalog access, cost shape and exit cost, plus a decision procedure for picking one per table.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/\" \/>\n<meta property=\"og:site_name\" content=\"John Nessime\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/J.Nessime\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/J.Nessime\" \/>\n<meta property=\"article:published_time\" content=\"2026-11-14T07:01:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/s3-tables-vs-iceberg-featured-image.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1536\" \/>\n\t<meta property=\"og:image:height\" content=\"1024\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"John Nessime\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"John Nessime\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"14 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/\"},\"author\":{\"name\":\"John Nessime\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"headline\":\"S3 Tables vs Iceberg on S3: When Managed Compaction Is Worth Paying For\",\"datePublished\":\"2026-11-14T07:01:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/\"},\"wordCount\":3075,\"publisher\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"image\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/s3-tables-vs-iceberg-featured-image.png\",\"keywords\":[\"Amazon Athena\",\"Amazon S3\",\"Apache Iceberg\",\"AWS Glue\",\"Cost Optimization\",\"Data Lake\",\"Iceberg REST Catalog\",\"Lake Formation\",\"Lakehouse\",\"Optimistic Concurrency\",\"Orphan Files\",\"S3 Tables\",\"Small Files\",\"Snapshots\",\"Table Compaction\",\"Table Maintenance\",\"Vendor Lock-In\"],\"articleSection\":[\"Cloud Computing\",\"Data Engineering\",\"FinOps\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/\",\"name\":\"S3 Tables vs Iceberg on S3: When Managed Wins\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/s3-tables-vs-iceberg-featured-image.png\",\"datePublished\":\"2026-11-14T07:01:31+00:00\",\"description\":\"S3 Tables vs Iceberg on S3, compared on maintenance, catalog access, cost shape and exit cost, plus a decision procedure for picking one per table.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/#primaryimage\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/s3-tables-vs-iceberg-featured-image.png\",\"contentUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/s3-tables-vs-iceberg-featured-image.png\",\"width\":1536,\"height\":1024,\"caption\":\"S3 Tables vs Apache Iceberg on S3 comparison illustrating managed compaction, automatic maintenance, Parquet data, Athena queries, and key storage considerations.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-vs-iceberg-on-s3\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"S3 Tables vs Iceberg on S3: When Managed Compaction Is Worth Paying For\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/\",\"name\":\"John Nessime\",\"description\":\"Cloud, DevOps, Data &amp; AI \u2014 Built, Tested, Explained\",\"publisher\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\",\"name\":\"John Nessime\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"contentUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"width\":512,\"height\":512,\"caption\":\"John Nessime\"},\"logo\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\"},\"description\":\"AWS Certified Solutions Architect helping businesses build reliable cloud, data, reporting, and automation solutions. I help startups, agencies, and growing businesses replace manual processes and disconnected data with practical AWS architectures, clean data pipelines, useful dashboards, and maintainable automation.\",\"sameAs\":[\"https:\\\/\\\/john-nessime.com\\\/blog\",\"https:\\\/\\\/www.facebook.com\\\/J.Nessime\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/john-m-nessime\"],\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/author\\\/johnnessime\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"S3 Tables vs Iceberg on S3: When Managed Wins","description":"S3 Tables vs Iceberg on S3, compared on maintenance, catalog access, cost shape and exit cost, plus a decision procedure for picking one per table.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/","og_locale":"en_US","og_type":"article","og_title":"S3 Tables vs Iceberg on S3: When Managed Wins","og_description":"S3 Tables vs Iceberg on S3, compared on maintenance, catalog access, cost shape and exit cost, plus a decision procedure for picking one per table.","og_url":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/","og_site_name":"John Nessime","article_publisher":"https:\/\/www.facebook.com\/J.Nessime","article_author":"https:\/\/www.facebook.com\/J.Nessime","article_published_time":"2026-11-14T07:01:31+00:00","og_image":[{"width":1536,"height":1024,"url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/s3-tables-vs-iceberg-featured-image.png","type":"image\/png"}],"author":"John Nessime","twitter_card":"summary_large_image","twitter_misc":{"Written by":"John Nessime","Est. reading time":"14 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/#article","isPartOf":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/"},"author":{"name":"John Nessime","@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"headline":"S3 Tables vs Iceberg on S3: When Managed Compaction Is Worth Paying For","datePublished":"2026-11-14T07:01:31+00:00","mainEntityOfPage":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/"},"wordCount":3075,"publisher":{"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"image":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/#primaryimage"},"thumbnailUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/s3-tables-vs-iceberg-featured-image.png","keywords":["Amazon Athena","Amazon S3","Apache Iceberg","AWS Glue","Cost Optimization","Data Lake","Iceberg REST Catalog","Lake Formation","Lakehouse","Optimistic Concurrency","Orphan Files","S3 Tables","Small Files","Snapshots","Table Compaction","Table Maintenance","Vendor Lock-In"],"articleSection":["Cloud Computing","Data Engineering","FinOps"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/","url":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/","name":"S3 Tables vs Iceberg on S3: When Managed Wins","isPartOf":{"@id":"https:\/\/john-nessime.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/#primaryimage"},"image":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/#primaryimage"},"thumbnailUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/s3-tables-vs-iceberg-featured-image.png","datePublished":"2026-11-14T07:01:31+00:00","description":"S3 Tables vs Iceberg on S3, compared on maintenance, catalog access, cost shape and exit cost, plus a decision procedure for picking one per table.","breadcrumb":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/#primaryimage","url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/s3-tables-vs-iceberg-featured-image.png","contentUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/s3-tables-vs-iceberg-featured-image.png","width":1536,"height":1024,"caption":"S3 Tables vs Apache Iceberg on S3 comparison illustrating managed compaction, automatic maintenance, Parquet data, Athena queries, and key storage considerations."},{"@type":"BreadcrumbList","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-vs-iceberg-on-s3\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/john-nessime.com\/blog\/"},{"@type":"ListItem","position":2,"name":"S3 Tables vs Iceberg on S3: When Managed Compaction Is Worth Paying For"}]},{"@type":"WebSite","@id":"https:\/\/john-nessime.com\/blog\/#website","url":"https:\/\/john-nessime.com\/blog\/","name":"John Nessime","description":"Cloud, DevOps, Data &amp; AI \u2014 Built, Tested, Explained","publisher":{"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/john-nessime.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105","name":"John Nessime","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","contentUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","width":512,"height":512,"caption":"John Nessime"},"logo":{"@id":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png"},"description":"AWS Certified Solutions Architect helping businesses build reliable cloud, data, reporting, and automation solutions. I help startups, agencies, and growing businesses replace manual processes and disconnected data with practical AWS architectures, clean data pipelines, useful dashboards, and maintainable automation.","sameAs":["https:\/\/john-nessime.com\/blog","https:\/\/www.facebook.com\/J.Nessime","https:\/\/www.linkedin.com\/in\/john-m-nessime"],"url":"https:\/\/john-nessime.com\/blog\/author\/johnnessime\/"}]}},"_links":{"self":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/683","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/comments?post=683"}],"version-history":[{"count":1,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/683\/revisions"}],"predecessor-version":[{"id":685,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/683\/revisions\/685"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/media\/1228"}],"wp:attachment":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/media?parent=683"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/categories?post=683"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/tags?post=683"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}