{"id":773,"date":"2026-10-22T09:02:31","date_gmt":"2026-10-22T06:02:31","guid":{"rendered":"https:\/\/john-nessime.com\/blog\/?p=773"},"modified":"2026-10-22T09:02:31","modified_gmt":"2026-10-22T06:02:31","slug":"s3-tables-compaction-production","status":"publish","type":"post","link":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/","title":{"rendered":"S3 Tables Compaction in Production: The Cost of Convenience"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">The table bucket had been in production a couple of months. Ingest volume was flat, retention said ninety days, and the storage line on the bill kept climbing anyway. Compaction was enabled. The console showed nothing wrong. Someone had already checked twice.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The answer was one API call away, and nobody had made it. Snapshot management had been failing for the entire table since a developer added an Iceberg branch for a backfill experiment. Compaction kept running happily. Old snapshots never expired, the files they pinned never became eligible for deletion, and the bucket grew. No alarm, no console banner, no email.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That is the shape of most S3 Tables compaction problems in production. The managed maintenance is good at what it does. What it is bad at is telling you when it has stopped doing it. This post covers the failure modes that bite, the storage tail that keeps billing after you think data is gone, how the compaction charge accumulates, and the handful of knobs you actually control.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What managed compaction actually does<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">S3 Tables runs three maintenance jobs, all on by default. Compaction and snapshot management are per table. Unreferenced file removal is per table bucket and cannot be overridden per table.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Compaction is the Iceberg <code>rewriteDataFiles<\/code> routine with scheduling taken off your hands. It combines small objects into larger ones, applies pending row-level deletes on the way through, and writes the result as the table&#8217;s newest snapshot. Target file size defaults to 512MB and accepts 64MB to 512MB. Parquet row groups inside use a 128MB default.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There are four strategies. <code>auto<\/code> is the default and picks for you: sort order defined in metadata means <code>sort<\/code>, otherwise <code>binpack<\/code>. <code>binpack<\/code> packs small files into bigger ones. <code>sort<\/code> orders rows hierarchically by your chosen columns so engines prune more files. <code>z-order<\/code> interleaves several columns into one sortable value, for queries filtering on multiple dimensions rather than one leading column.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Both <code>sort<\/code> and <code>z-order<\/code> need a <code>sort_order<\/code> in your Iceberg table properties and the <code>s3tables:GetTableData<\/code> permission to set. They also cost more per byte processed than <code>binpack<\/code>, because they do more work.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Failure family one: maintenance that fails without telling anyone<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">This is the one worth reading the post for. Maintenance jobs have four states: <code>Successful<\/code>, <code>Failed<\/code>, <code>Disabled<\/code>, and <code>Not_Yet_Run<\/code>. There is exactly one place that state is exposed on demand, and it is an API call you have to remember to make.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>aws s3tables get-table-maintenance-job-status \n  --table-bucket-arn arn:aws:s3tables:REGION:ACCOUNT_ID:bucket\/BUCKET_NAME \n  --namespace my_namespace \n  --name my_table<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">The response carries a block for each of <code>icebergCompaction<\/code>, <code>icebergSnapshotManagement<\/code>, and <code>icebergUnreferencedFileRemoval<\/code>, each with its own status, a <code>lastRunTimestamp<\/code>, and a <code>failureMessage<\/code>. The three are independent. Compaction reporting <code>Successful<\/code> tells you nothing about whether the other two are running.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The failure messages are worth knowing before you see them, because several of them describe conditions your own pipeline creates.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Tags and branches disable snapshot expiry for the whole table<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If any user-defined tag or branch exists on the table, snapshot management fails. Not partially. The whole table stops expiring snapshots, and the message says snapshot management does not support user-defined tags or references. This applies even when the tag or branch has a short retention period of its own.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Branches are a normal Iceberg workflow. Someone creating one for a write-audit-publish flow or a backfill is not doing anything wrong. They are just silently turning off garbage collection for that table, and the effect shows up on next month&#8217;s bill rather than in their terminal.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Iceberg retention properties conflict with the managed config<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If <code>history.expire.max-snapshot-age-ms<\/code> or <code>history.expire.min-snapshots-to-keep<\/code> is set as an Iceberg table property, snapshot management fails for the entire table regardless of the value. The managed service will not reconcile its own configuration against properties in <code>metadata.json<\/code>. It refuses instead.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This catches teams migrating a self-managed Iceberg table into a table bucket, because those properties are exactly what a self-managed setup uses. Remove them and let the maintenance API own retention.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>ALTER TABLE mydb.mytable UNSET TBLPROPERTIES ('history.expire.max-snapshot-age-ms');\nALTER TABLE mydb.mytable UNSET TBLPROPERTIES ('history.expire.min-snapshots-to-keep');<\/code><\/pre>\n\n\n\n<h3 class=\"wp-block-heading\">gc.enabled set to false<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Setting the Iceberg <code>gc.enabled<\/code> table property to <code>false<\/code> stops both snapshot management and unreferenced file removal. Leave it unset or explicitly <code>true<\/code>. Teams sometimes set it defensively during a migration and then forget, which produces a table that compacts forever and cleans up never.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">KMS key policy missing the maintenance principal<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">If the table is encrypted with a customer managed key, that key&#8217;s policy has to grant access to the service principal <code>maintenance.s3tables.amazonaws.com<\/code>. Without it you get an insufficient-access failure and all maintenance stops. Catch this in Terraform review rather than production, because the table reads and writes perfectly while maintenance is dead.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Failure family two: the storage tail you keep paying for<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Deleting rows from an Iceberg table does not delete bytes from S3. Neither does compacting them. Understanding the cascade is the difference between predicting your storage bill and being surprised by it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When compaction rewrites files, the originals stay because older snapshots still reference them. They become removable only once every referencing snapshot expires. Snapshot management drives that expiry from <code>minSnapshotsToKeep<\/code> (default 1) and <code>maxSnapshotAgeHours<\/code> (default 120). Both conditions must be satisfied before a snapshot goes.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When a snapshot expires, S3 marks objects referenced only by it as noncurrent. It does not delete them. They sit noncurrent for <code>nonCurrentDays<\/code> (default 10), then are deleted permanently.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Add that up. On defaults, a byte compaction superseded today is still on your bill roughly fifteen days from now. Files never referenced by any snapshot take a parallel path through <code>unreferencedDays<\/code> (default 3) into the same noncurrent window.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Two things follow. A table under heavy rewrite pressure carries a persistent storage overhang that is a function of your write rate and these windows, not of your logical table size. And those windows are your fastest cost lever if you do not need long time travel, configured in two different places.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Per table: how long snapshots survive\naws s3tables put-table-maintenance-configuration \n  --table-bucket-arn arn:aws:s3tables:REGION:ACCOUNT_ID:bucket\/BUCKET_NAME \n  --namespace my_namespace \n  --name my_table \n  --type icebergSnapshotManagement \n  --value '{\"status\":\"enabled\",\"settings\":{\"icebergSnapshotManagement\":{\"minSnapshotsToKeep\":3,\"maxSnapshotAgeHours\":48}}}'\n\n# Per bucket: how long dead files linger before deletion\naws s3tables put-table-bucket-maintenance-configuration \n  --table-bucket-arn arn:aws:s3tables:REGION:ACCOUNT_ID:bucket\/BUCKET_NAME \n  --type icebergUnreferencedFileRemoval \n  --value '{\"status\":\"enabled\",\"settings\":{\"icebergUnreferencedFileRemoval\":{\"unreferencedDays\":2,\"nonCurrentDays\":3}}}'<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Be deliberate. Shortening these windows shortens your time travel range and your recovery options. Deletion of noncurrent objects is permanent, and recovering them means a support case, if it is possible at all. I would rather pay for a few extra days of overhang on a table I might need to roll back than discover the limit during an incident.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Failure family three: write amplification and the compaction charge<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">S3 Tables bills storage, requests, a per-object monitoring fee for every object in the bucket, and, when compaction is on, both the objects and the bytes processed. Rates change and vary by region, so check the current S3 pricing page rather than a number from a blog post. What matters is the shape.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The shape is that compaction charges scale with how badly your writers behave, not with how big your table is. A batch job writing a few large files an hour barely registers. A streaming pipeline committing every few seconds produces a huge number of tiny objects, and every byte gets read and rewritten at least once, sometimes more as small compacted files merge again into larger ones.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">People reach for <code>targetFileSizeMB<\/code> first. Wrong lever. Changing the target changes the destination, not how much data has to move to get there.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The lever that works is the commit pattern on the write side. Buffering so the pipeline commits larger files less often reduces object count, monitoring fees, compaction object charges, and bytes processed all at once. This is the trade S3 Tables makes explicit: the service takes the maintenance work off your plate and prices it, so a writer that generates more work pays more for it.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Because compaction is a separate billing dimension, break it out in whatever cost tooling you already use. Cost Explorer with usage-type grouping will show it, and platforms like Vantage or CloudZero make it easier to watch alongside the rest of your S3 spend.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Failure family four: files compaction quietly skips<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Compaction supports Parquet, Avro, and ORC. It does not support the <code>Fixed<\/code> data type, nor the <code>brotli<\/code> or <code>lz4<\/code> compression codecs.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That last one matters because <code>lz4<\/code> is a reasonable choice for a write-heavy pipeline where encoding speed counts. Pick it and you get exactly the outcome you were avoiding: a table full of small files that the compaction you are paying for will not consolidate.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Compaction also writes Parquet by default even when source files are Avro or ORC. To keep the original format, set the <code>write.format.default<\/code> table property to <code>avro<\/code> or <code>orc<\/code>. Most people want Parquet for read performance, but if a downstream consumer expects a specific format, this is where you say so.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Failure family five: compaction and your writers fighting over the table<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Iceberg uses optimistic concurrency with conflict detection to arbitrate commits. As far as the table is concerned, compaction is just another writer. When a compaction commit and one of yours touch overlapping state, one loses.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When compaction loses you see <code>Failed<\/code> with a message about failing to commit because of out-of-date metadata, and a note that it will retry. Benign in isolation. Seen persistently, it means your write pattern and the maintenance schedule collide often enough that compaction is not keeping up, and the small-file problem quietly worsens while status flips between <code>Failed<\/code> and <code>Successful<\/code>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">When your job loses, you get a commit failure in your pipeline. AWS explicitly recommends retry logic on your side, and that is not boilerplate. A pipeline without commit retries, against a table with managed compaction on, will fail intermittently for reasons that have nothing to do with your code.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Turning managed compaction off, and when that is right<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Start with the honest case for leaving it on. Table maintenance is real operational work: a Spark cluster or scheduler, a job that has to run, alerting for when it does not, and someone who understands why it failed at 3am. Managed compaction removes all of that for a per-use fee. For most teams with a handful of tables and batch-shaped ingest, that is the right call. The cost is small and the alternative has a salary attached.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The case against is control. You cannot influence when compaction runs, hint from the write side about what to prioritise, or make it work harder after a large backfill. The published observability is a status field and a timestamp per job. If your workload has hot partitions needing attention now and cold partitions that never need touching, managed compaction cannot express that distinction and you pay to rewrite both.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Disabling is a per-table change, and it does not require zeroing the settings.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>aws s3tables put-table-maintenance-configuration \n  --table-bucket-arn arn:aws:s3tables:REGION:ACCOUNT_ID:bucket\/BUCKET_NAME \n  --namespace my_namespace \n  --name my_table \n  --type icebergCompaction \n  --value '{\"status\":\"disabled\",\"settings\":{\"icebergCompaction\":{\"targetFileSizeMB\":256}}}'<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">From there you run maintenance yourself. Trino exposes an <code>optimize<\/code> procedure through <code>ALTER TABLE ... EXECUTE<\/code>, and Spark has the standard Iceberg <code>rewrite_data_files<\/code> procedure. Both let you scope work to specific partitions on your own schedule, which is the entire point of taking it back.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Do this with your eyes open. You now own the scheduling, the compute, the failure alerting, and the snapshot expiry. Teams that disable managed compaction and never build the replacement end up worse off than if they had left it alone. If cost rather than control is driving this, fix the write pattern first and re-measure. That is usually cheaper than the engineering you are about to sign up for.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Troubleshooting S3 Tables compaction and maintenance<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Work through these in order. The first step resolves more cases than the rest combined.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Check all three job statuses, not one.<\/strong> Run <code>get-table-maintenance-job-status<\/code> and read every key in the response. A green compaction status next to a failed snapshot management status is the classic growing-bucket signature.<\/li>\n\n<li><strong>Read the failure message literally.<\/strong> It names the exact property or condition at fault. Tags or references means someone created a branch or tag. Incompatible table properties means <code>history.expire.*<\/code> is set.<\/li>\n\n<li><strong>Status is <code>Disabled<\/code> or <code>Not_Yet_Run<\/code> unexpectedly.<\/strong> Someone replaced the configuration. Confirm with <code>get-table-maintenance-configuration<\/code>, and remember <code>put-table-maintenance-configuration<\/code> replaces rather than merges.<\/li>\n\n<li><strong>Compaction runs but file count stays high.<\/strong> Check codec and data types against the unsupported list. Files using <code>brotli<\/code> or <code>lz4<\/code>, or containing the <code>Fixed<\/code> type, are outside what compaction touches.<\/li>\n\n<li><strong>Insufficient-access failures on an encrypted table.<\/strong> Check the KMS key policy grants the <code>maintenance.s3tables.amazonaws.com<\/code> service principal, and that the key is enabled and still exists.<\/li>\n\n<li><strong>Storage still growing with everything green.<\/strong> Do the arithmetic on your retention windows. Snapshot age plus noncurrent days is the floor on how long superseded data bills, and defaults put that at roughly two weeks.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">For anything historical, S3 Tables maintenance operations appear in CloudTrail management events, which is where you go to answer &#8220;when did this start&#8221; rather than &#8220;what is broken now&#8221;.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Common mistakes<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Treating the three maintenance jobs as one thing.<\/strong> They fail independently and for different reasons. &#8220;Maintenance is enabled&#8221; is not a status.<\/li>\n\n<li><strong>Creating branches or tags on a managed table.<\/strong> A normal Iceberg workflow that switches off snapshot expiry for the whole table.<\/li>\n\n<li><strong>Tuning <code>targetFileSizeMB<\/code> to reduce cost.<\/strong> It changes where the data lands, not how much has to be moved. Fix the commit pattern instead.<\/li>\n\n<li><strong>Setting <code>sort<\/code> or <code>z-order<\/code> without a sort order defined.<\/strong> The strategy needs <code>sort_order<\/code> in table properties and the <code>s3tables:GetTableData<\/code> permission, and it costs more per byte than <code>binpack<\/code>.<\/li>\n\n<li><strong>Running write pipelines with no commit retry.<\/strong> Compaction is a concurrent writer. Conflicts are expected behaviour, not an anomaly.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Best practices for running S3 Tables compaction in production<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Poll job status on a schedule and alert on it.<\/strong> A small scheduled function calling <code>get-table-maintenance-job-status<\/code> across your tables, emitting a metric per job type, closes the largest observability gap in the service. Alert on <code>Failed<\/code> and on a stale <code>lastRunTimestamp<\/code>, on whatever dashboard you already watch, Grafana or CloudWatch.<\/li>\n\n<li><strong>Make maintenance config part of your IaC.<\/strong> These are settings someone can change with one CLI call and no review. Codify them so drift is visible.<\/li>\n\n<li><strong>Set retention windows from a real requirement.<\/strong> Ask how far back anyone needs to time travel, then set <code>maxSnapshotAgeHours<\/code> and <code>nonCurrentDays<\/code> to that plus a margin, rather than accepting defaults by omission.<\/li>\n\n<li><strong>Batch on the write side before tuning anything else.<\/strong> Fewer, larger commits improve object count, monitoring cost, compaction cost, and query performance simultaneously.<\/li>\n\n<li><strong>Choose the strategy from the query pattern.<\/strong> One dominant filter column suggests <code>sort<\/code>. Several equally weighted predicates suggest <code>z-order<\/code>. Neither suggests leaving it on <code>auto<\/code>.<\/li>\n\n<li><strong>Track the compaction billing dimension separately.<\/strong> It moves independently of storage and it is the earliest signal that a writer&#8217;s behaviour has changed.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">How do I know if S3 Tables compaction is actually running?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Call <code>get-table-maintenance-job-status<\/code> and read the <code>icebergCompaction<\/code> block. It returns <code>Successful<\/code>, <code>Failed<\/code>, <code>Disabled<\/code>, or <code>Not_Yet_Run<\/code>, plus a last-run timestamp and a failure message. No dashboard surfaces this by default, so to know without asking you have to poll it and emit your own metric.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Why is my S3 Tables storage growing when compaction says it succeeded?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Two likely causes. Either snapshot management is failing separately, which stops old snapshots expiring and pins their data files indefinitely, or your retention windows are longer than you assumed. On defaults, superseded data sits for the snapshot age window plus the noncurrent window, around two weeks. Check the other two job statuses first.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can I run my own compaction alongside the managed one?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">You can, but it is a poor idea. Both are writers competing under Iceberg&#8217;s optimistic concurrency, so you generate commit conflicts and pay twice to rewrite the same data. Pick one. If you need engine-side control, disable managed compaction for that table first.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Does disabling compaction stop all S3 Tables charges?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. It stops the compaction object and byte-processing charges for that table. Storage, requests, and the per-object monitoring fee continue, and that monitoring fee scales with object count, which is exactly what stops improving once compaction is off. Disabling it on a small-file-heavy table can raise monitoring cost over time.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Is sort or z-order compaction worth the extra cost?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">It depends on whether queries filter on the sorted columns. Sorted layout lets engines skip files using Parquet column statistics, so the saving shows up in query cost and latency, not storage. Dominant queries filtering on one column favour <code>sort<\/code>. Several dimensions in varying combinations favour <code>z-order<\/code>. Queries that mostly scan everything justify neither.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Can I recover data deleted by S3 Tables maintenance?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Deletion of noncurrent objects is permanent. AWS documents contacting support to view or recover objects marked noncurrent, which is not a path to depend on. Treat retention window changes as a one-way door, set from a stated requirement rather than trimmed until the bill looks better.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The one thing worth remembering<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">S3 Tables compaction is a good trade for most teams. The service does real work you would otherwise be scheduling and babysitting, and it does it well enough that you can stop thinking about small files as a daily concern.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What it does not do is tell you when it stops. No alarm fires when a branch someone created for an afternoon experiment switches off snapshot expiry, and no banner appears when a KMS policy change kills maintenance across a bucket. The price of the convenience is that you build the one thing the service did not: a scheduled check on <code>get-table-maintenance-job-status<\/code> that tells you when the automation stopped automating. That is an afternoon of work. Skipping it is how a flat workload turns into a bill that grows every month for reasons nobody can explain.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Need help with Iceberg tables on AWS?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">I work with teams running Apache Iceberg on S3, and most problems are the ones above rather than anything exotic. Things I can help with:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Auditing every table in a bucket for failed or disabled maintenance jobs, and finding which have stopped expiring snapshots<\/li>\n\n<li>Building the scheduled job-status check and alerting that S3 Tables does not ship with, wired into CloudWatch or Grafana<\/li>\n\n<li>Working out where a bucket&#8217;s storage growth actually comes from, and setting retention windows against a real requirement<\/li>\n\n<li>Reworking streaming and CDC write patterns so commits produce fewer, larger files before compaction charges scale with them<\/li>\n\n<li>Deciding table by table whether managed or engine-side maintenance is right, and implementing whichever you pick<\/li>\n\n<li>Migrating self-managed Iceberg tables into table buckets without carrying over the properties that break managed maintenance<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">If something is not adding up, send me the output of <code>get-table-maintenance-job-status<\/code> for the table and a rough description of your write pattern. That is usually enough to say what is going on.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.upwork.com\/freelancers\/~01f15a912ad84a6620\" target=\"_blank\" rel=\"noreferrer noopener\">Work with me on Upwork<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>Managed Iceberg compaction in S3 Tables removes real operational work, but it will not tell you when it stops. A practical look at the maintenance jobs that fail silently, the retention cascade that keeps superseded data on your bill for two weeks, how compaction charges actually accumulate, and the handful of knobs worth touching.<\/p>\n","protected":false},"author":1,"featured_media":1205,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25,498,674],"tags":[161,164,187,160,237,584,959,433,581,582,579,941,583,958,1060],"class_list":["post-773","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-computing","category-data-engineering","category-finops","tag-amazon-s3","tag-apache-iceberg","tag-cost-optimization","tag-data-lake","tag-lakehouse","tag-optimistic-concurrency","tag-orphan-files","tag-parquet","tag-query-performance","tag-s3-tables","tag-small-files","tag-snapshots","tag-table-compaction","tag-table-maintenance","tag-write-amplification","entry","has-media"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.7 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>S3 Tables Compaction: The Cost of Convenience<\/title>\n<meta name=\"description\" content=\"S3 Tables compaction runs itself until it quietly stops. The failure modes, the storage tail that keeps billing, and the cost levers you control.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"S3 Tables Compaction: The Cost of Convenience\" \/>\n<meta property=\"og:description\" content=\"S3 Tables compaction runs itself until it quietly stops. The failure modes, the storage tail that keeps billing, and the cost levers you control.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/\" \/>\n<meta property=\"og:site_name\" content=\"John Nessime\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/J.Nessime\" \/>\n<meta property=\"article:author\" content=\"https:\/\/www.facebook.com\/J.Nessime\" \/>\n<meta property=\"article:published_time\" content=\"2026-10-22T06:02:31+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/aws-s3-tables-compaction-production-hidden-costs.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1536\" \/>\n\t<meta property=\"og:image:height\" content=\"1024\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"John Nessime\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"John Nessime\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"14 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/\"},\"author\":{\"name\":\"John Nessime\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"headline\":\"S3 Tables Compaction in Production: The Cost of Convenience\",\"datePublished\":\"2026-10-22T06:02:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/\"},\"wordCount\":2965,\"publisher\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"image\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/aws-s3-tables-compaction-production-hidden-costs.png\",\"keywords\":[\"Amazon S3\",\"Apache Iceberg\",\"Cost Optimization\",\"Data Lake\",\"Lakehouse\",\"Optimistic Concurrency\",\"Orphan Files\",\"Parquet\",\"Query Performance\",\"S3 Tables\",\"Small Files\",\"Snapshots\",\"Table Compaction\",\"Table Maintenance\",\"Write Amplification\"],\"articleSection\":[\"Cloud Computing\",\"Data Engineering\",\"FinOps\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/\",\"name\":\"S3 Tables Compaction: The Cost of Convenience\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/aws-s3-tables-compaction-production-hidden-costs.png\",\"datePublished\":\"2026-10-22T06:02:31+00:00\",\"description\":\"S3 Tables compaction runs itself until it quietly stops. The failure modes, the storage tail that keeps billing, and the cost levers you control.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/#primaryimage\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/aws-s3-tables-compaction-production-hidden-costs.png\",\"contentUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/aws-s3-tables-compaction-production-hidden-costs.png\",\"width\":1536,\"height\":1024,\"caption\":\"AWS S3 Tables compaction illustration showing successful compaction, failed snapshot management, retained Apache Iceberg snapshots, and rising storage costs.\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/s3-tables-compaction-production\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"S3 Tables Compaction in Production: The Cost of Convenience\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/\",\"name\":\"John Nessime\",\"description\":\"Cloud, DevOps, Data &amp; AI \u2014 Built, Tested, Explained\",\"publisher\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\",\"name\":\"John Nessime\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"contentUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"width\":512,\"height\":512,\"caption\":\"John Nessime\"},\"logo\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\"},\"description\":\"AWS Certified Solutions Architect helping businesses build reliable cloud, data, reporting, and automation solutions. I help startups, agencies, and growing businesses replace manual processes and disconnected data with practical AWS architectures, clean data pipelines, useful dashboards, and maintainable automation.\",\"sameAs\":[\"https:\\\/\\\/john-nessime.com\\\/blog\",\"https:\\\/\\\/www.facebook.com\\\/J.Nessime\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/john-m-nessime\"],\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/author\\\/johnnessime\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"S3 Tables Compaction: The Cost of Convenience","description":"S3 Tables compaction runs itself until it quietly stops. The failure modes, the storage tail that keeps billing, and the cost levers you control.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/","og_locale":"en_US","og_type":"article","og_title":"S3 Tables Compaction: The Cost of Convenience","og_description":"S3 Tables compaction runs itself until it quietly stops. The failure modes, the storage tail that keeps billing, and the cost levers you control.","og_url":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/","og_site_name":"John Nessime","article_publisher":"https:\/\/www.facebook.com\/J.Nessime","article_author":"https:\/\/www.facebook.com\/J.Nessime","article_published_time":"2026-10-22T06:02:31+00:00","og_image":[{"width":1536,"height":1024,"url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/aws-s3-tables-compaction-production-hidden-costs.png","type":"image\/png"}],"author":"John Nessime","twitter_card":"summary_large_image","twitter_misc":{"Written by":"John Nessime","Est. reading time":"14 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/#article","isPartOf":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/"},"author":{"name":"John Nessime","@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"headline":"S3 Tables Compaction in Production: The Cost of Convenience","datePublished":"2026-10-22T06:02:31+00:00","mainEntityOfPage":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/"},"wordCount":2965,"publisher":{"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"image":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/#primaryimage"},"thumbnailUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/aws-s3-tables-compaction-production-hidden-costs.png","keywords":["Amazon S3","Apache Iceberg","Cost Optimization","Data Lake","Lakehouse","Optimistic Concurrency","Orphan Files","Parquet","Query Performance","S3 Tables","Small Files","Snapshots","Table Compaction","Table Maintenance","Write Amplification"],"articleSection":["Cloud Computing","Data Engineering","FinOps"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/","url":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/","name":"S3 Tables Compaction: The Cost of Convenience","isPartOf":{"@id":"https:\/\/john-nessime.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/#primaryimage"},"image":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/#primaryimage"},"thumbnailUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/aws-s3-tables-compaction-production-hidden-costs.png","datePublished":"2026-10-22T06:02:31+00:00","description":"S3 Tables compaction runs itself until it quietly stops. The failure modes, the storage tail that keeps billing, and the cost levers you control.","breadcrumb":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/#primaryimage","url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/aws-s3-tables-compaction-production-hidden-costs.png","contentUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/aws-s3-tables-compaction-production-hidden-costs.png","width":1536,"height":1024,"caption":"AWS S3 Tables compaction illustration showing successful compaction, failed snapshot management, retained Apache Iceberg snapshots, and rising storage costs."},{"@type":"BreadcrumbList","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/s3-tables-compaction-production\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/john-nessime.com\/blog\/"},{"@type":"ListItem","position":2,"name":"S3 Tables Compaction in Production: The Cost of Convenience"}]},{"@type":"WebSite","@id":"https:\/\/john-nessime.com\/blog\/#website","url":"https:\/\/john-nessime.com\/blog\/","name":"John Nessime","description":"Cloud, DevOps, Data &amp; AI \u2014 Built, Tested, Explained","publisher":{"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/john-nessime.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105","name":"John Nessime","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","contentUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","width":512,"height":512,"caption":"John Nessime"},"logo":{"@id":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png"},"description":"AWS Certified Solutions Architect helping businesses build reliable cloud, data, reporting, and automation solutions. I help startups, agencies, and growing businesses replace manual processes and disconnected data with practical AWS architectures, clean data pipelines, useful dashboards, and maintainable automation.","sameAs":["https:\/\/john-nessime.com\/blog","https:\/\/www.facebook.com\/J.Nessime","https:\/\/www.linkedin.com\/in\/john-m-nessime"],"url":"https:\/\/john-nessime.com\/blog\/author\/johnnessime\/"}]}},"_links":{"self":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/773","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/comments?post=773"}],"version-history":[{"count":1,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/773\/revisions"}],"predecessor-version":[{"id":775,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/773\/revisions\/775"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/media\/1205"}],"wp:attachment":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/media?parent=773"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/categories?post=773"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/tags?post=773"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}