The question usually comes from finance, not engineering. Someone points at the bucket holding the analytics lake and asks why it is twice the size it was two quarters ago. Nobody added a pipeline. The tables are the same tables. One of them had a large chunk of history deleted on purpose, and the bucket still grew.<\/p>\n\n\n\n
That gap between what a table logically contains and what the object store physically holds is what Iceberg table maintenance exists to close. Iceberg never overwrites a file in place. Deletes and updates write new files and a new pointer, and the old files sit where they are until something explicitly removes them. If nothing does, the bill only moves one way.<\/p>\n\n\n\n
This post covers where those bytes hide, why expiring snapshots often frees nothing, how orphan files accumulate without showing up in any metadata you would think to check, and the order to run cleanup so you do not corrupt a table while shrinking it.<\/p>\n\n\n\n
Every write produces a snapshot: an immutable view of the table at a point in time, pointing at a manifest list, which points at manifests, which point at data files. Nothing in that chain is mutated. A new commit builds a new chain that reuses most of the old files.<\/p>\n\n\n\n
That design is what gives you time travel, rollback and safe concurrent writes. It is also why storage grows silently. A data file only becomes a deletion candidate once no<\/em> retained snapshot references it. Delete a year of rows and you free nothing, because the snapshot from before the delete still points at every one of those files.<\/p>\n\n\n\n So maintenance is not an optimization. It is the mechanism by which deletes become real.<\/p>\n\n\n\n When a lakehouse bucket balloons, the bytes are in one of four categories. Each needs a different tool, and confusing them is how teams end up running the same job for weeks with no effect.<\/p>\n\n\n\n This is the one everyone knows about and usually the largest share. Snapshots accumulate until something expires them, and nothing expires them by default.<\/p>\n\n\n\n Before changing anything, look at the table’s own metadata. Iceberg exposes it as queryable tables:<\/p>\n\n\n\n Compare that second number against what your object store reports for the table prefix. The difference is your maintenance debt. On a neglected table the referenced set is often a minority of the physical bytes.<\/p>\n\n\n\n Expiring snapshots is a Spark stored procedure:<\/p>\n\n\n\n Three arguments matter. The trap is that Iceberg treats these conditions as guidance rather than a strict contract, leaning toward retention when they conflict. An aggressive Branches and tags are separate: they are not subject to snapshot age at all. A tag created for a regulatory export a year ago pins its entire snapshot chain, and no amount of expiring will touch it. Reference age is governed by This is the invisible one, and the reason many teams conclude that maintenance “does not work.”<\/p>\n\n\n\n When a Spark task writes a data file and the job then fails before the commit lands, that file is already on disk. It is in no manifest, no snapshot, no metadata file. Expiry will never find it, because expiry compares the file sets of expired snapshots against retained ones, and a file that was never in a snapshot is in neither set.<\/p>\n\n\n\n Streaming writers, speculative execution, retried stages and killed compaction jobs all produce these. The only way to find them is to list the storage prefix and diff it against the metadata:<\/p>\n\n\n\n Read that output before you run it live. Recent timestamps or unfamiliar paths mean stop and investigate.<\/p>\n\n\n\n The safety mechanism is The second hazard is less obvious. Iceberg compares file paths as strings. If your metadata records Every commit writes a new table metadata JSON file. On a batch table that is a handful a day. On a streaming table committing every minute it is a lot of small objects, and those cost you in request charges and planning time as much as in bytes.<\/p>\n\n\n\n The first property is off by default. The second defaults to 100 and controls how many previous metadata files are tracked<\/em> in the metadata log. The catch is that cleanup only deletes tracked files. If you have been running with deletion disabled, files that fell out of the log are already untracked, and turning the property on later will not reach back for them. They are orphans now, and only orphan file removal clears them.<\/p>\n\n\n\n You can see the tracked set with Manifests rarely dominate the bill, but a table with thousands of tiny ones plans queries slowly. Most Iceberg guides skip this category, and it often explains a bill nobody can account for. Iceberg has no visibility into any of it.<\/p>\n\n\n\n A lifecycle configuration covers all three. On S3 the shape looks like this:<\/p>\n\n\n\n Now the warning that matters most here. Never put an age-based expiration or storage class transition rule on the prefix holding live Iceberg data files.<\/strong> A rule deleting objects older than ninety days will happily delete data files a current snapshot still references, and the table breaks with missing-file errors on the next read. Transitioning them to an archive class is just as bad, because reads then fail or stall on retrieval.<\/p>\n\n\n\n The rules above act only on things the table does not reference: abandoned upload parts, superseded versions, tombstones. Iceberg decides what live data to delete; lifecycle rules clean up after that decision.<\/p>\n\n\n\n This applies equally on S3-compatible storage. If you run a lakehouse on Cloudflare R2, Backblaze B2, Wasabi or Contabo Object Storage to dodge egress charges, check which lifecycle actions the provider actually implements. Support for aborting incomplete multipart uploads varies, and a provider that silently ignores the rule leaves you paying for parts you cannot see.<\/p>\n\n\n\n Order matters, and the reason is not stylistic. These procedures read each other’s inputs.<\/p>\n\n\n\n On cadence: expiry is cheap and should run often, daily or better on high-commit tables. Orphan removal is expensive because it lists the entire table prefix, and the official guidance is to run it periodically but not frequently. Weekly on a busy table, monthly on a quiet one. Where supported, the Procedures are what you run. Properties are what you set once so the procedures behave sensibly without a config file per table.<\/p>\n\n\n\n The snapshot age default is five days and the minimum to keep defaults to one. Setting both means a generic maintenance job can call The other property worth knowing is As often as your commit rate justifies. A streaming table committing every minute benefits from daily or hourly expiry; a nightly batch table is fine with weekly. Set the window from how far back you would realistically need to roll back, then keep a few extra snapshots as a floor.<\/p>\n\n\n\n It deletes data files that no retained snapshot references. Current table contents are never affected. What you lose is the ability to time travel to or roll back to the expired snapshots, so agree the window with anyone who queries the table as of a date.<\/p>\n\n\n\n The commit is atomic; the writes preceding it are not part of that atomicity. Data files are written first, then the commit makes them visible. If the job dies between those steps, the files exist and the commit never happened. Atomicity guarantees readers never see a partial table, not that failed writes clean up after themselves.<\/p>\n\n\n\n No, and trying is how tables get broken. Lifecycle rules act on object age. Whether Iceberg still needs a file has nothing to do with age; a data file written two years ago can be referenced by the current snapshot. Lifecycle rules are right for upload parts, noncurrent versions and delete markers, and wrong for anything the table might still point at.<\/p>\n\n\n\n Spark has the most complete set of procedures, but it is not the only route. Flink has its own maintenance API, Trino exposes expiry and orphan removal as table procedures, and several managed catalogs run scheduled maintenance for you. The mechanics are identical regardless of engine, because the work is defined by the table format rather than the compute. Only the syntax and the exposed arguments change.<\/p>\n\n\n\n Yes, once you have verified that the retention window comfortably exceeds your longest write and that the paths in your metadata match those your storage listing returns. Automate it after watching a few dry runs, not before. Erroring on a prefix mismatch is a stop signal, not something to configure away.<\/p>\n\n\n\n Iceberg table maintenance is not housekeeping you get to postpone. It is the step where logical deletes become physical ones, and until it runs, storage only grows. The bill is the visible symptom; the same neglect also makes query planning slower and recovery windows fuzzier.<\/p>\n\n\n\n Set retention as a table property so the policy lives with the data. Expire snapshots on a schedule matching your commit rate. Remove orphan files less often, always after expiry, always with a window longer than your slowest write. Then add the storage lifecycle rules that clean up what Iceberg cannot see. Any one alone leaves money on the table.<\/p>\n\n\n\n Most of the work here is not writing the procedures. It is working out which of the four categories your bytes are in, and building a maintenance path that will not break a table at three in the morning. That is the kind of thing I do:<\/p>\n\n\n\n If you want a second opinion, send me the output of your snapshots metadata table, your current table properties, or the counters from a run that did not do what you expected. That is usually enough to tell you where the bytes are.<\/p>\n\n\n\nThe four places your storage is actually sitting<\/h2>\n\n\n\n
1. Data files held alive by snapshots you never expired<\/h3>\n\n\n\n
-- How many snapshots exist, and how far back do they go?\nSELECT committed_at, snapshot_id, operation\nFROM db.orders.snapshots\nORDER BY committed_at;\n\n-- How much data does the CURRENT snapshot reference?\nSELECT count(*) AS file_count,\n sum(file_size_in_bytes) AS bytes\nFROM db.orders.files;<\/code><\/pre>\n\n\n\nCALL my_catalog.system.expire_snapshots(\n table => 'db.orders',\n older_than => TIMESTAMP '2000-01-01 00:00:00',\n retain_last => 10,\n stream_results => true\n);<\/code><\/pre>\n\n\n\nolder_than<\/code> sets the age cutoff. retain_last<\/code> is the floor: it keeps that many ancestor snapshots regardless of age, which is your rollback safety net. stream_results<\/code> streams the delete list back to the driver instead of collecting it at once, which on a table with long history is the difference between the job finishing and the driver running out of memory.<\/p>\n\n\n\nolder_than<\/code> plus a generous retain_last<\/code> gives you the generous answer. Safe, but it surprises people who expected the timestamp to win.<\/p>\n\n\n\nhistory.expire.max-ref-age-ms<\/code>.<\/p>\n\n\n\n2. Orphan files that no metadata points at<\/h3>\n\n\n\n
-- Always dry run first. This lists candidates without deleting.\nCALL my_catalog.system.remove_orphan_files(\n table => 'db.orders',\n dry_run => true\n);<\/code><\/pre>\n\n\n\nolder_than<\/code>, which defaults to three days. That default is not arbitrary caution. Orphan removal cannot distinguish a file abandoned by a dead job from one being written right now by a live one. Set the interval shorter than your longest-running write and the procedure can delete a file a job is about to commit, corrupting the table. Shorten it only when you know your write durations, and never below your longest backfill.<\/p>\n\n\n\ns3a:\/\/bucket\/...<\/code> and your listing returns s3:\/\/bucket\/...<\/code>, or you migrated between HDFS namenodes and the authority changed, every live file looks like an orphan. Running that deletes the table. The procedure defends against it by erroring on a prefix mismatch by default, via prefix_mismatch_mode<\/code> with ERROR<\/code>, IGNORE<\/code> and DELETE<\/code> as options. Resolve a mismatch with equal_schemes<\/code> or equal_authorities<\/code>. Do not reach for DELETE<\/code> to silence it.<\/p>\n\n\n\n3. Metadata JSON files and manifests<\/h3>\n\n\n\n
ALTER TABLE db.orders SET TBLPROPERTIES (\n 'write.metadata.delete-after-commit.enabled' = 'true',\n 'write.metadata.previous-versions-max' = '20'\n);<\/code><\/pre>\n\n\n\nSELECT timestamp, file FROM db.orders.metadata_log_entries<\/code>.<\/p>\n\n\n\nrewrite_manifests<\/code> consolidates them. Treat it as a query performance job and run it after the deletion work, so it is not reorganizing metadata about to be thrown away.<\/p>\n\n\n\n4. The object store’s own hidden inventory<\/h3>\n\n\n\n
\n
{\n \"Rules\": [\n {\n \"ID\": \"abort-stale-multipart\",\n \"Status\": \"Enabled\",\n \"Filter\": { \"Prefix\": \"warehouse\/\" },\n \"AbortIncompleteMultipartUpload\": { \"DaysAfterInitiation\": 7 }\n },\n {\n \"ID\": \"expire-noncurrent-versions\",\n \"Status\": \"Enabled\",\n \"Filter\": { \"Prefix\": \"warehouse\/\" },\n \"NoncurrentVersionExpiration\": { \"NoncurrentDays\": 14 },\n \"Expiration\": { \"ExpiredObjectDeleteMarker\": true }\n }\n ]\n}<\/code><\/pre>\n\n\n\nRunning Iceberg table maintenance in a safe order<\/h2>\n\n\n\n
\n
rewrite_data_files<\/code> writes new files and leaves the old ones referenced by older snapshots. Doing it after expiry means the space you just freed is immediately replaced.<\/li>\n\n\n\nprefix_listing<\/code> option makes that scan substantially cheaper on object stores, where recursive directory listing is the slow part.<\/p>\n\n\n\nLetting table properties do the boring part<\/h2>\n\n\n\n
ALTER TABLE db.orders SET TBLPROPERTIES (\n 'history.expire.max-snapshot-age-ms' = '604800000', -- 7 days\n 'history.expire.min-snapshots-to-keep' = '10'\n);<\/code><\/pre>\n\n\n\nexpire_snapshots<\/code> with no tuning and each table gets its own policy. A regulated reporting table keeps ninety days; a staging table keeps an hour. That beats maintaining per-table retention in your scheduler.<\/p>\n\n\n\ngc.enabled<\/code>. Setting it false blocks physical deletion entirely. It exists for tables whose files are shared with something else, and it is correct for those. It is also a common accidental cause of “maintenance runs successfully and frees nothing.” Check it before debugging anything else.<\/p>\n\n\n\nTroubleshooting: the job succeeded and nothing shrank<\/h2>\n\n\n\n
\n
expire_snapshots<\/code> returns counts of deleted data files, delete files, manifests and manifest lists. All zero means nothing qualified. Check whether retain_last<\/code> or the table’s minimum-to-keep property is holding everything.<\/li>\n\n\n\ngc.enabled<\/code> is false, or the table was created by the snapshot<\/code> procedure, which is barred from expiry precisely because it does not own its data files.<\/li>\n\n\n\nstream_results<\/code> and raise max_concurrent_deletes<\/code> so deletes spread across a thread pool instead of serializing.<\/li>\n<\/ul>\n\n\n\nCommon mistakes<\/h2>\n\n\n\n
\n
Best practices worth adopting<\/h2>\n\n\n\n
\n
Frequently asked questions<\/h2>\n\n\n\n
How often should I expire Iceberg snapshots?<\/h3>\n\n\n\n
Does expiring snapshots delete my data?<\/h3>\n\n\n\n
Why do orphan files exist if Iceberg commits are atomic?<\/h3>\n\n\n\n
Can I use an S3 lifecycle rule instead of Iceberg maintenance?<\/h3>\n\n\n\n
Do I need Spark to run Iceberg table maintenance?<\/h3>\n\n\n\n
Is orphan file removal safe to automate?<\/h3>\n\n\n\n
\n\n\n\nThe one thing to take away<\/h2>\n\n\n\n
\n\n\n\nNeed help getting your lakehouse storage under control?<\/h2>\n\n\n\n
\n