{"id":405,"date":"2026-09-06T09:00:00","date_gmt":"2026-09-06T06:00:00","guid":{"rendered":"https:\/\/john-nessime.com\/blog\/?p=405"},"modified":"2026-09-14T16:12:59","modified_gmt":"2026-09-14T13:12:59","slug":"amazon-macie-pii-detection","status":"publish","type":"post","link":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/","title":{"rendered":"Amazon Macie PII Detection: The Buckets It Never Opened"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">The email came from legal, not from security. A customer had exercised a data access request, someone pulled the export by hand, and it contained email addresses and phone numbers sitting in a bucket Macie had labeled <em>Not sensitive<\/em>. Nobody had done anything wrong. The console was telling the truth as it understood it. It just didn&#8217;t understand very much about that bucket.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">That is the failure mode worth internalizing before anything else: Amazon Macie PII detection reports on what it managed to read, not on what is there. Objects it never opened produce no findings, and no findings looks identical to a clean result. The gap between &#8220;we scanned this&#8221; and &#8220;we found nothing&#8221; is where most Macie deployments quietly fail their first audit.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This post covers where that gap comes from and how to close it: the eligibility chain every object goes through, the four distinct reasons an object gets skipped, how to tune identifiers so the findings are worth reading, what actually drives the bill, and how to route results somewhere a human will act on them.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The eligibility chain behind Amazon Macie PII detection<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Every object passes the same gates before a byte gets inspected. The order matters, because each gate fails differently and needs a different fix.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li>Is it in a general purpose S3 bucket? Directory buckets are out of scope entirely.<\/li>\n\n\n\n<li>Is the storage class supported? Standard, Standard-IA, One Zone-IA, Intelligent-Tiering, Glacier Instant Retrieval and Reduced Redundancy are in. Glacier Deep Archive and S3 Express One Zone are not.<\/li>\n\n\n\n<li>Does the object key carry a recognized file extension? This is a string check on the name, not a content sniff.<\/li>\n\n\n\n<li>Can Macie retrieve and decrypt it, given the bucket policy, object ACL and encryption key?<\/li>\n\n\n\n<li>Does the content parse, and stay inside the per-file quotas?<\/li>\n\n\n\n<li>Do the active data identifiers match anything in it?<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">Only the last gate produces a finding. Everything before it produces silence. Macie does record why each object was skipped, but that lives in coverage data and object samples, not on the findings page most people look at.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One detail explains a lot of confusion: when you first enable automated sensitive data discovery, every bucket gets a sensitivity score of 50 and the label <em>Not yet analyzed<\/em>. A bucket whose permissions block Macie stays there permanently. It never turns red. It sits mid-list, looking unremarkable next to buckets that genuinely were analyzed.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Gap one: objects Macie never opened<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Macie calls these <em>unclassifiable<\/em>. They fail the storage class or extension check, so no retrieval is attempted. This is the largest source of false confidence I&#8217;ve seen, and the cheapest to fix.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Missing file extensions<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Classifiability comes from the file name extension. A file full of customer records named <code>part-00003-a4f9<\/code> is invisible. The same bytes named <code>part-00003-a4f9.json<\/code> get inspected and produce findings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This bites hardest on streaming ingestion. Amazon Data Firehose writes to S3 without appending an extension unless compression, format conversion, or the explicit file extension setting adds one. An uncompressed JSON stream lands as a tree of extensionless objects. Athena and Glue read them fine, because they infer format from the table definition. Macie skips every one.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Fix it upstream: set the S3 file extension on the delivery configuration, or enable GZIP compression or Parquet conversion, both of which append a recognized extension. If you can&#8217;t change the producer, the fallback is a copy-and-rename step into a scan prefix, which costs you storage plus a pipeline to maintain. Fixing the producer is almost always cheaper.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Unsupported storage classes<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">A lifecycle rule transitioning old exports to Glacier Deep Archive quietly removes them from scope. That may be the right storage decision, but it needs to be a conscious one, because archived customer data is still customer data when a regulator asks.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Macie exposes these counts per bucket, split by cause. This is the first query I run against a new account:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Buckets where Macie can't classify objects, broken down by cause\naws macie2 describe-buckets \n  --query 'buckets[?unclassifiableObjectCount.total &gt; `0`].{\n      bucket: bucketName,\n      classifiable: classifiableObjectCount,\n      skipped_total: unclassifiableObjectCount.total,\n      skipped_extension: unclassifiableObjectCount.fileType,\n      skipped_storage_class: unclassifiableObjectCount.storageClass\n    }' \n  --output table<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Read the ratio, not the raw number. Forty thousand classifiable objects and twelve skipped is fine. Three classifiable and four hundred thousand skipped is a reporting artifact pretending to be a scan result, and that bucket&#8217;s sensitivity score means nothing.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Gap two: objects it opened and couldn&#8217;t read<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">These are classification errors rather than eligibility failures. Macie selected the object, tried to fetch it, and was refused. Three causes, and they need three different people to fix them.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Customer-provided keys (SSE-C).<\/strong> Macie cannot supply the key material, so it cannot retrieve the object. No permissions fix exists. Re-encrypt with S3 managed or KMS keys if you want coverage.<\/li>\n\n\n\n<li><strong>KMS key policy.<\/strong> For customer managed keys, the key policy must allow the Macie service-linked role, <code>AWSServiceRoleForAmazonMacie<\/code>, to decrypt. Cross-account buckets need the key owner to grant it, not the bucket owner. AWS publishes a permission analyzer script in the <code>aws-samples\/amazon-macie-scripts<\/code> repository that enumerates every key Macie needs and generates the CLI commands to fix them.<\/li>\n\n\n\n<li><strong>Restrictive bucket policies.<\/strong> An explicit <code>Deny<\/code> conditioned on source IP or VPC endpoint blocks Macie along with everything else. The working pattern excludes the service-linked role ARN from the deny using the <code>aws:PrincipalArn<\/code> condition key.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">To see which objects Macie actually touched in a bucket, pull the object samples. It&#8217;s the closest thing to a scan log you get:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Objects automated discovery selected, with per-object status\naws macie2 list-resource-profile-artifacts \n  --resource-arn arn:aws:s3:::your-bucket-name<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Anything with a status of <code>SKIPPED<\/code> is a lead. Take the key, check its encryption settings in S3, and you usually have your answer inside a minute.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Gap three: it read the file and nothing matched<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Macie ships managed data identifiers covering common PII, financial data and credentials across many countries. Automated discovery uses a recommended subset by default rather than all of them. Sensible, but it is a default, and it will miss things that matter to you.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Add the managed identifiers for your actual jurisdictions.<\/strong> If you hold records for customers in a country whose national ID identifier isn&#8217;t in the recommended set, you&#8217;re scanning for the wrong things with perfect efficiency. Pull the current list with <code>aws macie2 list-managed-data-identifiers<\/code> and compare it against where your customers live.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Write custom identifiers for what only you know is sensitive.<\/strong> Internal account numbers, case references, employee IDs. A custom identifier is a regex plus optional keywords and a proximity rule, and the keywords are what save you. A bare <code>[0-9]{8}<\/code> matches timestamps, order totals and row counts, and you drown. The same regex with keywords and a match distance only fires when the number sits near a word that gives it meaning.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Test the pattern before it goes near a job. This runs the criteria against sample text and returns match counts without creating anything:<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># Dry-run a custom identifier before creating it\naws macie2 test-custom-data-identifier \n  --regex 'ACC-[0-9]{4}-[0-9]{4}' \n  --keywords account customer acct \n  --maximum-match-distance 30 \n  --sample-text 'customer account ACC-4821-9930 updated; total 1234-5678'<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">Run it against real samples, including the awkward ones, then against a file you know is clean and confirm zero. A pattern you only tested on positives is a pattern you haven&#8217;t tested.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The mirror image is allow lists, which tell Macie to ignore specific text or patterns: your support inbox address, your published press contact, the seeded test records in everyone&#8217;s staging fixtures. Without them the same non-issues appear in every scan until the team stops reading findings. Allow lists accept predefined text or regex, and the per-account and per-job quotas are tight enough that you should curate rather than dump.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Gap four: quota truncation inside large files<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Subtle, because the object <em>is<\/em> analyzed. Just not completely, and a partial result reports like a full one.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Per-file size quotas differ by format. Exceed the quota for a type and Macie analyzes none of that file, not part of it.<\/li>\n\n\n\n<li>Archives have separate ceilings for nesting depth, extracted bytes and file count. If the metadata shows a breach up front, nothing is extracted. Cross a limit partway through and Macie stops, reporting only what it processed.<\/li>\n\n\n\n<li>JSON and JSON Lines have a nested depth limit. Past it, the file is not analyzed at all.<\/li>\n\n\n\n<li>Full names and mailing addresses cap out per file. After the cap Macie stops counting and stops reporting locations for that type, so the occurrence count is a floor, never a total.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Findings carry a status reason naming the limit that was hit, with values like <code>FILE_EXCEEDS_SIZE_LIMIT<\/code>, <code>ARCHIVE_NESTING_LEVEL_OVER_LIMIT<\/code> and <code>JSON_NESTING_LEVEL_OVER_LIMIT<\/code>. Filter on those before declaring a data lake of large compressed exports clean. Check current quota values in the Macie documentation rather than memorizing them, since they differ per format.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Automated discovery or a discovery job?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Automated sensitive data discovery<\/strong> evaluates your inventory daily and samples representative objects across as many buckets as it can, grouping by metadata like prefix, extension and last-modified date. It is breadth-first and it does not re-analyze unchanged objects. Its job is to tell you which buckets deserve attention. It is not designed to prove a bucket is clean, and reading it that way is the root of most Macie misunderstandings.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Sensitive data discovery jobs<\/strong> are the depth tool. You choose the buckets, scope by prefix, extension, size or object tag, choose the identifiers, and run once or on a schedule. This is what you point at the bucket automated discovery just flagged.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code># One-time job over a single prefix, recommended identifiers, 30% sampling\naws macie2 create-classification-job \n  --job-type ONE_TIME \n  --name \"exports-prefix-review\" \n  --managed-data-identifier-selector RECOMMENDED \n  --sampling-percentage 30 \n  --s3-job-definition '{\n    \"bucketDefinitions\": [\n      {\"accountId\": \"111122223333\", \"buckets\": [\"your-bucket-name\"]}\n    ],\n    \"scoping\": {\n      \"includes\": {\n        \"and\": [\n          {\"simpleScopeTerm\": {\n             \"comparator\": \"STARTS_WITH\",\n             \"key\": \"OBJECT_KEY\",\n             \"values\": [\"exports\/\"]\n          }}\n        ]\n      }\n    }\n  }'<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">One detail about <code>--sampling-percentage<\/code> that trips people up: it selects a random percentage of eligible <em>objects<\/em> and then analyzes each selected object completely. It does not read a fraction of each file. For a bucket where every object comes from one pipeline, 20 or 30 percent tells you what you need. For heterogeneous uploads, sampling is a coin flip and you want the full pass.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What actually drives the bill<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Macie bills on three dimensions, and the one that surprises people is not the one they budget for.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>Bucket evaluation.<\/strong> Per general purpose bucket monitored, prorated daily, up to the account ceiling. Predictable, driven by bucket count rather than data volume.<\/li>\n\n\n\n<li><strong>Object monitoring.<\/strong> Per object tracked while automated discovery is enabled. This is inventory bookkeeping, not content reading, and it runs whether or not anything gets scanned that day. On an account with a log archive holding hundreds of millions of tiny objects, it can dwarf the other two.<\/li>\n\n\n\n<li><strong>Data inspected.<\/strong> Per GB actually analyzed, covering both automated discovery and jobs.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">So excluding buckets from automated discovery is a genuine cost lever, not just noise reduction. Log archives, CloudTrail destinations, build artifact stores and backup targets are usually safe exclusions. Exclude for the right reason though: a backup bucket holding database dumps of your customer table is exactly what you want scanned, however boring its name sounds.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The three dimensions appear as distinct usage types in the Cost and Usage Report, so they split cleanly. A FinOps platform like Vantage or CloudZero, or just a Cost Explorer view grouped by usage type, tells you within a day whether object monitoring or inspection is driving the number. That changes the fix entirely: object monitoring is solved by scoping, inspection by sampling and scheduling.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Getting findings somewhere a human sees them<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Findings that only exist in the Macie console get read during onboarding week and never again. Two things are worth wiring up on day one.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Publish findings outward.<\/strong> Macie emits findings to EventBridge and integrates with Security Hub. EventBridge earns its keep because you can filter on severity and finding type and route only what matters, whether that&#8217;s a Slack channel, a ticket, or a Lambda that tags the bucket. If you already centralize alerts in Grafana Cloud or similar, that path keeps sensitive data findings next to everything else on call sees.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Configure the discovery results repository.<\/strong> Findings tell you what was found and where. The detailed discovery results, including analysis logs for objects where nothing was found, need an S3 bucket and a KMS key configured to persist. Nothing is retained long term until you set this up, and &#8220;we scanned it and found nothing&#8221; is exactly the record an auditor asks for.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">One operational note on the reveal feature, which retrieves sample occurrences so a human can confirm a finding is real. It needs a customer managed KMS key and it&#8217;s genuinely useful for triage. It also means someone is now looking at live customer PII, which belongs in your access model and audit trail. Same goes if an engineer downloads an object to investigate locally: that copy is unmanaged PII on a laptop and needs a secure deletion step, whether that&#8217;s a wipe tool from a vendor like O&amp;O Software or a documented and enforced process. Findings triage has a habit of creating the exposure it was meant to prevent.<\/p>\n\n\n\n<hr class=\"wp-block-separator has-alpha-channel-opacity\"\/>\n\n\n\n<h2 class=\"wp-block-heading\">Troubleshooting a bucket that reports nothing<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">When Amazon Macie PII detection reports nothing for a bucket you have doubts about, work down this in order. Each step rules out one gate, and stopping at the first hit saves you the rest.<\/p>\n\n\n\n<ol class=\"wp-block-list\">\n<li><strong>Check the sensitivity label.<\/strong> <em>Not yet analyzed<\/em> at score 50 means nothing was analyzed. Do not read that as clean.<\/li>\n\n\n\n<li><strong>Check the coverage page.<\/strong> Macie names the issue directly: access denied, invalid encryption, invalid KMS key, permission denied, or unclassifiable.<\/li>\n\n\n\n<li><strong>Compare classifiable and unclassifiable counts<\/strong> with the <code>describe-buckets<\/code> query above.<\/li>\n\n\n\n<li><strong>List a few object keys.<\/strong> Run <code>aws s3 ls<\/code> against a prefix and look for extensions. Extensionless keys explain the whole thing.<\/li>\n\n\n\n<li><strong>Check the storage class.<\/strong> A lifecycle rule may have moved everything somewhere Macie doesn&#8217;t read.<\/li>\n\n\n\n<li><strong>Check encryption.<\/strong> SSE-C is unfixable without re-encryption. A customer managed KMS key is fixable with a key policy change.<\/li>\n\n\n\n<li><strong>Check object samples<\/strong> with <code>list-resource-profile-artifacts<\/code> to see exactly what was selected and skipped.<\/li>\n\n\n\n<li><strong>Only now question the identifiers.<\/strong> Run a targeted job with a custom identifier for the data you expect and see whether it fires.<\/li>\n<\/ol>\n\n\n\n<h2 class=\"wp-block-heading\">Common mistakes<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Treating a low sensitivity score as evidence of no PII. It reflects what was found <em>and<\/em> how much was analyzed, so low coverage produces a low score.<\/li>\n\n\n\n<li>Enabling Macie in one region and assuming account-wide coverage. Settings and results are per region, every time.<\/li>\n\n\n\n<li>Running default identifiers in a business operating outside the jurisdictions the recommended set covers.<\/li>\n\n\n\n<li>Building custom identifiers from regex alone, then abandoning Macie over the false positive volume.<\/li>\n\n\n\n<li>Skipping the discovery results repository, then having no evidence trail when someone asks what was scanned and when.<\/li>\n\n\n\n<li>Excluding buckets by name pattern without checking contents. Backup and export buckets are frequently the highest-risk ones you own.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Best practices<\/h2>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Measure coverage before findings. A coverage report is the first deliverable of a Macie rollout, not an afterthought.<\/li>\n\n\n\n<li>Fix extensions at the producer rather than building a rename pipeline downstream.<\/li>\n\n\n\n<li>Run the KMS permission analyzer script once per account, then again whenever a new customer managed key appears.<\/li>\n\n\n\n<li>Use automated discovery for breadth and targeted jobs for depth, letting the first choose targets for the second.<\/li>\n\n\n\n<li>Test every custom identifier against both a positive and a negative sample before it reaches a job.<\/li>\n\n\n\n<li>Curate allow lists early. The cost of a noisy first month is a team that stops reading findings.<\/li>\n\n\n\n<li>Re-check coverage after any change to bucket policies, KMS keys, lifecycle rules or ingestion pipelines. All four silently remove data from scope.<\/li>\n<\/ul>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently asked questions<\/h2>\n\n\n\n<h3 class=\"wp-block-heading\">Can Amazon Macie scan anything other than S3?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">No. Macie analyzes objects in S3 general purpose buckets only. The documented workaround is bringing data to it: export RDS or Aurora snapshots to S3 in Parquet, or export a DynamoDB table to S3, then run a discovery job against the export. That works, but the export is now a second copy of your sensitive data with its own encryption and deletion requirements.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Does Macie scan new objects as they land?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Not per object. Automated discovery evaluates inventory on a daily cycle and prioritizes objects that are new or recently changed, so fresh data moves up the queue rather than triggering an immediate scan. If you need scanning tied to arrival, drive a discovery job from S3 event notifications through EventBridge and accept that you own that orchestration.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Why does a bucket I know has PII show a low sensitivity score?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Almost always coverage rather than detection. Check whether the objects are classifiable, whether Macie can decrypt them, and whether the label reads <em>Not yet analyzed<\/em>. If Macie genuinely read them and found nothing, the next suspect is the identifier set, and a targeted job with a custom identifier will tell you in one run.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Do I need both automated discovery and discovery jobs?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">For most teams, yes. Automated discovery gives you the map at predictable cost. Jobs give you proof for a specific bucket at a specific time, which is what compliance evidence actually requires. Running only jobs means you never discover the bucket nobody told you about.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">How do I cut false positives without missing real PII?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">In order of preference: add keywords and a proximity rule so matches need context; use allow lists for specific known-benign values like your published contact details and test fixtures; and only then remove managed identifiers that are consistently wrong for your data. Removing identifiers is the bluntest option and the easiest to regret, so leave it last.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">Does Macie work across accounts in an organization?<\/h3>\n\n\n\n<p class=\"wp-block-paragraph\">Yes, through AWS Organizations with a delegated administrator. Design around this: the administrator&#8217;s automated discovery settings apply to member accounts, members can&#8217;t change them, and members see coverage and sensitivity data for their own buckets but not the sensitive data findings themselves. That shapes who can actually remediate what.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">The one thing worth remembering<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Amazon Macie PII detection is only as good as its coverage, and coverage fails silently by design. An object with the wrong extension, the wrong storage class, or an unreachable encryption key produces no finding, and no finding renders identically to a clean result. Before you hand a Macie report to anyone who will make a decision from it, put the coverage numbers next to it: how many objects were classifiable, how many were analyzed, how many were skipped and why.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Do that once and the rest of the service becomes straightforward. Skip it and you&#8217;re maintaining an expensive dashboard reporting on whichever subset of your data happened to be readable.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Need help getting Macie to actually see your data?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">Most of the work in a Macie rollout isn&#8217;t turning it on. It&#8217;s the unglamorous part: proving what got scanned, fixing the reasons things didn&#8217;t, and making the output land somewhere a human acts on it. That&#8217;s the part I help with.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Coverage audits across your S3 estate, with per-bucket classifiable, analyzed and skipped counts and the specific cause of each gap<\/li>\n\n\n\n<li>Fixing those causes: KMS key policies, bucket policies blocking the service-linked role, lifecycle rules archiving data out of scope, pipelines writing extensionless objects<\/li>\n\n\n\n<li>Custom data identifiers and allow lists tuned against your real data and tested on positive and negative samples first<\/li>\n\n\n\n<li>Cost shaping: working out whether object monitoring or data inspection drives your bill, and scoping discovery so the number is defensible<\/li>\n\n\n\n<li>Findings pipelines through EventBridge or Security Hub into the alerting and ticketing you already run, with severity filtering that keeps signal above noise<\/li>\n\n\n\n<li>Multi-account setup under AWS Organizations, including the delegated administrator model and who can see and remediate what<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">If you want a second opinion, send me the output of the <code>describe-buckets<\/code> query above, or a screenshot of your coverage page, and I&#8217;ll tell you what&#8217;s actually being scanned.<\/p>\n\n\n\n<div class=\"wp-block-buttons is-layout-flex wp-block-buttons-is-layout-flex\">\n<div class=\"wp-block-button\"><a class=\"wp-block-button__link wp-element-button\" href=\"https:\/\/www.upwork.com\/freelancers\/~01f15a912ad84a6620\" target=\"_blank\" rel=\"noreferrer noopener\">Work with me on Upwork<\/a><\/div>\n<\/div>\n","protected":false},"excerpt":{"rendered":"<p>A Macie bucket labeled &#8220;Not sensitive&#8221; often just means Macie never read it. Extensionless objects, unsupported storage classes, unreachable KMS keys and quota truncation all produce silence that looks identical to a clean result. Here&#8217;s how to measure coverage, fix the four gaps, tune identifiers, and keep the bill honest.<\/p>\n","protected":false},"author":1,"featured_media":925,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25,504,498],"tags":[593,588,161,93,283,287,103,286,187,592,589,160,284,224,390,488,591,590],"class_list":["post-405","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-cloud-computing","category-compliance","category-data-engineering","tag-amazon-data-firehose","tag-amazon-macie","tag-amazon-s3","tag-aws","tag-aws-kms","tag-aws-organizations","tag-cloud-security","tag-compliance","tag-cost-optimization","tag-data-classification","tag-data-governance","tag-data-lake","tag-encryption","tag-eventbridge","tag-pii-redaction","tag-s3-lifecycle-rules","tag-security-hub","tag-sensitive-data-discovery","entry","has-media"],"yoast_head":"<!-- This site is optimized with the Yoast SEO plugin v28.4 - https:\/\/yoast.com\/product\/yoast-seo-wordpress\/ -->\n<title>Amazon Macie PII Detection: Close the Coverage Gaps<\/title>\n<meta name=\"description\" content=\"Amazon Macie PII detection silently skips objects it can&#039;t read. Find the coverage gaps, tune identifiers, and control cost before trusting a score.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Amazon Macie PII Detection: Close the Coverage Gaps\" \/>\n<meta property=\"og:description\" content=\"Amazon Macie PII detection silently skips objects it can&#039;t read. Find the coverage gaps, tune identifiers, and control cost before trusting a score.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/\" \/>\n<meta property=\"og:site_name\" content=\"John Nessime\" \/>\n<meta property=\"article:published_time\" content=\"2026-09-06T06:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2026-09-14T13:12:59+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/amazon-macie-pii-detection-coverage.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1448\" \/>\n\t<meta property=\"og:image:height\" content=\"1086\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"John Nessime\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"John Nessime\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"15 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/\"},\"author\":{\"name\":\"John Nessime\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"headline\":\"Amazon Macie PII Detection: The Buckets It Never Opened\",\"datePublished\":\"2026-09-06T06:00:00+00:00\",\"dateModified\":\"2026-09-14T13:12:59+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/\"},\"wordCount\":3200,\"publisher\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"image\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/amazon-macie-pii-detection-coverage.png\",\"keywords\":[\"Amazon Data Firehose\",\"Amazon Macie\",\"Amazon S3\",\"AWS\",\"AWS KMS\",\"AWS Organizations\",\"Cloud Security\",\"Compliance\",\"Cost Optimization\",\"Data Classification\",\"Data Governance\",\"Data Lake\",\"Encryption\",\"EventBridge\",\"PII Redaction\",\"S3 Lifecycle Rules\",\"Security Hub\",\"Sensitive Data Discovery\"],\"articleSection\":[\"Cloud Computing\",\"Compliance\",\"Data Engineering\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/\",\"name\":\"Amazon Macie PII Detection: Close the Coverage Gaps\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/amazon-macie-pii-detection-coverage.png\",\"datePublished\":\"2026-09-06T06:00:00+00:00\",\"dateModified\":\"2026-09-14T13:12:59+00:00\",\"description\":\"Amazon Macie PII detection silently skips objects it can't read. Find the coverage gaps, tune identifiers, and control cost before trusting a score.\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/#primaryimage\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/amazon-macie-pii-detection-coverage.png\",\"contentUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/09\\\/amazon-macie-pii-detection-coverage.png\",\"width\":1448,\"height\":1086,\"caption\":\"Amazon Macie PII detection coverage diagram: a grid of S3 object tiles where most are dimmed as never opened, a few amber for unreadable and teal for analyzed, three red for sensitive data found, next to a sensitivity score of 50 labeled Not yet analyzed\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/cloud-computing\\\/amazon-macie-pii-detection\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Amazon Macie PII Detection: The Buckets It Never Opened\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/\",\"name\":\"John Nessime\",\"description\":\"Cloud, DevOps, Data &amp; AI \u2014 Built, Tested, Explained\",\"publisher\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/#\\\/schema\\\/person\\\/ede0b56d0c808f123f57d5d796902105\",\"name\":\"John Nessime\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"contentUrl\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\",\"width\":512,\"height\":512,\"caption\":\"John Nessime\"},\"logo\":{\"@id\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/wp-content\\\/uploads\\\/2026\\\/07\\\/cropped-jn.png\"},\"description\":\"AWS Certified Solutions Architect helping businesses build reliable cloud, data, reporting, and automation solutions. I help startups, agencies, and growing businesses replace manual processes and disconnected data with practical AWS architectures, clean data pipelines, useful dashboards, and maintainable automation.\",\"sameAs\":[\"https:\\\/\\\/john-nessime.com\\\/blog\",\"https:\\\/\\\/www.linkedin.com\\\/in\\\/john-m-nessime\"],\"url\":\"https:\\\/\\\/john-nessime.com\\\/blog\\\/author\\\/johnnessime\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Amazon Macie PII Detection: Close the Coverage Gaps","description":"Amazon Macie PII detection silently skips objects it can't read. Find the coverage gaps, tune identifiers, and control cost before trusting a score.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/","og_locale":"en_US","og_type":"article","og_title":"Amazon Macie PII Detection: Close the Coverage Gaps","og_description":"Amazon Macie PII detection silently skips objects it can't read. Find the coverage gaps, tune identifiers, and control cost before trusting a score.","og_url":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/","og_site_name":"John Nessime","article_published_time":"2026-09-06T06:00:00+00:00","article_modified_time":"2026-09-14T13:12:59+00:00","og_image":[{"width":1448,"height":1086,"url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/amazon-macie-pii-detection-coverage.png","type":"image\/png"}],"author":"John Nessime","twitter_card":"summary_large_image","twitter_misc":{"Written by":"John Nessime","Est. reading time":"15 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/#article","isPartOf":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/"},"author":{"name":"John Nessime","@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"headline":"Amazon Macie PII Detection: The Buckets It Never Opened","datePublished":"2026-09-06T06:00:00+00:00","dateModified":"2026-09-14T13:12:59+00:00","mainEntityOfPage":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/"},"wordCount":3200,"publisher":{"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"image":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/#primaryimage"},"thumbnailUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/amazon-macie-pii-detection-coverage.png","keywords":["Amazon Data Firehose","Amazon Macie","Amazon S3","AWS","AWS KMS","AWS Organizations","Cloud Security","Compliance","Cost Optimization","Data Classification","Data Governance","Data Lake","Encryption","EventBridge","PII Redaction","S3 Lifecycle Rules","Security Hub","Sensitive Data Discovery"],"articleSection":["Cloud Computing","Compliance","Data Engineering"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/","url":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/","name":"Amazon Macie PII Detection: Close the Coverage Gaps","isPartOf":{"@id":"https:\/\/john-nessime.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/#primaryimage"},"image":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/#primaryimage"},"thumbnailUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/amazon-macie-pii-detection-coverage.png","datePublished":"2026-09-06T06:00:00+00:00","dateModified":"2026-09-14T13:12:59+00:00","description":"Amazon Macie PII detection silently skips objects it can't read. Find the coverage gaps, tune identifiers, and control cost before trusting a score.","breadcrumb":{"@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/#primaryimage","url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/amazon-macie-pii-detection-coverage.png","contentUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/09\/amazon-macie-pii-detection-coverage.png","width":1448,"height":1086,"caption":"Amazon Macie PII detection coverage diagram: a grid of S3 object tiles where most are dimmed as never opened, a few amber for unreadable and teal for analyzed, three red for sensitive data found, next to a sensitivity score of 50 labeled Not yet analyzed"},{"@type":"BreadcrumbList","@id":"https:\/\/john-nessime.com\/blog\/cloud-computing\/amazon-macie-pii-detection\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/john-nessime.com\/blog\/"},{"@type":"ListItem","position":2,"name":"Amazon Macie PII Detection: The Buckets It Never Opened"}]},{"@type":"WebSite","@id":"https:\/\/john-nessime.com\/blog\/#website","url":"https:\/\/john-nessime.com\/blog\/","name":"John Nessime","description":"Cloud, DevOps, Data &amp; AI \u2014 Built, Tested, Explained","publisher":{"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/john-nessime.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/john-nessime.com\/blog\/#\/schema\/person\/ede0b56d0c808f123f57d5d796902105","name":"John Nessime","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","url":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","contentUrl":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png","width":512,"height":512,"caption":"John Nessime"},"logo":{"@id":"https:\/\/john-nessime.com\/blog\/wp-content\/uploads\/2026\/07\/cropped-jn.png"},"description":"AWS Certified Solutions Architect helping businesses build reliable cloud, data, reporting, and automation solutions. I help startups, agencies, and growing businesses replace manual processes and disconnected data with practical AWS architectures, clean data pipelines, useful dashboards, and maintainable automation.","sameAs":["https:\/\/john-nessime.com\/blog","https:\/\/www.linkedin.com\/in\/john-m-nessime"],"url":"https:\/\/john-nessime.com\/blog\/author\/johnnessime\/"}]}},"_links":{"self":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/405","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/comments?post=405"}],"version-history":[{"count":1,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/405\/revisions"}],"predecessor-version":[{"id":414,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/posts\/405\/revisions\/414"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/media\/925"}],"wp:attachment":[{"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/media?parent=405"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/categories?post=405"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/john-nessime.com\/blog\/wp-json\/wp\/v2\/tags?post=405"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}