Apache Airflow on AWS: Building SaaS and API Pipelines That Don’t Lie to You

Most API pipeline failures are green DAGs producing incomplete data. A practical guide to running Apache Airflow on AWS for SaaS and API extraction: choosing between MWAA provisioned, MWAA Serverless and self-managed, the pool setting that silently stops throttling when you go deferrable, retry and pagination design, secrets handling, and the four cost lines that actually move.

Continue ReadingApache Airflow on AWS: Building SaaS and API Pipelines That Don’t Lie to You

Building a Jira Analytics Pipeline with AWS Lambda and Athena (Without Double-Counting Everything)

Jira's built-in reports stop at the board boundary. This guide walks through a Jira analytics pipeline built on AWS Lambda, S3 and Athena, organised around the four failure families that actually bite: the removed search endpoint, silently truncated changelogs, incremental loads that duplicate rows, and an S3 layout that quietly inflates your query bill.

Continue ReadingBuilding a Jira Analytics Pipeline with AWS Lambda and Athena (Without Double-Counting Everything)

Redshift Cost Optimization for SaaS Analytics: The Levers That Actually Move the Bill

In a SaaS analytics product, the Redshift bill tracks how often queries arrive, not how much data they touch. Here is how the meter actually works, why connection pools bill you while nobody is using the product, how to attribute spend to a tenant, and which isolation choices quietly cost more than they save.

Continue ReadingRedshift Cost Optimization for SaaS Analytics: The Levers That Actually Move the Bill