How we use Airflow with Composer
At Lastminute.com, we use Apache Airflow, managed through Google Cloud Composer, to orchestrate complex workflows. This article explores why we chose Composer as our Airflow distribution, how we use it to schedule a wide range of tasks, and how we organize files and resources to ensure scalability and maintainability. Additionally, we delve into our CI/CD practices, highlighting how we test workflows and manage costs, offering insights for teams seeking efficient and reliable data orchestration at scale in big tech company.
