Airflow Platform Engineer

About the job Airflow Platform Engineer Remote Key skill(if you dont have please dont apply, skill should be in the projects Apache Airflow 3.x DAG Development Python Kubernetes / AKS DBT Jenkins Docker Git Azure Azure Key Vault Snowflake Astronomer / Astro REST APIs Splunk Prometheus CI/CD Unit Testing Integration Testing Workflow Orchestration Data Pipelines Key Responsibilities DAG Development & Developer Enablement • Design, develop, and optimize Airflow DAGs for enterprise use cases • Demonstrate strong experience with operators such as PythonOperator, SQL operators, API/HTTP operators, KubernetesPodOperator etc. • Apply advanced DAG patterns including dynamic DAGs, branching, sensors, retries, and error handling • Troubleshoot DAG failures, performance issues, and integration challenges • Enable orchestration of modern data workflows including DBT pipelines • Provide reusable templates, best practices, and onboarding support to drive self-service adoption Platform Operations & Kubernetes Support • Support Airflow platform deployed on AKS, covering scheduler, webserver/API, workers, and metadata services • Troubleshoot issues related to task execution, scheduling delays, and resource bottlenecks • Diagnose Kubernetes-level issues involving pods, networking, storage, and RBAC • Work with Azure services such as Key Vault, Storage, and networking • Support CI/CD pipelines for DAG and container-based deployments • Improve platform stability, monitoring, and alerting Required Skills • Strong experience with Apache Airflow (3.x or later) in production environments • Proven experience in DAG development using operators and orchestration patterns • Working knowledge of Kubernetes (AKS preferred) to support Airflow runtime • Expert-level Python knowledge for building, debugging, and maintaining production- grade workflows and supporting libraries • Expertise in writing unit, integration, and DAG validation tests for Airflow workflows, including mocking operators, validating dependencies, and ensuring reliability in CI/CD pipelines • Experience with DBT and modern data pipelines • Experience with CI/CD tools (Git, Jenkins, Docker) • Familiarity with Azure or other cloud platforms Preferred Skills • Experience with Astronomer / Astro Private Cloud • Exposure to data platforms such as Snowflake, and APIs • Experience with monitoring tools such as Splunk or Prometheus • Knowledge of secrets management using Azure Key Vault Summary This role is critical to ensuring reliable platform operations and scalable adoption of the Airflow-based orchestration platform. The ideal candidate combines strong Airflow expertise, Kubernetes knowledge, and hands-on development experience to support both platform stability and developer productivity.