Remote Senior Data Engineer

Description:

• Join a team modernizing OAG’s core Flight Status data platform.
• Build high-throughput, near real-time ingestion and processing pipelines using Azure Databricks and Delta Lake.

Requirements:

• Strong experience in building and scaling production-grade streaming and batch ETL/ELT data pipelines.
• Proficiency with Azure Databricks, Apache Spark (PySpark), Structured Streaming, Delta Lake, and Unity Catalog.
• Deep understanding of modern data patterns and pipeline design.
• Experience with Databricks Workflows and event streaming platforms.
• Strong background in automated testing, CI/CD tools, and data observability practices.
• Experience using AI coding assistants to enhance development.
• Excellent analytical skills and technical communication abilities.

Benefits:

• Flexible working arrangements with options for remote work.
• Attractive compensation package including private health insurance, bonus scheme, and pension.
• 22 days holiday plus additional benefits for service and personal milestones.
• Annual learning and development budget.
• Opportunity to work remotely anywhere in the world for up to 1 month per year.
• Free lunch and snacks provided in the office.