The Data Engineer will architect, develop, and maintain scalable data pipelines within a medallion architecture, specifically the bronze, silver (base vault with DBT and orchestration tools, business vault), and gold layers.
This role is key in enabling high-quality, business-ready datasets by leveraging modern data engineering technologies and orchestration practices.
Responsibilities include designing, building, and managing end-to-end data pipelines across the medallion architecture.
The engineer will ingest and process raw data using Spark and Amazon EMR for scalable, distributed computation.
The role involves developing and automating data transformations for the base vault using DBT (Data Build Tool) to standardize and model data efficiently.
Requirements:
At least 5 years of experience as an Elasticsearch Data Engineer is required.
A BS in Computer Science, Data Engineering (Big Data, AWS certification), Data Modeling, or a similar field is necessary.
Full English fluency is mandatory.
A strong understanding of data modeling, governance, and best practices in modern data architectures is essential.
Excellent analytical, problem-solving, and communication skills are required.
Must have strong experience with Elasticsearch, including cluster optimization, query development, data modeling, performance tuning, and administration (4-6 years).
Deep experience with Spark, Python, ETLs, and Amazon EMR is a must.
Hands-on experience with DBT for data transformation and modeling is required.
Familiarity with Apache Airflow, AWS Step Functions, or similar orchestration tools is necessary.
Expert knowledge of Amazon S3 and Apache Iceberg for data storage and management is required.
Experience with Kubernetes for container orchestration is preferred.
Experience with Dremio, Looker, or equivalent business view/semantic layer technologies is a plus.
Intermediate knowledge of AWS Cloud services, including AWS Lambda, Step Functions, IAM, SNS, API Gateway, VPC, and Transit Gateway (3-4 years).
Intermediate experience with JSON (4-6 years) is required.
Intermediate experience with Jenkins for data pipelines (4-6 years) is a plus.
Intermediate experience with CloudWatch (4-6 years) is also a plus.
Benefits:
The position offers a competitive salary and performance-based bonuses.
A comprehensive benefits package is included.
There are career development and training opportunities available.
Flexible work arrangements, including remote and/or office-based options, are provided.
The work culture is dynamic and inclusive within a globally renowned group.
Private health insurance is part of the benefits.
A pension plan is offered.
Paid time off is included.
Training and development opportunities are available.
