Description:

As an AWS Data Engineer, you will design, develop, and maintain scalable data pipelines on AWS.
You will collaborate closely with technical analysts, client stakeholders, data scientists, and other team members to ensure data quality and integrity.
Your role will involve optimizing data storage solutions for performance and cost-efficiency.
You will lead and support the delivery of data platform modernization projects.
Responsibilities include designing and developing robust and scalable data pipelines leveraging AWS native services.
You will optimize ETL processes to ensure efficient data transformation.
You will migrate workflows from on-premise to AWS cloud, ensuring data quality and consistency.
Your tasks will include designing automations and integrations to resolve data inconsistencies and quality issues.
You will perform system testing and validation to ensure successful integration and functionality.
Implementing security and compliance controls in the cloud environment will be part of your responsibilities.
You will ensure data quality pre- and post-migration through validation checks and addressing issues regarding completeness, consistency, and accuracy of data sets.
Collaboration with data architects and lead developers to identify and document manual data movement workflows and design automation strategies will be essential.

Requirements:

You must have 10+ years of experience with a core data engineering skillset leveraging AWS native technologies such as AWS Glue, Python, Snowflake, S3, and Redshift.
Experience in the design and development of robust and scalable data pipelines leveraging AWS native services is required.
Proficiency in leveraging Snowflake for data transformations, optimization of ETL pipelines, and scalable data processing is necessary.
You should have experience with streaming and batch data pipeline/engineering architectures.
Familiarity with DataOps concepts and tooling for source control and setting up CI/CD pipelines on AWS is expected.
Hands-on experience with Databricks and a willingness to grow capabilities is essential.
You must have experience with data engineering and storage solutions including AWS Glue, EMR, Lambda, Redshift, and S3.
Strong problem-solving and analytical skills are required.
Knowledge of Dataiku is needed.
A Graduate/Post-Graduate degree in Computer Science or a related field is required.
You should have experience with AWS S3 for data storage, export, and recall.
Familiarity with Athena for querying data lakes is necessary.
Experience with data pipelines (batch & near-real-time) and integration with external systems (FHIR) is expected.
Knowledge of secure data handling (KMS, Macie) and cloud-native analytics is required.
You should be familiar with multi-account, multi-region data architecture and BI integrations such as Power BI, Tableau, and QuickSight.

The position offers the opportunity to work with cutting-edge AWS technologies and tools.
You will be part of a collaborative team environment that values innovation and continuous improvement.
The role provides a chance to lead significant data modernization projects that impact the organization.
You will have opportunities for professional growth and development in data engineering and cloud technologies.
The company promotes a culture of learning and encourages employees to expand their skill sets.
Competitive compensation and benefits package will be provided.

Skills