Remote AWS Data Engineer

Description:

• As an AWS Data Engineer, you will design, develop, and maintain scalable data pipelines on AWS.
• You will work closely with technical analysts, client stakeholders, data scientists, and other team members to ensure data quality and integrity.
• Your role involves optimizing data storage solutions for performance and cost-efficiency.
• You will leverage AWS native technologies and Databricks for data transformations and scalable data processing.
• Responsibilities include leading and supporting the delivery of data platform modernization projects.
• You will design and develop robust and scalable data pipelines leveraging AWS native services.
• Optimizing ETL processes to ensure efficient data transformation is a key responsibility.
• You will migrate workflows from on-premise to AWS cloud, ensuring data quality and consistency.
• Designing automations and integrations to resolve data inconsistencies and quality issues is part of your role.
• You will perform system testing and validation to ensure successful integration and functionality.
• Implementing security and compliance controls in the cloud environment is essential.
• You will ensure data quality pre- and post-migration through validation checks and addressing issues regarding completeness, consistency, and accuracy of data sets.
• Collaboration with data architects and lead developers to identify and document manual data movement workflows and design automation strategies is required.

Requirements:

• You must have 10+ years of experience with a core data engineering skillset leveraging AWS native technologies such as AWS Glue, Python, Snowflake, S3, and Redshift.
• Experience in the design and development of robust and scalable data pipelines leveraging AWS native services is required.
• Proficiency in leveraging Snowflake for data transformations, optimization of ETL pipelines, and scalable data processing is necessary.
• You should have experience with streaming and batch data pipeline/engineering architectures.
• Familiarity with DataOps concepts and tooling for source control and setting up CI/CD pipelines on AWS is needed.
• Hands-on experience with Databricks and a willingness to grow capabilities is essential.
• Experience with data engineering and storage solutions including AWS Glue, EMR, Lambda, Redshift, and S3 is required.
• Strong problem-solving and analytical skills are necessary for this position.
• Knowledge of Dataiku is needed.
• A Graduate/Post-Graduate degree in Computer Science or a related field is required.
• Familiarity with AWS S3 for data storage, export, and recall is necessary.
• Experience with Athena for querying data lakes is required.
• You should have experience with data pipelines (batch & near-real-time) and integration with external systems (FHIR).
• Knowledge of secure data handling (KMS, Macie) is essential.
• Experience with cloud-native analytics and multi-account, multi-region data architecture is required.
• Familiarity with BI integrations such as Power BI, Tableau, and QuickSight is necessary.

Benefits:

• The position offers the opportunity to work with cutting-edge AWS technologies and tools.
• You will be part of a collaborative team environment that values innovation and continuous improvement.
• The role provides opportunities for professional growth and development in data engineering.
• You will have the chance to work on impactful projects that modernize data platforms.
• Competitive salary and benefits package will be provided.
• Flexible working arrangements may be available to support work-life balance.