Remote Senior Data Engineer at IT Labs

Description:

As a Senior Data Engineer at IT Labs, you will be responsible for maximizing the value of extensive data processed through the API platform, which handles billions in payments monthly for millions of users.
Your main tasks will include crafting and executing a technical vision for AWS-based data pipelines, supporting new product developments and data science initiatives, and ensuring optimal system performance.
Key aspects of the role involve proficiency in infrastructure-as-code and CI/CD systems like Terraform, Troposphere, CloudFormation, and Jenkins, as well as experience in administering Microsoft SQL Server, especially in high-availability setups.
You will spend approximately 50% of your time building and enhancing data lakes and pipelines, 20% on defining and implementing DataOps methodologies, 20% on writing and optimizing queries and reports, and 10% on supporting and monitoring existing databases.

Minimum of 7 years in a Data Engineering position.
Expert knowledge in Python.
Proven experience with AWS technologies for building and managing data pipelines and performing ETL processes.
Proficiency in using AWS warehousing and analysis tools like Redshift, Quicksight, and Athena.
Skilled in infrastructure-as-code practices, including CloudFormation, CDK, and Serverless frameworks.
Expertise in designing and reviewing complex Python programs, ensuring compliance with style and security standards.
Demonstrated ability to optimize SQL queries for maximum performance.
Hands-on experience with AWS RDS.
Ability to empathize with end-users to inform and guide decision-making processes.
Excellent analytical thinking, problem-solving capabilities, and superior written and verbal communication skills.
Fluent in spoken and written English.