Remote Senior Data Engineer

Description:

• We are seeking an experienced Sr Data Engineer to oversee the development and utilization of data systems.
• You will be reporting to the Sr. Manager – Data Engineering, to join our dynamic team in the Foreign Exchange payments processing industry.
• The ideal candidate is responsible for defining and implementing the data ETL pipelines and models and ensuring robust data governance across the organization.
• This role requires a deep understanding of business processes, technology, data management, and regulatory compliance.
• The successful candidate will work closely with business and IT leaders to ensure that the enterprise data platform supports business goals, and that data governance policies and standards are adhered to across the organization.
• Your responsibilities will include working closely with product managers, data architects, analysts, cross-functional teams, and other stakeholders to ensure that our data platform meets the needs of our organization and supports our data-driven initiatives.
• It also includes building a new data platform, integrating data from various sources, and ensuring data availability for various application and reporting needs.
• You will lead a team of data and cloud engineers and be hands-on in design and implementation, well-versed in development and deploying solutions on the AWS platform using CloudFormation, Glue, Lambda, CodePipeline, Step Functions, S3, RDS, Athena, Quicksight, Sagemaker, SQS, and SNS.
• You will design and implement a real-time prediction service integrated with the CRM system for new requirements using AWS services.
• You will productionize automated infrastructure DevOps for cloud and on-prem using Ansible, Puppet, and CloudFormation.
• You will deploy and manage a multi-node MapR/Hadoop cluster with various components.
• You will design and develop ETL/ELT procedures, create mappings and sessions for data conversion and loading.
• You will implement data transformation and processing solutions using Big Data platform tools.
• You will be proactive in solution development and process improvements.
• You will participate in the decision-making process in design architecture, solution development, and architectural and code review.
• You will handle data cloud engineering responsibilities like data modeling, database schema design in AWS, and Python scripting within AWS.
• You will transform data into new schema and index it into Elasticsearch.
• You will collaborate directly with customer's architects, leads, and engineers to align on exact solution requirements.
• You will collaborate with PM, PO, other engineers, architects, QA, and other colleagues and stakeholders.
• You will participate in delivering the product roadmap, planning, and creation of estimations.
• You will deliver high-quality data and cloud engineering deliverables and guide the team with best practices to be highly performant.
• You will think outside the box to come up with solutions that scale, are fault-tolerant, easy to grasp, and elegant engineering output.

Requirements:

• You should have 10+ years of experience as a Cloud Data Engineer with proficiency in programming languages such as Python, Java, or Scala, focusing on data processing frameworks.
• You must have strong cloud data engineering experience in large software development projects.
• You should have strong experience with data architecture principles, including data modeling, ETL/ELT processes, and data management, along with hands-on experience with Big Data technologies.
• You must have hands-on experience with AWS and good knowledge of its sub-components and services.
• You should be strong in SQL, Redshift, Athena, data modeling, and Elasticsearch.
• You must have experience with CloudFormation and/or Terraform.
• You should have experience with data warehouses.
• You must have experience with Agile projects in product development by following Agile Development methodologies.
• You should have an experienced understanding of the Software Development Lifecycle.
• You must possess excellent communication and collaboration skills.
• You should have the ability to multi-task, prioritize, and execute tasks in a fast-paced environment.
• You must have a strong sense of ownership and responsibility for deliverables.
• You should have great team ethics and collaboration skills for internationally distributed and remote work with the team.

Benefits:

• The position offers the flexibility of working from home in the USA.
• You will be part of a dynamic team in the Foreign Exchange payments processing industry.
• You will have the opportunity to lead a team of data and cloud engineers.
• You will work with cutting-edge technologies and tools in cloud data engineering.
• You will have the chance to collaborate with various stakeholders and contribute to significant data-driven initiatives.
• The role provides opportunities for professional growth and development in a fast-paced environment.
• You will be involved in innovative solution development that impacts the organization positively.