Remote PP - Data Engineer - 175

Description:

• We are seeking a highly skilled and motivated Senior Data Engineer to join our Credit Platform Data team.
• In this pivotal role, you will be responsible for designing, building, and maintaining scalable and reliable data pipelines and ETL processes that empower our internal business units with efficient data processing and utilization.
• Your expertise will directly impact the quality and accessibility of data, enabling data-driven decision-making across the organization.
• Collaborating closely with product managers, analysts, and stakeholders, you will translate complex data requirements into robust engineering solutions that handle large volumes of data with high performance and scalability.
• Responsibilities include architecting, developing, and maintaining scalable data pipelines and ETL workflows, implementing automated data quality checks, working with stakeholders to gather data requirements, monitoring and optimizing data systems, diagnosing and resolving data-related issues, maintaining documentation, staying updated on emerging technologies, and providing guidance to junior data engineers.

Requirements:

• A Bachelor’s degree in Computer Science, Engineering, or a related field is required.
• Expert-level proficiency in SQL for querying, manipulating, and optimizing relational databases is necessary.
• Strong programming skills in Python, including experience with data processing libraries such as Pandas, are required.
• Proficiency in using PySpark for distributed data processing on large-scale datasets is essential.
• A deep understanding of ETL concepts and hands-on experience designing and implementing ETL pipelines is required.
• Expertise in data modeling techniques to design logical and physical data models is necessary.
• Experience working with relational database management systems (RDBMS) such as Oracle, MySQL, or similar platforms is required.
• A solid understanding of data warehousing concepts, architectures, and best practices is necessary.
• Proficiency in Unix/Linux operating systems for managing data workflows is required.
• Ability to write shell scripts to automate routine tasks is essential.
• Experience implementing automated testing frameworks for data pipelines and ETL processes is required.
• A minimum of 3+ years of proven experience as a Data Engineer or in a similar role is necessary.

Benefits:

• The position offers the opportunity to work in a dynamic and innovative environment.
• You will have the chance to impact the organization by enabling data-driven decision-making.
• There is potential for professional growth and development through mentorship and collaboration with experienced team members.
• The role provides exposure to emerging data engineering technologies and best practices.
• You will be part of a culture that values knowledge sharing and technical excellence.