Remote Senior Data Engineer with Spark (m/f/d)

Description:

• MentorMate is seeking a skilled Senior Data Engineer with a strong background in Spark and AWS to join their team.
• The role involves building and managing data pipelines with Spark and Databricks, working with both internal and third-party data sources.
• Responsibilities include developing and maintaining scalable data pipelines using Apache Spark, running data processing tasks within Databricks notebooks, and ingesting data from various sources such as Salesforce and NetSuite.
• The Senior Data Engineer will utilize AWS tools, including S3, Redshift, and Aurora PostgreSQL, to manage and store data.
• The position requires implementing a medallion architecture for data organization and processing.
• Collaboration with cross-functional teams to understand data requirements and deliver high-quality data solutions is essential.
• The role also involves maintaining and enhancing existing data pipelines with a focus on performance and scalability, as well as ensuring data quality, integrity, and security across all managed systems.

Requirements:

• Candidates must have 7+ years of experience in Data Engineering.
• Proven experience as a Senior Data Engineer with a strong focus on Apache Spark is required.
• Experience with AWS services such as S3, Redshift, and Aurora PostgreSQL is necessary.
• A strong knowledge of ingesting and processing data from diverse sources is essential.
• Candidates should have a good understanding and practical experience with medallion architecture.
• Experience with modern SDLC, CI/CD, and SCM is required.
• Excellent problem-solving skills and the ability to work independently or as part of a team are necessary.
• Strong communication skills to collaborate effectively with both technical and non-technical stakeholders are essential.
• Experience working with Databricks for data processing is considered a significant advantage.

Benefits:

• MentorMate offers a friendly and global people-oriented environment that values every team member.
• The company provides flexible working models, competitive pay, and benefits.
• There are opportunities for career advancement within the organization.
• Employees can work on impactful, global projects for recognizable brands as part of a team of over 10,000 experts.
• MentorMate promotes a culture of energy and personality, encouraging employees to engage in various interests outside of work.