Description:

MentorMate is seeking a skilled Data Engineer with a strong background in Spark and AWS to join their team.
The role involves building and managing data pipelines with Spark and Databricks.
Responsibilities include developing and maintaining scalable data pipelines using Apache Spark.
The Data Engineer will work within Databricks notebooks to run data processing tasks.
The position requires ingesting data from various sources, including enterprise software such as Salesforce and NetSuite, as well as internal software systems.
The role utilizes AWS tools, including S3, Redshift, and Aurora PostgreSQL, to manage and store data.
The Data Engineer will implement a medallion architecture for data organization and processing.
Collaboration with cross-functional teams is essential to understand data requirements and deliver high-quality data solutions.
The position also involves maintaining and enhancing existing data pipelines with a focus on performance and scalability.
Ensuring data quality, integrity, and security across all managed systems is a key responsibility.

Requirements:

Candidates must have 3+ years of experience in Data Engineering.
Proven experience as a Data Engineer with a strong focus on Apache Spark is required.
Experience with AWS services such as S3, Redshift, and Aurora PostgreSQL is necessary.
A strong knowledge of ingesting and processing data from diverse sources is essential.
Candidates should have a good understanding and practical experience with medallion architecture.
Experience with modern SDLC, CI/CD, and SCM is required.
Excellent problem-solving skills and the ability to work independently or as part of a team are necessary.
Strong communication skills are needed to collaborate effectively with technical and non-technical stakeholders.
Experience working with Databricks for data processing is considered a significant advantage.