Remote Big Data Engineer

Description:

• You will work closely with cross-functional teams to deliver high-quality solutions in various domains such as Supply Chain, Finance, Operations, Customer Experience, HR, Risk Management, and Global IT.
• You will be part of the technical plan for the migration, which includes data ingestion, transformation, storage, and access control in Azure's Data Factory and data lake.
• You will design and implement scalable and efficient data pipelines to ensure smooth data movement from multiple sources using Azure Databricks.
• You will develop scalable and reusable frameworks for ingesting data sets.
• You will ensure data quality and integrity throughout the entire data pipeline by implementing robust data validation and cleansing mechanisms.
• You will work with event-based/streaming technologies to ingest and process data.
• You will provide support to the team by resolving any technical challenges or issues that may arise during the migration and post-migration phases.
• You will stay up to date with the latest advancements in cloud computing, data engineering, and analytics technologies, and recommend best practices and industry standards for implementing the data lake solution.

Requirements:

• You must have 2+ years of experience working within the Data Engineering field.
• You should have hands-on working experience with Azure Databricks.
• You need to have experience in Data Modelling and Source System Analysis.
• Familiarity with PySpark is required.
• You must have mastery of SQL.
• You should possess knowledge of components such as Azure Data Factory, Azure Data Lake, Azure SQL DW, and Azure SQL.
• Experience with the Python programming language used for data engineering purposes is necessary.
• You must have the ability to conduct data profiling, cataloging, and mapping for technical design and construction of technical data flows.
• Experience in data visualization/exploration tools is required.
• You should have excellent communication skills, with the ability to effectively convey complex ideas to both technical and non-technical stakeholders.
• You must be a strong team player with excellent interpersonal and collaboration skills.

Benefits:

• You will be part of a dynamic and non-hierarchical work culture.
• You will have the opportunity to work with a diverse team of over 17,500 experts across 37 countries.
• You will be involved in exciting projects that inspire, excite, and delight.
• You will have the chance to stay up to date with the latest advancements in cloud computing and data engineering technologies.
• You will be supported in your professional growth and development within the company.