Remote Senior Data Engineer / Architect

Description:

• The position is for a Senior Data Engineer / Architect with over 8 years of experience.
• The role is remote, allowing for flexible work arrangements.
• Responsibilities include designing, developing, and maintaining scalable and robust data solutions in the cloud using Apache Spark and Databricks.
• The candidate will gather and analyze data requirements from business stakeholders and identify opportunities for data-driven insights.
• Building and optimizing data pipelines for data ingestion, processing, and integration using Spark and Databricks is a key task.
• Ensuring data quality, integrity, and security throughout all stages of the data lifecycle is essential.
• The role involves collaborating with cross-functional teams to design and implement data models, schemas, and storage solutions.
• The candidate will optimize data processing and analytics performance by tuning Spark jobs and leveraging Databricks features.
• Providing technical guidance and expertise to junior data engineers and developers is expected.
• Staying up to date with emerging trends and technologies in cloud computing, big data, and data engineering is important.
• The candidate will contribute to the continuous improvement of data engineering processes, tools, and best practices.

Requirements:

• A Bachelor’s or master’s degree in computer science, engineering, or a related field is required.
• The candidate must have 10+ years of experience as a Data Engineer, Software Engineer, or in a similar role, focusing on building cloud-based data solutions.
• Strong knowledge and experience with the Azure cloud platform, Databricks, EventHub, Architecture, Spark, Kafka, ETL Pipeline, Python/Pyspark, and SQL are necessary.
• The candidate should have strong experience with cloud platforms, particularly Azure.
• Experience with big data systems, including Apache Spark and Kafka, is required.
• The candidate must have experience contributing to the architecture and design of large-scale distributed systems.
• Expertise in the Databricks Lakehouse Platform, its architecture, and its capabilities is essential.
• Experience building production pipelines using Databricks and Azure services is required.
• Proficiency in multiple coding languages such as Python or SQL is necessary.

Benefits:

• The position offers the flexibility of remote work.
• Candidates can join immediately or within a specified notice period, with options for permanent or contract roles.
• The role provides opportunities for professional growth and development in a cutting-edge field.
• The candidate will have the chance to work with advanced technologies and contribute to innovative data solutions.