Remote Senior Data Engineer ETL

Description:

• MediaRadar is seeking an experienced Senior Data Engineer to take ownership of their data infrastructure.
• The role is remote within the US, and candidates must be legally authorized to work in the United States without visa sponsorship.
• The Senior Data Engineer will architect, build, and scale robust data pipelines to ensure flawless data quality.
• Responsibilities include designing, implementing, and optimizing scalable ELT/ETL pipelines using Databricks, Spark, Python, and Store Procedures.
• The engineer will lead and mentor a team of data engineers, providing technical direction and resolving challenges.
• Ensuring data integrity through comprehensive testing frameworks, data validation rules, and QA plans is essential.
• The role involves optimizing and troubleshooting system performance, tuning complex SQL queries, and performing root-cause analysis.
• Collaboration in an Agile environment with cross-functional teams is expected, along with evaluating and introducing new technologies.

Requirements:

• A bachelor’s or master’s degree in computer science, engineering, or a related field, or equivalent practical experience is required.
• Candidates must have 7+ years of hands-on experience in data engineering, focusing on Databricks, Apache Spark, and one RDBMS solution for processing large-scale datasets.
• Expert-level proficiency in Python, including PySpark, and a strong understanding of data structures, algorithms, and OOP principles are necessary.
• Deep expertise in SQL and Spark SQL, with proven experience in writing and optimizing complex analytical queries, is required.
• Hands-on experience with the Databricks ecosystem, including Delta Lake, Unity Catalog, and Data frame APIs, is essential.

Benefits:

• The position offers a comprehensive benefits package, including medical, dental, and vision insurance.
• Employees will have access to a 401k plan with company match.
• Flexible PTO is provided to support work/life balance.
• Commuter benefits and gym discounts are included.
• The company observes Summer Fridays, allowing for additional time off during the summer months.