Remote Senior Data Engineer

Description:

• MediaRadar is seeking an experienced Senior Data Engineer to take ownership of their data infrastructure.
• The position is remote but requires availability during core working hours in the EST/CST time zones.
• The Senior Data Engineer will architect, build, and scale robust data pipelines to ensure flawless data quality.
• Responsibilities include designing, implementing, and optimizing scalable ELT/ETL pipelines using Databricks, Spark, Python, and Store Procedures.
• The role involves leading and mentoring a team of data engineers, providing technical direction, and resolving challenges.
• The engineer will develop and implement testing frameworks, data validation rules, and QA plans to ensure data integrity.
• Proactive monitoring of system performance, tuning complex SQL queries, and troubleshooting production issues are key tasks.
• Collaboration in an Agile environment with cross-functional teams is essential, along with evaluating and introducing new technologies.

Requirements:

• A bachelor’s or master’s degree in computer science, engineering, or a related field, or equivalent practical experience is required.
• Candidates must have 7+ years of hands-on experience in data engineering, focusing on Databricks, Apache Spark, and one RDBMS solution for large-scale datasets.
• Expert-level proficiency in Python, including PySpark, and a strong understanding of data structures, algorithms, and OOP principles are necessary.
• Deep expertise in SQL and Spark SQL, with proven experience in writing and optimizing complex analytical queries, is required.
• Hands-on experience with the Databricks ecosystem, including Delta Lake, Unity Catalog, and Data frame APIs, is essential.

Benefits:

• MediaRadar offers a comprehensive benefits package that includes medical, dental, and vision insurance.
• Employees can participate in a 401k plan with company match.
• The company provides flexible PTO to support work-life balance.
• Commuter benefits and gym discounts are available to employees.
• MediaRadar also offers Summer Fridays, allowing for extended weekends during the summer months.