Remote Staff Data Engineer (Data Lake)

Description:

• Red Canary is a cyber security company focused on empowering organizations to make better security decisions.
• The company is building a state-of-the-art security data lake to enhance its capabilities in detecting and responding to cyber threats.
• As a Staff Data Engineer, you will contribute to the development and optimization of data pipelines and frameworks that deliver actionable insights.
• Your role will involve designing, developing, and maintaining data pipelines and workflows with a focus on cybersecurity.
• You will collaborate with various departments to integrate cybersecurity data insights into Red Canary’s services.
• The position requires overseeing data quality, reliability, and security while implementing modern tools and technologies.
• You will troubleshoot data pipeline challenges and actively participate in discussions regarding architecture and technical designs.

Requirements:

• You must have demonstrated impact at scale in data engineering, preferably with a background in cybersecurity or a related field.
• Expertise in programming and scripting languages such as Python, SQL, and Scala or PySpark is required.
• Experience with cloud platforms, particularly AWS (Glue, EMR, Athena, S3, Redshift), is essential.
• A strong understanding of modern data storage formats like Parquet and Delta Lake, as well as open table formats (Iceberg, Hudi), is necessary.
• Familiarity with big data frameworks (Apache Spark, Kafka) and orchestration tools like Apache Airflow is expected.
• Experience with ETL/ELT pipelines and transformation tools (e.g., dbt) is required.
• A solid understanding of containerization and orchestration technologies, such as Docker and Kubernetes, is needed.
• Strong problem-solving skills and the ability to debug complex data workflows are essential.
• Effective communication skills for collaboration across technical and non-technical teams are necessary.
• A desire to mentor and develop others, sharing expertise to elevate the team's capabilities, is important.

Benefits:

• Red Canary offers a 100% paid plan option for medical, dental, and vision for you and your dependents, with no waiting period.
• Access to mental health services, an Employee Assistance Program, and additional wellness programs are provided.
• All new hires are eligible for fertility benefits from their first day.
• The company offers flexible time off, allowing employees to take the time needed to recharge, including vacation, sick leave, and holidays.
• Paid parental leave is provided at full base pay to bond with or care for a new child.
• Red Canary offers various pre-tax plans, including FSA, HRA, and HSA, with employer funding to help offset out-of-pocket healthcare expenses.
• A flexible work environment is available, with the option to work virtually from almost anywhere in the US.