Remote Data Engineer

Description:

• The Data Engineer will be responsible for designing, developing, and maintaining data pipelines for threat intelligence ingestion, validation, and export automation flows.
• Collaborate with threat analysts to understand data requirements and develop appropriate solutions.
• Develop automation scripts and workflows for data export processes to external systems or partners.
• Optimize and enhance existing data pipelines for improved performance and scalability.
• Monitor data pipelines, troubleshoot issues, and ensure continuous data availability and integrity.
• Document technical specifications, data flows, and procedures for data pipeline maintenance and support.
• Stay updated on emerging technologies and best practices in data engineering.
• Provide technical guidance and support to team members on data engineering best practices.

Requirements:

• Bachelor's degree in Computer Science, Engineering, or related field.
• Proven experience as a Data Engineer with a focus on data ingest, validation, and export automation.
• Strong proficiency in Python.
• Experience with data pipeline orchestration tools like Apache Airflow, Apache NiFi, or similar.
• Solid understanding of database systems, data modeling, SQL, and cloud platforms like Snowflake, AWS, Azure, or Google Cloud Platform.
• Experience with data validation techniques, containerization technologies, and schema validation.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills.

Benefits:

• Fully remote position.
• Opportunity to work with a skilled Threat Research team.
• Chance to contribute to building scalable and efficient data pipelines.
• Exposure to emerging technologies and best practices in data engineering.
• Collaborative team environment with technical guidance and support available.