Remote Data Engineer

Description:

• We are a team of highly-driven individuals who are passionate about technology, algorithmic trading, and solving intellectually challenging problems.
• Being a part of Sparkland means you get to work with some of the brightest people in one of the world’s fastest-growing and most exciting industries.
• We are fully remote and have a flat corporate structure that values open-mindedness, entrepreneurial spirit, commitment to excellence, and continuous learning.
• We are looking for a Data Engineer to help us build and maintain the data backbone of our trading platform.
• You will be working on high-volume data pipelines, ensuring the reliability and observability of our infrastructure, and preparing the system for upcoming ML initiatives.
• Responsibilities include designing and maintaining robust data pipelines to support real-time and batch processing, managing and optimizing our Clickhouse data warehouse, ensuring data quality and governance, collaborating with various teams, supporting internal initiatives, and taking ownership of scheduling and workflow orchestration.

Requirements:

• At least 5 years of professional experience in data engineering or backend infrastructure is required.
• Proficiency in Python, including object-oriented programming and testing, is necessary.
• Solid experience with SQL, including complex joins, window functions, and performance optimization, is essential.
• Hands-on experience with Clickhouse, especially the MergeTree engine family, or similar columnar databases is required.
• Familiarity with workflow schedulers such as Argo Workflows, Airflow, or Kubeflow is necessary.
• Understanding of Kafka architecture, including topics, partitions, producers, and consumers, is required.
• Comfort with CI/CD pipelines, including GitLab CI, ArgoCD, and GitHub Actions, is essential.
• Experience with monitoring and BI tools such as Grafana for technical and business dashboards is required.

Benefits:

• You will have the opportunity to work in a fully remote environment.
• You will be part of a flat corporate structure that encourages open-mindedness and entrepreneurial spirit.
• The role offers the chance to collaborate with some of the brightest minds in the industry.
• Continuous learning and commitment to excellence are core values of the company, providing a supportive environment for professional growth.