Remote Senior Data Engineer

Description:

• Octopus by RTG is seeking a Senior Data Engineer to connect top organizations with exceptional talent.
• The role involves designing, developing, and maintaining robust data pipelines to support machine learning workflows and GenAI applications.
• Responsibilities include implementing data ingestion, transformation, and storage solutions for both structured and unstructured data.
• The Senior Data Engineer will ensure data quality, integrity, and consistency across the entire pipeline.
• The position requires optimizing data infrastructure for scalability, performance, and cost-efficiency.
• The role also involves implementing real-time data processing workflows.
• Collaboration with ML engineers and data scientists is essential to ensure seamless integration of data pipelines with models and applications.

Requirements:

• Candidates must have proficiency in programming languages for data processing, such as Python, Scala, or Java.
• A strong experience with big data technologies, including Hadoop and Spark, as well as ETL tools, is required.
• Familiarity with data storage systems, such as SQL databases, NoSQL databases, and data lakes, is necessary.
• Applicants should have strong experience with vector databases and embedding stores.
• Experience with cloud platforms and data services, including AWS Redshift, Google BigQuery, and Azure Data Factory, is essential.
• Knowledge of data modeling, warehousing, and real-time processing frameworks, such as Kafka and Flink, is required.
• Strong problem-solving skills and the ability to work in cross-functional teams are necessary for this position.

Benefits:

• The job offers the opportunity to work with top-notch organizations and talents globally.
• Employees will have the chance to design and implement cutting-edge data solutions.
• The position provides a collaborative work environment with professionals in machine learning and data science.
• There are opportunities for professional growth and development in the field of data engineering.
• The role allows for the implementation of innovative technologies and methodologies in data processing.