Description:

Octopus by RTG is seeking a Senior Data Engineer to connect top organizations with exceptional talent.
The role involves designing, developing, and maintaining robust data pipelines to support machine learning workflows and GenAI applications.
Responsibilities include implementing data ingestion, transformation, and storage solutions for both structured and unstructured data.
The Senior Data Engineer will ensure data quality, integrity, and consistency across the entire pipeline.
The position requires optimizing data infrastructure for scalability, performance, and cost-efficiency.
The role also involves implementing real-time data processing workflows.
Collaboration with ML engineers and data scientists is essential to ensure seamless integration of data pipelines with models and applications.

Requirements:

Candidates must have proficiency in programming languages for data processing, such as Python, Scala, or Java.
A strong experience with big data technologies, including Hadoop and Spark, as well as ETL tools, is required.
Familiarity with data storage systems, such as SQL databases, NoSQL databases, and data lakes, is necessary.
Applicants should have strong experience with vector databases and embedding stores.
Experience with cloud platforms and data services, including AWS Redshift, Google BigQuery, and Azure Data Factory, is essential.
Knowledge of data modeling, warehousing, and real-time processing frameworks, such as Kafka and Flink, is required.
Strong problem-solving skills and the ability to work in cross-functional teams are necessary for this position.

The job offers the opportunity to work with top-notch organizations and talents globally.
Employees will have the chance to design and implement cutting-edge data solutions.
The position provides a collaborative work environment with professionals in machine learning and data science.
There are opportunities for professional growth and development in the field of data engineering.
The role allows for the implementation of innovative technologies and methodologies in data processing.