Remote Senior Data Engineer (AWS)

Description:

• We are seeking a talented and experienced Data Engineer to join our team at Provectus.
• You will collaborate with a multidisciplinary team of data engineers, machine learning engineers, and application developers.
• The role involves encountering numerous technical challenges and contributing to Provectus’ open source projects.
• You will build internal solutions and engage in R&D activities, providing an excellent environment for professional growth.
• Responsibilities include collaborating closely with clients to understand their IT environments, applications, business requirements, and digital transformation goals.
• You will collect and manage large volumes of varied data sets and work directly with Data Scientists and ML Engineers to create robust data pipelines.
• The position requires defining data models that integrate disparate data across the organization and designing, implementing, and maintaining ETL/ELT data pipelines.
• You will perform data transformations using tools such as Spark, Trino, and AWS Athena to handle large volumes of data efficiently.
• Additionally, you will develop, continuously test, and deploy Data API Products with Python and frameworks like Flask or FastAPI.

Requirements:

• You must have experience in data engineering.
• Experience working with Cloud Solutions is required, preferably AWS, but GCP or Azure is also acceptable.
• You should have experience with Cloud Data Platforms such as Snowflake or Databricks.
• Proficiency with Infrastructure as Code (IaC) technologies like Terraform or AWS CloudFormation is necessary.
• You must have experience handling real-time and batch data flow and data warehousing with tools like Airflow, Dagster, Kafka, Apache Druid, Spark, and dbt.
• Proficiency in programming languages relevant to data engineering, such as Python and SQL, is required.
• Experience in building scalable APIs is essential.
• You should have experience in building Generative AI Applications, such as chatbots or RAG systems.
• Familiarity with Data Governance aspects like Quality, Discovery, Lineage, Security, Business Glossary, Modeling, Master Data, and Cost Optimization is important.
• Advanced or Fluent English skills are required.
• Strong problem-solving skills and the ability to work collaboratively in a fast-paced environment are necessary.

Benefits:

• You will have the opportunity to work in a diverse and multidisciplinary team.
• The role offers a chance to engage in R&D activities and contribute to open source projects.
• You will encounter numerous technical challenges that promote professional growth.
• The position allows for collaboration with clients to understand and meet their digital transformation goals.
• You will have the opportunity to work with cutting-edge technologies and tools in data engineering and machine learning.