Remote Sr Big Data Engineer Airflow and Oozie (GCP)
Posted
Apply now
Please, let Rackspace know you found this job
on RemoteYeah.
This helps us grow 🌱.
Description:
We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team.
The ideal candidate will have a strong background in developing batch processing systems.
Extensive experience in Oozie, the Apache Hadoop ecosystem, Airflow, and a solid understanding of public cloud technologies, especially GCP, is required.
This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively.
Key responsibilities include developing scalable and robust code for batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, Map Reduce, Spark (Java), Python, and Hbase.
The candidate will develop, manage, and optimize data workflows using Oozie and Airflow within the Apache Hadoop ecosystem.
Leveraging GCP for scalable big data processing and storage solutions is essential.
Implementing automation/DevOps best practices for CI/CD, IaC, etc., is also part of the role.
Requirements:
A Bachelor's degree in Computer Science, software engineering, or a related field of study is required.
Experience with managed cloud services and understanding of cloud-based batch processing systems are critical.
Proficiency in Oozie, Airflow, Map Reduce, and Java is necessary.
Strong programming skills with Java (specifically Spark), Python, Pig, and SQL are required.
Expertise in public cloud services, particularly in GCP, is essential.
Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, and Map Reduce is needed.
Familiarity with BigTable and Redis is preferred.
Experience in Infrastructure and Applied DevOps principles in daily work is required.
The candidate should utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform to automate and improve development and release processes.
Proven experience in engineering batch processing systems at scale is necessary.
Must have 5+ years of experience in customer-facing software/technology or consulting.
Must have 5+ years of experience with “on-premises to cloud” migrations or IT transformations.
Must have 5+ years of experience building and operating solutions built on GCP (ideally) or AWS/Azure.
Benefits:
The anticipated starting pay range for Colorado is $116,100 - $170,280.
The anticipated starting pay range for Hawaii and New York (not including NYC) is $123,600 - $181,280.
The anticipated starting pay range for California, New York City, and Washington is $135,300 - $198,440.
The role may include variable compensation in the form of bonus, commissions, or other discretionary payments based on company and/or individual performance.
Actual compensation is influenced by a wide array of factors including skill set, level of experience, licenses and certifications, and specific work location.
Rackspace Technology offers a commitment to equal employment opportunity without regard to various legally protected characteristics.
The company is dedicated to embracing technology, empowering customers, and delivering the future.
Apply now
Please, let Rackspace know you found this job
on RemoteYeah
.
This helps us grow 🌱.