Remote Sr Big Data Engineer Airflow and Oozie (GCP)

Posted

Apply now
Please, let Rackspace know you found this job on RemoteYeah. This helps us grow 🌱.

Description:

  • We are seeking a highly skilled and experienced Senior Big Data Engineer to join our dynamic team.
  • The ideal candidate will have a strong background in developing batch processing systems.
  • Extensive experience in Oozie, the Apache Hadoop ecosystem, Airflow, and a solid understanding of public cloud technologies, especially GCP, is required.
  • This role involves working in a remote environment, requiring excellent communication skills and the ability to solve complex problems independently and creatively.
  • Key responsibilities include developing scalable and robust code for batch processing systems using technologies like Hadoop, Oozie, Pig, Hive, Map Reduce, Spark (Java), Python, and Hbase.
  • The candidate will develop, manage, and optimize data workflows using Oozie and Airflow within the Apache Hadoop ecosystem.
  • Leveraging GCP for scalable big data processing and storage solutions is essential.
  • Implementing automation/DevOps best practices for CI/CD, IaC, etc., is also part of the role.

Requirements:

  • A Bachelor's degree in Computer Science, software engineering, or a related field of study is required.
  • Experience with managed cloud services and understanding of cloud-based batch processing systems are critical.
  • Proficiency in Oozie, Airflow, Map Reduce, and Java is necessary.
  • Strong programming skills with Java (specifically Spark), Python, Pig, and SQL are required.
  • Expertise in public cloud services, particularly in GCP, is essential.
  • Proficiency in the Apache Hadoop ecosystem with Oozie, Pig, Hive, and Map Reduce is needed.
  • Familiarity with BigTable and Redis is preferred.
  • Experience in Infrastructure and Applied DevOps principles in daily work is required.
  • The candidate should utilize tools for continuous integration and continuous deployment (CI/CD), and Infrastructure as Code (IaC) like Terraform to automate and improve development and release processes.
  • Proven experience in engineering batch processing systems at scale is necessary.
  • Must have 5+ years of experience in customer-facing software/technology or consulting.
  • Must have 5+ years of experience with “on-premises to cloud” migrations or IT transformations.
  • Must have 5+ years of experience building and operating solutions built on GCP (ideally) or AWS/Azure.

Benefits:

  • The anticipated starting pay range for Colorado is $116,100 - $170,280.
  • The anticipated starting pay range for Hawaii and New York (not including NYC) is $123,600 - $181,280.
  • The anticipated starting pay range for California, New York City, and Washington is $135,300 - $198,440.
  • The role may include variable compensation in the form of bonus, commissions, or other discretionary payments based on company and/or individual performance.
  • Actual compensation is influenced by a wide array of factors including skill set, level of experience, licenses and certifications, and specific work location.
  • Rackspace Technology offers a commitment to equal employment opportunity without regard to various legally protected characteristics.
  • The company is dedicated to embracing technology, empowering customers, and delivering the future.
Apply now
Please, let Rackspace know you found this job on RemoteYeah . This helps us grow 🌱.
About the job
Posted on
Job type
Salary
$ 116,100 - 198,440 USD / year
Report this job

Job expired or something else is wrong with this job?

Report this job
Leave a feedback