Remote Data Engineer / Cloud Consultant

Description:

• Pythian is seeking a Data Engineer / Cloud Consultant to work remotely in Canada or the USA.
• The role involves designing and creating impactful software for enterprise data platform solutions focused on Cloud Platforms.
• Responsibilities include the design and development of end-to-end cloud-based solutions, translating complex requirements into detailed designs, and writing high-performance, reliable, and maintainable code.
• The position requires developing test automation and tooling, working on complex cloud-based projects, and maintaining DataOps tasks such as performance optimization and troubleshooting.
• The Data Engineer will perform DevOps activities related to data pipelines, analyze potential issues, and follow up on fixes.
• Additional tasks include reviewing bug descriptions, performance tuning for data processing, securing cloud data platform components, and developing data pipelines for ingestion, transformation, and cleansing.
• The role also involves integration with external systems and data access tools, foundational CI/CD for infrastructure components, and assisting client application developers with data access.

Requirements:

• Proficiency in a programming language such as Python, Java, Go, or Scala is required.
• Experience with big data cloud technologies like EMR, Athena, Glue, Big Query, Dataproc, and Dataflow is necessary.
• Strong hands-on experience with Google Cloud Platform data technologies, particularly Google BigQuery and Google DataFlow, is ideal.
• Understanding of Spark fundamentals, including PySpark or SparkSQL, and experience in developing automated and reliable data pipelines is essential.
• Strong SQL skills and knowledge of popular database and data warehouse technologies from Google, Amazon, or Microsoft are required.
• Familiarity with Data Orchestration solutions like Airflow, Oozie, or Luigi, and knowledge of DBT or DataForm is necessary.
• Experience with Apache Iceberg, Hudi, and query engines like Presto is a plus.
• Knowledge of Data Catalogs, Data Governance, and Data Quality Solutions is an added advantage.
• Experience in software engineering best practices, data modeling, and performing DevOps activities using IaC is required.
• Good to have knowledge of GenAI tools and frameworks, including proficiency in prompt engineering.

Benefits:

• Pythian offers a competitive total rewards package, including a substantial training allowance for skill development and professional growth.
• Employees enjoy flexible remote work options with no daily travel requirements.
• The company fosters a collaborative environment with talented coworkers in the industry.
• Pythian provides all necessary equipment for remote work, including a laptop and an annual budget for personal workspace customization.
• An annual wellness budget is provided to support health and well-being, along with generous paid vacation and sick days, and a day off to volunteer for a charity.