Welcome to RemoteYeah 2.0! Find out more about the new version here.

Remote Consultant Data Engineer (Python/Databricks)

at Softermii

Posted 21 hours ago | 2 applied

Description:

  • Softermii is an outsourcing/outstaffing company with over 8 years of experience in web and mobile application development, employing more than 120 specialists from 6 countries.
  • The company has successfully navigated challenges such as the COVID-19 pandemic and war aggression against Ukraine, ensuring sustainable development and growth.
  • Softermii is expanding its portfolio in various sectors including Healthcare, Real Estate, Livestream E-commerce, Fintech, and Real Time Communications.
  • The company is developing two proprietary solutions for video/audio calls and real estate applications with 3D visualization.
  • Softermii is seeking a part-time Data Engineering Consultant / Tech Lead to conduct technical interviews, assist with projects, and engage in complex development tasks, particularly in data pipeline design and optimization on Databricks.
  • Responsibilities include interviewing and hiring Data Engineers, supervising their work, developing scalable data pipelines using Databricks, collaborating with data scientists and analysts, and ensuring data quality and governance.

Requirements:

  • Candidates must have 5+ years of experience in data engineering or big data development with production-level work.
  • They should be able to architect and develop scalable data solutions on the Databricks platform, utilizing Apache Spark, Delta Lake, and lakehouse architecture.
  • Experience in designing, building, and maintaining production-grade data pipelines using Python (or Scala) and SQL is required.
  • Candidates must lead the implementation of Databricks features such as Delta Live Tables, Unity Catalog, and Workflows.
  • They should optimize Spark performance and resource utilization, and integrate data from cloud-based sources like AWS S3.
  • Experience in managing orchestration and automation of data workflows using tools like Airflow or Databricks Jobs is necessary.
  • Candidates must collaborate with cross-functional teams and mentor junior engineers while ensuring adherence to data governance and security policies.
  • Continuous evaluation of new technologies and practices is expected, along with experience in designing and maintaining data pipelines using Apache Airflow.
  • An upper-intermediate level of English is required.

Benefits:

  • The company offers stable and efficient processes with clear roles and responsibilities, allowing for quick decision-making.
  • Employees have the independence to make impactful decisions on projects and company operations.
  • Softermii fosters a collaborative environment with a team of like-minded experts who enjoy both work and leisure together.
  • The company supports professional development by covering 50% of the costs for courses, conferences, or speaking clubs.
  • An individual development plan will be created for each employee, with mentoring from seniors and leaders.
  • A referral bonus is available for recommending friends who are looking for job opportunities.
  • Employees receive 21 working days off and 5 sick leaves per year.
  • Opportunities for volunteering are provided, with support for various initiatives.
  • The company has a flexible working schedule, allowing employees to choose between remote or office work.