Remote Senior Data Engineer (Python, Databricks)

Description:

• Capco Poland is seeking a Senior Data Engineer based in Poland for a remote position.
• The role involves designing, developing, and implementing robust data architecture solutions using modern data platforms like Databricks.
• Responsibilities include ensuring scalable, reliable, and secure data environments that meet business requirements and support advanced analytics.
• The candidate will lead the migration of data from traditional RDBMS systems to Databricks environments.
• The role requires architecting and designing scalable data pipelines and infrastructure to support the organization's data needs.
• The Senior Data Engineer will develop and manage ETL processes using Databricks for efficient data extraction, transformation, and loading.
• Optimizing ETL workflows to enhance performance and maintain data integrity is essential.
• The candidate will monitor and optimize the performance of data systems to ensure reliability, scalability, and cost-effectiveness within Databricks environments.
• Collaboration with cross-functional teams, including data engineers, data scientists, analysts, and product managers, is necessary to understand data requirements and deliver solutions.
• The role includes defining best practices and standards for data engineering/data warehouse processes and ensuring adherence to them.
• Evaluating and implementing new technologies and tools to improve the efficiency and effectiveness of data pipelines is part of the job.
• The candidate will provide technical leadership and mentorship to junior members of the data engineering team.
• Close collaboration with DevOps and infrastructure teams to deploy and manage data systems in production environments is required.
• Ensuring compliance of all data solutions with organizational policies, industry standards, and regulatory requirements is crucial.
• The Senior Data Engineer will work with enterprise architects and IT leadership to align data architecture with overall IT architecture and strategies.

Requirements:

• Extensive experience with Databricks, including ETL processes and data migration is required.
• Experience with additional cloud platforms like AWS, Azure, or GCP is necessary.
• Knowledge of big data technologies and frameworks is essential.
• Strong knowledge of data warehousing concepts, data modeling, and SQL is required.
• Experience with RDBMS systems and transitioning data to modern cloud platforms is necessary.
• Proficiency in programming languages such as Python, SQL, and scripting languages is required.
• Knowledge of data governance frameworks, data quality management practices, and data security principles is essential.
• Proficiency in database technologies, including relational databases like SQL Server, and in-depth knowledge of data modeling and database design is required.
• Familiarity with containerization technologies such as Docker and orchestration tools like Kubernetes is necessary.
• Excellent problem-solving and analytical skills are required.
• Strong communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams, are essential.
• A Bachelor or Master Degree in Computer Science or a related field is required.
• Certifications in Databricks are a nice-to-have.

Benefits:

• The position offers an employment contract and/or Business to Business arrangement, depending on preference.
• There is a possibility to work remotely.
• Daily communication in English, mainly with foreign stakeholders and peers, is part of the role.
• Multiple employee benefits packages are available, including MyBenefit Cafeteria, private medical care, and life insurance.
• Access to a platform with over 3,000 business courses (Udemy) is provided.
• Required IT equipment will be supplied.
• A paid referral program is available.
• Participation in charity events, such as Szlachetna Paczka, is encouraged.
• Ongoing learning opportunities are offered to help acquire new skills or deepen existing expertise.
• The candidate will be part of the core squad focused on the growth of the Polish business unit.
• A flat, non-hierarchical structure allows for working with senior partners and directly with clients.
• The work culture emphasizes innovation and creating lasting value for clients and employees.