Remote Data Engineer

Description:

• Particle41 is seeking a talented and versatile Data Engineer to join their innovative team.
• The Data Engineer will play a key role in designing, building, and maintaining robust data pipelines and infrastructure to support clients' data needs.
• Responsibilities include working on end-to-end data solutions and collaborating with cross-functional teams to ensure high-quality, scalable, and efficient data delivery.
• The role offers an exciting opportunity to contribute to impactful projects, solve complex data challenges, and grow skills in a supportive and dynamic environment.
• Key responsibilities include designing, developing, and maintaining scalable ETL (Extract, Transform, Load) pipelines to process large volumes of data from diverse sources.
• The Data Engineer will build and optimize data storage solutions, such as data lakes and data warehouses, to ensure efficient data retrieval and processing.
• Integration of structured and unstructured data from various internal and external systems to create a unified view for analysis is essential.
• Ensuring data accuracy, consistency, and completeness through rigorous validation, cleansing, and transformation processes is required.
• Comprehensive documentation for data processes, tools, and systems must be maintained while promoting best practices for efficient workflows.
• Collaboration with product managers and other stakeholders to gather requirements and translate them into technical solutions is expected.
• Participation in Agile development processes, including sprint planning, daily stand-ups, and sprint reviews, is necessary.
• The Data Engineer will conduct thorough testing and debugging to ensure the reliability, security, and performance of applications.
• Continuous learning and innovation are encouraged, with a focus on staying updated with the latest technologies and trends in full-stack development.

Requirements:

• A Bachelor's degree in Computer Science, Engineering, or a related field is required.
• Proven experience as a Data Engineer, with a minimum of 3 years of experience is necessary.
• Proficiency in the Python programming language is essential.
• Experience with database technologies such as SQL (e.g., MySQL, PostgreSQL) and NoSQL (e.g., MongoDB) databases is required.
• A strong understanding of programming libraries/frameworks and technologies such as Flask, API frameworks, data warehousing/lakehouse principles, database and ORM, data analysis tools like Databricks, Pandas, Spark, Pyspark, Machine Learning, OpenCV, and scikit-learn is needed.
• Familiarity with utilities and tools such as logging, requests, subprocess, regex, and pytest is important.
• Experience with the ELK stack, Redis, and distributed task queues is preferred.
• A strong understanding of data warehousing/lakehousing principles and concurrent/parallel processing concepts is required.
• Familiarity with at least one cloud data engineering stack (Azure, AWS, or GCP) and the ability to quickly learn and adapt to new ETL/ELT tools across various cloud providers is necessary.
• Familiarity with version control systems like Git and collaborative development workflows is expected.
• Competence in working on Linux OS and creating shell scripts is required.
• A solid understanding of software engineering principles, design patterns, and best practices is essential.
• Excellent problem-solving and analytical skills, with a keen attention to detail, are necessary.
• Effective communication skills, both written and verbal, and the ability to collaborate in a team environment are required.
• Adaptability and willingness to learn new technologies and tools as needed are essential.

Benefits:

• Particle41 values empowering leadership, innovation, teamwork, and excellence, driving everything they do to achieve the ultimate outcomes for clients.
• The company provides equal employment opportunities to all employees and applicants, ensuring that hiring and employment decisions are based on merit and qualifications.
• Individuals from all backgrounds are welcomed, and the company is committed to its mission and values.
• Assistance during the application or interview process is available for candidates who need it.