Remote Data Engineer (PySpark) - Technology Industry

Description:

• Truelogic is a leading provider of nearshore staff augmentation services headquartered in New York, delivering technology solutions for over two decades.
• The company is seeking a Semi-Senior PySpark Data Engineer who is eager to learn and contribute to the development of high-performance and scalable data pipelines.
• Responsibilities include designing, developing, and optimizing data pipelines using PySpark and Apache Spark, integrating data from multiple sources, implementing efficient data transformations, optimizing code for performance, collaborating with various teams, monitoring and debugging data processes, applying best practices in data engineering, and staying updated with Big Data trends.

Requirements:

• Candidates must have 1-3 years of experience working with PySpark and Apache Spark in Big Data environments.
• Experience with SQL and relational and NoSQL databases such as PostgreSQL, MySQL, and MongoDB is required.
• Knowledge of ETL processes and data processing in distributed environments is necessary.
• Familiarity with Apache Hadoop, Hive, or Delta Lake is preferred.
• Experience with cloud storage solutions like AWS S3, Google Cloud Storage, or Azure Blob is essential.
• Proficiency in Git and version control is required.
• Strong problem-solving skills and a proactive attitude are necessary.
• A passion for learning and continuous improvement is expected.
• Nice to have: Experience with workflow orchestration tools like Airflow or Luigi, familiarity with Data Lakes and Data Warehouses, knowledge of real-time data streaming, experience in performance tuning of Spark jobs, familiarity with Docker or Kubernetes, understanding of data security and governance, and experience with CI/CD and data pipeline automation.

Benefits:

• The position offers 100% remote work, allowing employees to work from any location with a reliable internet connection.
• Highly competitive pay in USD that exceeds typical market offerings is provided.
• Paid time off policies are in place to ensure employees can unwind and recharge.
• Employees enjoy autonomy in managing their time, focusing on results rather than hours worked.
• The opportunity to work with top American companies on innovative, high-impact projects is available.
• The company fosters a culture that values well-being and work-life balance, offering engagement activities and dynamic teams.
• Employees can connect with a diverse, global network of over 600 professionals in 25+ countries.
• Team members will collaborate with skilled professionals, ensuring they work alongside seasoned experts in their field.