Remote Sr. Data Engineer

Description:

• Abacus Insights is a mission-driven, growth-stage technology company focused on transforming the healthcare industry.
• The company aims to create a more personalized patient experience, improve health outcomes, and lower the overall cost of healthcare.
• Abacus provides a flexible, efficient, and secure platform that organizes and exchanges healthcare data from various sources and formats.
• The platform allows customers to uncover differentiated insights that address their client's needs.
• Employees play an active role in keeping customer data safe and ensuring comprehensive policies and practices are met.
• The company has built an innovative data integration and management platform that allows healthcare payers access to historically siloed and inaccessible data.
• The role of Sr. Data Engineer involves collaborating with customers, data vendors, and internal teams to integrate data into a state-of-the-art data platform.
• The platform is constructed on highly scalable, flexible, and resilient cloud architectures.
• The Sr. Data Engineer will facilitate the successful adoption of Abacus’s core data management solutions and ensure seamless data ingestion and management.

Requirements:

• A Bachelor’s Degree in computer science, computer engineering, or any related IT field is required.
• Experience in communicating with both technical and non-technical stakeholders is necessary.
• Proficiency in Python and SQL is essential.
• The candidate must be able to build and manage efficient ETL pipelines using Databricks workflows or other orchestration frameworks.
• Familiarity with both structured and semi-structured data and the ability to ingest and process this data using pyspark is required.
• Fundamental knowledge of AWS services (S3, SQS) or similar services for other clouds is needed.
• Experience with Terraform and GitLab CICD is necessary.
• The candidate should have skills in query tuning and performance optimization in SQL and/or SparkSQL.
• Familiarity with data warehousing solutions like Snowflake or similar is required.

Benefits:

• Abacus Insights offers a dynamic and rapidly expanding work environment.
• Employees have the opportunity to contribute to the success of a mission-driven technology company.
• The company promotes a culture of diversity and provides equal employment opportunities to all applicants.
• Employees are encouraged to stay updated on industry trends and receive mentoring opportunities.
• The role includes conducting training for team members and clients to enhance understanding and utilization of data solutions.