Remote Senior Data Engineer

Description:

• Design, develop, and maintain scalable ETL/ELT pipelines and systems for projects of varying sizes and complexities.
• Collaborate with cross-functional teams to understand data requirements and implement effective pipelines.
• Optimize SQL queries and data processing workflows to improve pipeline performance and scalability.
• Develop and optimize data storage and retrieval solutions for structured and unstructured data.
• Apply best practices in data governance, security, and privacy, especially for sensitive government and healthcare data.
• Define and implement data quality checks and governance processes to ensure integrity, accuracy, and consistency.
• Perform data profiling, cleansing, and transformation tasks to support data science and machine learning efforts.
• Monitor and troubleshoot data pipeline performance across different project scales.
• Lead architecture discussions with clients and team members, articulating complex solutions clearly.
• Contribute to business development activities by proposing data engineering solutions.
• Stay updated with the latest data engineering technologies and tools.

Requirements:

• Strong proficiency in Python and SQL, with experience in at least one other programming language.
• Expertise in developing and deploying data pipeline code.
• Experience with big data technologies like Apache Spark, Databricks, Hadoop, or Hive.
• Proficiency in database systems and data warehousing technologies.
• Experience with DevOps practices and tools, especially CI/CD processes.
• Experience with Delta Lake and various file storage formats.
• Experience monitoring and troubleshooting data pipelines.
• Exposure to cloud platforms, particularly Microsoft Azure.
• Adherence to data engineering and security best practices.
• Understanding of distributed computing paradigms and concepts.
• Working knowledge of containerization and orchestration technologies.
• Excellent problem-solving and communication skills.
• Ability to work independently and collaborate with cross-functional teams.
• Bachelor's degree in computer science, information systems, or a related field.
• 5+ years of professional experience in data engineering or related fields.
• Must be a US Citizen and able to obtain a public trust clearance.

Benefits:

• Competitive salary.
• Generous annual leave and paid holidays.
• Comprehensive group health and dental plans.
• 401(k) with company match.
• Life insurance and AD&D coverage.
• Ongoing training and professional development opportunities.