Remote Data Engineer (data warehouse, Azure Data Factory, SQL)

Description:

• DMV IT Service LLC is seeking a skilled Data Engineer with extensive experience in ETL processes and data warehousing, particularly within the healthcare sector.
• The successful candidate will be responsible for developing and managing ETL pipelines, primarily using Azure Data Factory and Snowflake.
• The role involves ensuring efficient data transformation and storage that meets the needs of end users.
• The Data Engineer will develop and manage effective working relationships with other departments to coordinate ETL processes.
• Communication with ETL architects to understand requirements and transform data accordingly is essential.
• The candidate will assist in the overall architecture of the ETL design, providing proactive inputs and implementing automated ETL flows.
• Investigating data issues within ETL pipelines, notifying end-users, and proposing adequate solutions is a key responsibility.
• The role includes developing idempotent ETL processes using Azure Data Factory dataflows and pipelines.
• The Data Engineer will work within Snowflake Virtual Warehouses and automate data pipelines using Snowpipe.
• Capturing changes in data dimensions and maintaining versions using Stream sets in Snowflake is required.
• The candidate will optimize data movement to enhance performance and build a highly efficient orchestrator for scheduling jobs and executing workflows.
• Testing ETL systems, performing root cause analysis, resolving production issues, and conducting routine database and data flow testing are part of the job.
• Documentation of implementations, test cases, and build deployment documents for CI/CD is necessary.

Requirements:

• The candidate must have 5+ years of Data engineering experience with a focus on Data Warehousing.
• Extensive experience with ETL tools like Informatica PowerCenter, SSIS, and Azure Data Factory is required.
• Strong proficiency in SQL with various databases such as Oracle, Snowflake, and SQL Server is essential.
• Familiarity with version control systems like GitHub or SVN is necessary.
• Experience with healthcare data standards such as HL7 and FHIR is required.
• Preferred skills include batch or PowerShell scripting experience and Python scripting capabilities.
• Data modeling experience in a data warehouse environment is preferred.
• Familiarity with Informatica Intelligent Cloud Services and API design in Snowflake and ADF is a plus.
• Azure certifications in data engineering or analytics are preferred.

Benefits:

• The position offers a full-time employment opportunity with a remote work option.
• Employees will have the chance to work with a trusted IT consulting firm that specializes in optimizing IT infrastructure.
• The role provides opportunities for professional growth and development in the field of data engineering.
• Employees will be part of a client-focused approach that emphasizes long-term IT success through training and job placements.