Azure Data Engineer

• Engineered end-to-end migration of multiple complex data workflows from RapidMiner to Azure Data Factory, ensuring 100% functional parity with legacy systems. • Designed and developed scalable ADF data pipelines integrating multiple structured data sources. • Rebuilt and optimized transformation logic using Databricks and PySpark, improving scalability and maintainability. • Performed rigorous data validation and reconciliation, achieving 80–90% data accuracy and consistency across migrated datasets. • Monitored production pipelines, proactively troubleshooting issues and reducing job failure rates by ~30%. • Collaborated with business stakeholders and technical teams to validate outputs and ensure smooth cloud transition. • Created technical documentation including design specifications and data flow diagrams.

• Designed and implemented high-volume Spark & Databricks ETL pipelines processing 5+ TB of data daily. • Developed and maintained optimized data models to support analytics and BI reporting. • Implemented data quality checks and validation rules, improving data reliability for downstream consumers. • Built a POC using OpenAI APIs to enrich, cleanse, and correct company datasets, improving data completeness. • Worked closely with Data Analysts, BI Developers, Architects, and QA teams in Agile delivery models. • Supported production deployments, monitored pipeline performance, and participated in issue resolution. • Documented transformation logic, best practices, and operational runbooks in project wiki.