Senior Web Scraping | Python Developer | Automation Specialist

• Developed Python ETL pipelines and parsers to ingest vendor catalogs from emails, spreadsheets, and scraped feeds, normalizing pricing, SKUs, and product data into the platform catalog. • Partnered with operations and vendor IT teams to onboard new distributors, configure account credentials, and validate production order submissions. • Built and maintained core backend features for order guides, restaurant–vendor associations, and admin tooling using a PHP/MySQL/MongoDB stack. • Designed and implemented EDI integrations with major foodservice distributors (e.g. Sysco, US Foods, PFG, GFS, Shamrock, Ben E. Keith), generating X12/CSV/TXT order files and delivering them through FTP/SFTP. • Extended inbound EDI workflows for order acknowledgments and invoices, improving order visibility and reducing manual reconciliation efforts.

• Built real-time observability dashboards in Metabase tracking coverage rates, ingestion speed, deduplication accuracy, and scraper health indicators, enabling fast issue detection and data-driven decisions on pipeline prioritisation. • Integrated Sentry for advanced error tracking and alerting across the scraping infrastructure, capturing failure patterns per source and significantly reducing mean time to resolution for pipeline and processing issues. • Streamlined deployment and operations using GitHub Actions for CI/CD, Docker for containerisation, and Dozzle for centralised real-time log streaming, enabling reliable, repeatable production releases with minimal manual intervention. • Engineered high-performance automation pipelines with multithreading, adaptive rate limiting, and automated retry logic, processing large volumes of listings continuously while maintaining data freshness. • Architected a large-scale web scraping and automation platform covering 40+ job platforms, implementing purposebuilt scrapers using Selenium, Playwright, Scrapy, and Requests matched to each target site's structure, including dynamic rendering, authentication flows, and anti-bot measures.

• Developed web automation scripts for autofilling and submitting job applications, handling CAPTCHA, Cloudflare, and Akamai challenges through adaptive browser behavior and smart retries. • Built high-performance Python crawlers and automation bots using Playwright, Selenium, and Requests, implementing multi-threading, async execution, and proxy rotation for stable, high-throughput scraping. • Integrated backend modules to generate enriched, standardized, and AI-optimized job descriptions, improving data quality and automation coverage. • Designed and maintained a large-scale Job Application Tracking System (ATS) handling over 35M job listings and automating data collection across 40+ job platforms. • Implemented robust data pipelines and scheduling systems for continuous scraping, data refresh, and validation, ensuring consistent uptime and reliable updates. • Optimized MongoDB and PostgreSQL workflows for performance and scalability, improving query efficiency and reducing data duplication and latency.

• Assisted with automation workflows by creating scripts and API integrations to streamline data processing, improve system efficiency, and reduce manual operations. • Developed and maintained software applications by writing clean, efficient code, fixing bugs, and delivering feature enhancements across development and production environments. • Supported cloud-based applications and infrastructure by monitoring services, troubleshooting technical issues, and collaborating with senior engineers to improve system reliability. • Worked with databases, backend services, and application servers to optimize performance, resolve production issues, and ensure stable application operations. • Participated in Agile development processes, including code reviews, testing, deployments, and debugging, while following software development best practices and version control workflows.