Description:

Design and build data ingestion pipelines into Databricks from various sources (e.g., SAP, web scraping) using PySpark and PySQL.
Conduct scenario-based testing and collaborate with stakeholders to finalize code changes and acceptance criteria.
Understand and implement Databricks Unity Catalog permissions, OBO tokens, and Service Principals for various applications.
Build and deploy Databricks Apps using Visual Studio Code.
Send custom notifications from Databricks via APIs to team channel webhooks.

Requirements:

Proficiency in FastAPI/React for building Single Page Applications (SPAs) with Chat UI interfaces and Claude Models/APIs.
Skills in prompt engineering, including the use of System and User Prompts.
Basic knowledge of Token Economics and rule enforcement in prompts.
Ability to analyze and reengineer code to meet user specifications.
Experience using Graphs for detecting interdependencies and circular references.
Familiarity with Claude Vision API and standard text processing.
Knowledge of Retrieval Augmented Generation techniques specific to Databricks.
Understanding of AI safety guard rails and various caching mechanisms.
Experience with Claude Code and Genie tools.

Skills