Remote Lead Software Engineer II, AI Operations

Description:

• Best Egg is a market-leading, tech-enabled financial platform that helps people build financial confidence through various installment lending solutions and financial health tools.
• The company is looking for a Lead Software Engineer II for AI Operations to design, ship, and operate production-grade LLM applications, agents, and automations across the business.
• The role involves owning the end-to-end path from prototype to stable deployment, including building RAG pipelines, instituting evaluations and guardrails, and driving cost/performance optimization.
• The technology stack includes Python, Metaflow on Outerbounds, AWS (including Bedrock), OpenAI/ChatGPT, and Cursor, with Databricks being evaluated.
• Key responsibilities include building and shipping LLM apps and agents, owning RAG pipelines, designing scalable AWS architectures, adding observability and SRE for AI, enforcing guardrails and governance, managing CI/CD for AI artifacts, optimizing cost and performance, enabling developers, integrating platforms, and maintaining production posture.

Requirements:

• Candidates should have 5–10 years of professional software engineering experience, with at least 2 years focused on building AI/LLM applications and a portfolio of shipped AI projects.
• A demonstrated passion for exploring the latest AI models, frameworks, and tooling is essential.
• Hands-on experience with OpenAI, Bedrock, Huggingface/Ollama/vLLM, and related technologies is required.
• Practical experience in designing and tuning retrieval systems and measuring retrieval quality is necessary.
• Candidates should be comfortable building APIs/services and have strong fundamentals in Python and modern packaging/testing.
• Experience with CI/CD, containers, cloud fundamentals (AWS), and operating services in production is required.
• Familiarity with Metaflow (Outerbounds) and Databricks is preferred, along with the ability to integrate data/feature pipelines.
• Expertise in observability and testing for AI, including tools like Datadog, Dynatrace, or Grafana, is essential.
• A mindset focused on cost, quality, and risk optimization is necessary.
• Strong collaboration and mentorship skills are required to partner effectively with data scientists, analysts, and engineers.
• Bonus points for experience in fine-tuning or distillation, exposure to Kubernetes or FastAPI, and familiarity with Snowflake or similar warehousing.

Benefits:

• The salary for this position ranges from $150,000 to $170,000 per year, with eligibility for an annual incentive bonus based on individual and company performance, targeting 20% of the base salary.
• Best Egg offers a competitive retirement savings plan with pre-tax and post-tax options and a company matching program.
• Employees receive generous paid time-off plans, including vacation, personal/sick time, paid short- and long-term disability leaves, paid parental leave, and paid company holidays.
• Multiple health care plans are available, including dental and vision options.
• Flexible Spending Plans for Health Care, Dependent Care, and Health Reimbursement Accounts are offered.
• Company-paid benefits include life insurance, wellness platforms, employee assistance programs, and Health Advocate programs.
• Additional discounted benefits include identity theft protection, pet insurance, fitness center reimbursements, and more.