Description:

As a Senior Backend Software Engineer on the LLM Services team, you will integrate AI into healthcare delivery.
Your responsibilities include designing and implementing Python-based services that leverage Large Language Models to enhance the core healthcare platform.
You will architect and build scalable micro-services that power LLM-enabled features.
Collaboration with Software, ML, and Infrastructure engineers is essential to optimize LLM integration and deployment.
You will design and implement LLM orchestration services, including prompt management, model switching, and response streaming.
Building robust evaluation pipelines to measure and improve LLM output quality and consistency is part of your role.
You will develop scalable APIs for LLM-powered features, including context injection and retrieval augmentation.
Implementing efficient caching and optimization strategies for LLM inference is required.
You will create monitoring systems for tracking token usage, latency, and other LLM-specific metrics.
Working on prompt engineering and chain-of-thought implementations is expected.
Ensuring compliance with healthcare regulations while working with LLM outputs is crucial.
You will participate in technical design discussions and code reviews.
Monitoring and maintaining services in production, including LLM Observability using tools like LangFuse, is part of your duties.

Requirements:

Strong proficiency in Python and experience building production-grade backend services are required.
You must have an understanding of prompt engineering principles and LLM evaluation metrics.
Experience with vector databases (like Pinecone, Weaviate, or similar) for semantic search is necessary.
Familiarity with streaming architectures for real-time LLM responses is expected.
You should have experience implementing rate limiting and failover strategies for API services.
A strong grasp of software engineering best practices, including testing, documentation, and version control, is essential.
Excellent problem-solving skills and attention to detail are required.
Strong written and verbal communication skills are necessary.

This is an all-remote position, allowing for flexibility in your work environment.
Flexible working hours are offered to accommodate different schedules.
You may have occasional late meetings to collaborate with US-based team members.