Description:

Build infrastructure for evaluating coding agents and language models, including benchmark datasets and automated task quality checks.
Own features across the stack, from data models to user interfaces, ensuring reliability and performance.

Requirements:

5+ years of experience in production backend systems, with ownership of at least one service.
Strong proficiency in Python, experience with Django, PostgreSQL, and task queues like Celery.
Hands-on experience in LLM evaluation and understanding of benchmark task construction and scoring.
Ability to assess task quality and make sound design tradeoffs.
Ownership mentality, capable of independently completing tasks from request to deployment.
Familiarity with TypeScript and React for frontend changes.
Clear communication skills in writing pull request descriptions and design notes.

Skills