Remote Prompt Engineer (LLM Systems, Evals & Safety)

Description:

• The position is for a Prompt Engineer specializing in LLM Systems, Evals, and Safety at webook.com, Saudi Arabia's leading event ticketing and experience booking platform.
• The role involves designing high-quality prompts, system instructions, and tooling to ensure LLM features are accurate, safe, and cost-effective.
• Responsibilities include authoring, refactoring, and chaining prompts for various tasks, creating evaluation harnesses, and building prompt libraries with versioning and A/B testing.
• The engineer will work on reducing hallucinations through verification and implementing safety measures such as jailbreak tests and content policy checks.
• Collaboration with engineers to integrate prompts into production features is essential.
• The role requires demonstrated prompt design experience across multiple task types and models, as well as experience in building evaluation datasets and automated scoring.
• Familiarity with retrieval-augmented generation concepts and strong scripting skills in Python or TypeScript for data preparation and analysis are necessary.
• Clear writing skills are required to translate business goals into measurable prompt specifications.

Requirements:

• Candidates must have experience in authoring, refactoring, and chaining prompts for varied tasks.
• A strong background in creating offline and online evaluation harnesses, including rubrics and metrics, is required.
• Experience in building prompt libraries with versioning, A/B testing, and telemetry is essential.
• Candidates should have skills in reducing hallucinations through verification and constrained decoding.
• Knowledge of implementing safety measures, including jailbreak/prompt-injection tests and content policy checks, is necessary.
• The ability to partner with engineers for prompt integration into production features is required.
• Demonstrated experience in prompt design across multiple task types and models is a must.
• Candidates should have experience in building evaluation datasets and automated scoring metrics.
• Familiarity with retrieval-augmented generation concepts and strong scripting skills in Python or TypeScript for data preparation and analysis are required.
• Clear writing skills to translate business goals into measurable prompt specifications are essential.

Benefits:

• Working at webook.com offers the opportunity to make a significant impact and transform people's lives through innovative technology.
• Employees will be part of a dynamic team that values disruption of the normal and average.
• The position provides a chance to work on high-profile projects, serving some of the largest mega events in Saudi Arabia.
• The role offers a collaborative environment with engineers and other professionals in the field.
• Employees can expect to engage in continuous learning and improvement in a rapidly evolving industry.