Remote Lead QA Engineer (AI Voice)

Description:

• Design the voice AI test strategy and process, covering speech input, recognition, LLM, tools, and output.
• Define automation approach for multilingual coverage (around 100 languages).
• Test predictable and non-deterministic AI behavior.
• Evaluate prompts and LLM responses using test datasets.
• Lead UAT planning, scenarios, test data, and coordination of business testers.
• Gather business acceptance criteria and convert them into measurable targets.
• Manage defects and risks, providing release go/no-go recommendations.
• Communicate options when development is delayed and mentor junior engineers.

Requirements:

• Minimum 6 years of software QA experience.
• At least 2 years in testing voice agents or conversational AI.
• Experience leading E2E and UAT with limited guidance.
• Familiarity with international or distributed teams.
• Knowledge of voice AI testing metrics and quality metrics.
• Understanding of LLM functionality and failures.
• Proficiency in test automation using Python or JavaScript/TypeScript.
• Experience with cloud speech services (Azure, Google, AWS).
• Familiarity with Jira or Azure DevOps and test management tools.
• Experience troubleshooting using logs and transcripts.
• Knowledge of voice testing platforms and LLM evaluation tools is a plus.
• Understanding of telephony basics and Responsible-AI testing is a plus.
• Strong communication skills with technical and business stakeholders.
• English proficiency at least B2.

Benefits:

• Stable employment with a company established since 2008.
• Flexible work model: remote or office options available.
• Workation policy for working from inspiring locations.
• Great Place to Work® certified employer.
• Flexible working hours and contract options.
• Comprehensive online onboarding with a Buddy.
• Collaboration with top-tier engineers and experts.
• Unlimited access to Udemy learning platform from day 1.
• Opportunities for certificate training and upskilling support.
• Internal promotion opportunities (76% of managers promoted internally).
• Diverse and inclusive community with autonomy in work methods.
• Community-building activities and health support initiatives.
• Opportunities to donate to charities and support environmental causes.
• Modern office equipment provided.