Senior Product Manager, Multilingual AI and Evals
You will own quality assurance and evaluation infrastructure for multilingual AI agents. You will improve automated translation quality gates, build AI-driven localization testing, create datasets and annotation tools, establish evaluation metrics and dashboards, and turn human feedback into agent improvements across global products.
Responsibilities
- Enhance the evaluation agent that determines whether translations are ready for publication
- Balance automation coverage and risk across content tiers
- Diagnose systematic quality-gate failures and drive fixes into the agent
- Build an AI-driven tool that detects localization defects
- Ship an MVP that scans mobile and web applications for localization issues
- Integrate localization tests into internal product testing infrastructure
- Build golden datasets, annotation tools, automated evaluations, metrics dashboards, and feedback loops
- Integrate standardized real-time annotation workflows for linguists
Requirements
- Bachelor's degree or higher
- 3+ years in product management, with hands-on AI agent experience potentially substituting for part of the requirement
- Experience building content platforms, testing platforms, or SaaS products
- Hands-on experience with AI agents, agent harnesses, and evaluations
- Ability to design evaluation harnesses and golden datasets
- Experience shipping products across multiple markets or languages
- Cross-functional leadership across engineering, AI teams, linguists, design, and external partners
Benefits
- Education subsidy
- Team building programs
- Company events
- Wellness allowance
- Meal allowance
- Comprehensive healthcare schemes for employees and dependants