The role, as published
About the company
Our client is an applied AI research company developing training environments and datasets. Its technical teams build the systems that help AI models learn from complex tasks and data.
The role
Raydar is recruiting for this role on behalf of our client. Own quality across environments and datasets as a Member of Technical Staff, Head of Quality. This is a hands-on evaluation and data-review role with responsibility for automation and a growing quality team.
What you'll do
- Review human data and environment outputs in detail.
- Build automated evaluations, graders, and QA agents.
- Detect reward hacking, broken verifiers, and distribution mismatches.
- Define measurable quality metrics and acceptance criteria.
- Own delivery acceptance for customer projects.
- Hire, onboard, and manage quality engineers and reviewers.
What we're looking for
- 1 to 5 years of relevant LLM evaluation, QA engineering, or ML data-quality experience.
- Experience building automated quality systems for ML data, agents, or large-scale pipelines.
- Technical understanding of what makes training data and environments useful for learning.
- Hands-on work with LLM evaluations, annotation, or human-in-the-loop review.
- Ability to build graders, judge models, telemetry, and dashboards.
- Strong attention to detail and willingness to spend substantial time reviewing data manually.
Bonus points
- RL environment, verifier, or reward-design experience.
- Founder or zero-to-one product-building experience.
- A STEM degree.
Location and work model
- San Francisco, California, United States.
- Fully onsite role; exact weekly office-day count is not specified.
- Visa transfers are supported.
Compensation, as the posting describes it
- Equity: Competitive equity.