← Back to job postings
V

LLM Evaluation Lead

Vellum Labs · San Francisco · Full⁠-⁠time · $190k to $240k

The role

You will own data and ml for a small team shipping AI features to real customers.

  • Ship work that reaches customers within your first month.
  • Work directly with the founders and the product lead.
  • Shape the practices and tooling the team grows into.

What we look for

Judgement and shipped work matter more than years on paper.

  • A track record of shipping in a fast-moving SaaS team.
  • Clear written communication with technical and non-technical people.
  • Curiosity about how AI products behave in production.

Hiring process

Three steps, usually finished inside two weeks.

  • Intro call, 30 minutes.
  • Work sample review with the team, 60 minutes.
  • Final conversation and offer.
LLM evaluationPythonStatistics