About
A model release decision is only as good as the numbers behind it. This team owns the framework that produces them: orchestration, runtimes and observability for every benchmark, across web apps, desktop applications and the command line. It runs around 50 benchmarks today and needs to handle 100-200 soon without getting slower or more expensive.
This is a well-funded frontier AI company building computer-use agents and the models behind them. Developers use them through a managed API, and a forward-deployed team takes them into enterprise workflows.
Researchers are the biggest users. They pick checkpoints and decide what ships based on your results. The rest varies week to week: cluster tuning one week, a browser extension or desktop environment the next.
What you'll do
- Own a chunk of the framework – scaling, scheduling or observability – so the cluster isn't sitting idle while jobs queue
- Set the bar for how new benchmarks get added, and build the checks that enforce it
- Make it quicker for researchers and forward-deployed engineers to plug in new benchmarks
- Make results reproducible, so a release call rests on numbers that hold up
- Talk to customers about what they want measured, then automate it into the harnesses
What you'll need
- 5+ years of backend work with production Python at the core. Backend-leaning full-stack is fine
- You've built AI-adjacent tooling, platforms or infra that other people relied on.
- Strong familiarity with Kubernetes - at least at the application layer, even better if you have run a cluster
- Shipped things end to end: APIs, integrations, databases, queues (SQS, RabbitMQ or Kafka)
- You instrument as you build: metrics, tracing, monitoring
- You use coding agents to go faster without letting quality slip
Optional Bonus
- Evals experience – a real plus, not a requirement
- Playwright, Selenium, or a browser extension you wrote
- Temporal, FastAPI, Postgres, Grafana or Datadog
Shortlisted candidates will be contacted within 48 hours.
