Head of ML Infra

Responsibilities:

 Within one month, you will…

• Meet your team, which consists initially of 3 to 5 ML engineers (2 already on the team)

• Start building a trusting relationship with your direct reports and peers.

• Come up to speed on the current state of machine learning in the Cody ecosystem.

• Be set up for local development and familiar with Cody’s architecture.

• Define our short-term roadmap for ML Infrastructure on GCP.

• Ship a substantial feature, experiment, or evaluation.

Within three months, you will…

• Set up the at-scale infrastructure for running benchmarks that compare coding assistants.

• Have defined a strategy for how we will address getting GPUs at scale for various personas.

• Have defined a rough roadmap for how to cost-optimize our ML spend.

• Have defined our on-prem/self-hosted roadmap and recommended configurations for ML infra.

• Be up to speed and driving Sourcegraph’s ML Infra strategy.

Within six months, you will…

• Have hired a world-class team of ML engineers.

• With the help of our research team, have delivered a  ML-driven quality, benchmarking, and evaluation framework for coding assistants that runs at scale

• Have established a longer-term roadmap that keeps us aligned with expected advances in LLMs.

• Be running dozens to hundreds of experiments with prompting, embedding, fine-tuning and other techniques.

About you 

You have been working squarely in ML Infra since LLMs landed, if not longer.

• You’re deeply familiar with at least one end-to-end system for ML pipelines at scale, and you are broadly familiar with the competition in the space and what options are available, and when.
• In an ideal world, you are most deeply familiar with GCP’s machine learning stack, and you have a lot of practice operationalizing PyTorch experiments on that stack. It’s also great if you have Apache Spark in general.
• You should be the kind of person who lives and breathes GPUs, and you should come armed with opinions about how best to deploy and cost-optimize Cody for our various customer classes, from large enterprises to casual hackers, particularly when it comes to the Cloud-side deployments.
• In a perfect world, you would already be comfortable with options that enterprise customers might want for self-hosted ML infra, for running their own pipelines, e.g. other Cloud-hosted offerings, and/or OSS. Although we are pushing hard to have everything on GCP, the market is evolving rapidly and we could, for instance, come across customers who want to provide their own GPUs.
• Any familiarity you have with deploying enterprise SaaS is a huge bonus because it is a part of the role. However, it’s something that you can pick up if you are already familiar with Cloud options.
• Bonus if you have any background in graph theory or anything that would be relevant to our code graph, which plays a key role in the production of both training data and in acting as a source of truth for verifying model outputs.
• We would love it if you are actively following developments in open-source models and training systems, and can come prepared with opinions about when and to what extent we should adopt them. Or more importantly, how we set up infrastructure that will tell us when they are ready, by evaluating their performance on Cody tasks.