Inside OpenAI’s Forward Deployed Engineer Role
Insights based on my conversation with the Global Head of Forward Deployed Engineering at OpenAI. What does it take to be a great forward deployed engineer at OpenAI?
Intro
The Forward Deployed Engineer (FDE) role has gained a lot of popularity in recent years, and OpenAI is a good example of fast scaling of their FDE organization.
Additionally, they recently built a separate entity called the OpenAI Deployment Company, focused on helping customers use OpenAI’s products.
I had the pleasure of talking with Colin Jarvis, Global Head of Forward Deployed Engineering at OpenAI.
We talked all about how their team works, what makes a great FDE, how they work with customers, and how engineers can prepare for the role.
This is an article for paid subscribers, and here is the full index:
- How OpenAI built its forward deployed engineering organization
- Forward deployed engineer role at OpenAI
- What does an FDE do on a daily basis?
- About 50% of their time is spent on the customer site
🔒 The hiring process for a new FDE
🔒 Onboarding process for a new FDE
🔒 What makes an FDE successful
🔒 Moving from a software engineer to FDE, how to prepare?
🔒 Last words
Let’s start!
How OpenAI built its forward deployed engineering organization
OpenAI started the forward deployed engineering organization last year, with a focus on gathering insights from customers, learning what the use cases are, and bringing the insights back to product and research.
They started with just 2 people and have now grown to around 140 people in the FDE org. The FDE org sits between the GTM (go-to-market) org and product and research, working with both of them, acting as a “link”.
After some time working with different customers, they have noticed repeatable problems that people and companies are dealing with, and that’s also the reason why they recently started OpenAI Deployment Company (which is a separate entity, but still part of OpenAI).
It has now grown to about 200 people, and I suspect it’s going to grow a lot more in the future. Its goal is entirely to help solve customer problems, which is at the core of what FDE is all about.
Colin mentioned that they intentionally want to keep the internal FDE org at OpenAI relatively small and focused on getting insights for the product and research, but they want to keep expanding the OpenAI Deployment Company in order to support the customers in need of integrating their products.
Now, let’s get more into the FDE role in general.
Forward deployed engineer role at OpenAI
When I asked Colin on how he’d define the FDE role at OpenAI, he mentioned the following:
A forward deployed engineer is focused on outcomes. Which means that their goal should be to work on problems where solving them has a much bigger impact than the work itself.
So, in practical terms, the value is not in writing code, instead, the focus is to solve the problem and, ideally, make themselves obsolete as well, so they are not needed anymore.
And if they are needed again (for the same problem), that usually means that the problem has not been solved entirely.
They want strong software engineers, but at the same time also good communicators, who can really dig into what the actual problem is and explain the solution in a way that resonates with everyone.
And (ideally), these engineers would also have some domain expertise in the field that the specific customer operates in.
For example, a software engineer who has worked in the fintech industry could be a great fit for an FDE helping out a customer who works in fintech, because they understand the domain.
However, an important thing that Colin mentioned is that they want to mix the domain experts and FDEs, who are not. And the reason is that a person who is not an expert in the specific industry, but has experience from another, is going to be much more likely to “challenge the status quo” and have unique ideas to solve certain problems.
What does an FDE do on a daily basis?
Colin shared an example of a project they are working on at the moment in Japan, specifically for “code modernization”. That specific customer has a lot of legacy applications, and migrating one of these systems can cost hundreds of millions of dollars.
They are helping that customer with the migration from older tech to the new tech using AI.
An FDE might spend the full day talking with the payments team to understand what specific things are important to them so that the migration would be successful.
And then the next day, start building tests based on the insights shared from the payments team in order to ensure that the migration is actually successful.
And then another day they might be writing actual migration code, or building the UI to show the progress of the migration, and/or specific steps of it.
So the role is really a mix of engineering and working directly with domain experts (and also having a good understanding of that field), going back and forth between the two.
There’s also a feedback loop with the product and research teams that’s important. And in this specific project, they found out that a specific model used for the migration kept making some simple mistakes.
And as Colin mentioned, they don’t just send the exact customer problem back to the research team. They try to be as specific as possible, for example: “The model struggles with these types of COBOL-to-Java tasks. We need more training data around these examples.”
They’ll do whatever is needed to make a certain customer successful, but the bigger goal is to make sure the model keeps getting better for every customer.
About 50% of their time is spent on the customer site
They generally tell new joiners to expect around 50% of their time on site.
But also, as Colin mentioned, it varies by project. For example, some of their people in the UAE are on site around 80% of the time. People based in Europe or the US might spend 2-3 weeks per month with customers. In Japan, people are also on site around 80% of the time.
But in general, they focus on trying to have the biggest impact possible when on-site, and they’d rather spend 2 or 3 impactful days instead of 4-5 less impactful.
And also, a lot of their customers are spread across different locations, so being on-site all the time isn’t practical. But, as a general guideline, around 50% of the time on-site is a good estimate.