This as-told-to essay is based on a conversation with Prakhar Agarwal, an applied researcher at Meta Superintelligence Labs who previously worked at OpenAI. The following has been edited for length and clarity. Business Insider has verified his employment and academic history.
My day-to-day varies a lot depending on what stage of the project we are in versus what the immediate deliverables are.
At OpenAI and Meta, you have these milestones — say, a big training or reinforcement-learning run — in 10 months. It gets intense when we’re approaching the deadline.
Whatever work I identify is always based on the current iteration of the model. If I say the model isn’t good at X and my solution helps fix X, it is based on that version of the model. If I miss the deadline, I don’t know whether the next version will have the same issues or not.
If we are further away from that deadline, then we’re mostly working on evaluations and trying to find failure cases and issues with the existing model.
Every time Lee Chong Ming publishes a story, you’ll get an alert straight to your inbox!
Stay connected to Lee Chong Ming and get more of their work as it publishes.
The work is super dynamic. Sometimes you think something is super easy and you’ll get it done in a day. Other times, it’s the opposite — because there are so many unknowns, it might take a week.
Working at frontier labs feels very different from Big Tech
What we’re limited by in these foundational labs is compute. It’s not like Big Tech or other places where you can keep hiring a bunch of people and give them small pieces of a task to do.
Everyone needs compute to actually do something, and as soon as you have a lot of people, the compute gets divided, so no one will be able to do anything.
You also want high-bandwidth communication between stakeholders — you don’t want 10 different layers of communication. The speed of iteration is much faster. These core groups tend to be much smaller and tighter.
The idea of a “team” is also very fluid. Each person has their own projects, but they collaborate with others to work on joint projects. At Meta and OpenAI, there are a lot of senior people and not a lot of junior people, so everyone has a decent scope of projects.
Sometimes I collaborate more with people outside my immediate team than within it. Your scope isn’t restricted to four or five people. Your scope is the problem you’re trying to solve.
Communication and going deep with coding are key
Communication is the most important aspect in these labs. Because a lot of things aren’t documented, you need to be able to articulate what you’re doing, why you’re doing it, what the next steps are, convey your results, and get feedback on your work.
Becoming comfortable going through the code and identifying the specifics is one of the most important skills I’ve seen. The speed at which the code evolves is much faster than the documentation. If you’re stuck on something, read the code and try to understand it yourself.
Having some understanding of what’s happening across different verticals also gives you a good overview of the ideas and approaches people are trying. Because everything is super related, you might learn something from there or find ways to contribute.
The biggest advantage these labs have is knowing what doesn’t work
A research paper tells you, “I did X, Y, and Z in this specific order, and it works.” But what you don’t see is that before doing X, Y, and Z, I tried 50 different things that didn’t work — and people don’t talk about that.
That, to me, is the real strength of these foundation labs. Because of all the experimentation and all the work that has already been done, the teams have built really strong intuitions. They know which things won’t work or won’t scale, and which are going to work well.
People outside often look for the gains, but they miss the point that even the misses are very valuable.
Advice for those who want to work in top labs
I don’t have a good answer for managing burnout. You’re pretty much just going with the flow. You’re working at the cutting edge, and to put it simply, if you want to be here, you can’t think about it on a strict day-to-day basis.
What I would tell my younger self is to be comfortable exploring new avenues and new ideas. What I’ve seen is that we try to play to our strengths or stay in a deterministic setting where we know we’ll do fine. But in these domains, the speed at which things are moving is so fast that you need to be able to switch to a new topic.
Build the muscle to handle being thrown into something completely new. Sometimes, it’s more psychological than a skill issue.
Do you have a story to share about working at a top AI lab? Contact this reporter at cmlee@businessinsider.com.
