Siddhartha Ahluwalia 1:08
Hi, this is Siddhartha Ahluwalia. Welcome to The Neon Show. I’m your host and managing partner of Neon Fund, a fund that has invested at seed stages in some of the best enterprise AI companies coming from India, building for the US market and for the globe, like Atomic Work, SpotDraft, CloudSEK. Today I have with me Jonathan Siddharth, co-founder and CEO of Turing. Turing is accelerating super intelligence. Jonathan, Turing is one of the most defining AI companies of our era. It’s such a proud thing to see that the founders who are from Chennai and achieving so much today. So first of all, congratulations on all the success.

Jonathan Siddharth 1:51
Thank you, Siddhartha. And thank you for having me. It’s great to be here. And it’s impressive how large the Neon Show is in terms of subscribers.

Siddhartha Ahluwalia 2:00
Thank you. So Jonathan, I want to understand from you, Turing started in the hiring business when you initially started. You became a unicorn in that business. And then you started the business of being the provider to the frontier AI labs on the data side. How did this transition happen? If you can walk us through the journey.

Jonathan Siddharth 2:25
Great. Maybe for the audience, I’ll give a little bit of an overview of what Turing does. And then I’ll walk through the journey for how we got here. So our mission is to accelerate super intelligence to drive real economic progress. And we do that in two specific ways. First, we work with all the frontier AI labs to advance the models in SWE, enterprise, and frontier STEM by providing high quality data, evals, and RL environments.

In doing that, we discover the jagged intelligence of these frontier models and agents. We discover what different models and agents are good at. We leverage those insights as a differentiated deployment partner in the enterprise. And at the enterprise, we’ve gone deep in financial services. We work with some of the largest asset management firms in the world and build end-to-end AI systems for them. And when we build these systems, we discover where these models and agents break. We leverage our knowledge of those capability gaps to create even more realistic data, evals, RL environments for the frontier AI labs. So we can solve even bigger problems in enterprise, see where things break, generate even more realistic data, solve even bigger problems in enterprise. And we execute this loop. So most data companies don’t see deployment and most deployment companies don’t see data. So Turing is the one company that works with the frontier AI labs and Fortune 500 enterprises. And we are a trusted bridge between research and deployment. On one hand, it’s like humans making AI smarter. And on the other hand, it’s AI making humans smarter. And these both compound.

That’s what we do. And you’re right that the journey here is super interesting. So Turing started as a platform that used AI to source and vet software engineers from all over the world. We have the world’s largest software engineering talent platform, Silicon Valley caliber software engineers from all over the planet. And we were growing really fast. The world kind of changed when OpenAI came to us when they were training GPT-3 and they wanted to teach GPT-3 to code. So we collaborated with them on teaching the models to code and to do tool use, function calling. And then when ChatGPT launched and it took off, it was the birth of the AGI race. And it became clear to us that this is going to be a huge shift in the way software engineering is done and how all knowledge work in front of a computer will be done. So we leaned into it and we built a data platform on top of our talent platform and expanded beyond coding to enterprise knowledge work, frontier STEM. And it’s been incredible working with all the frontier AI labs to advance the models in coding, knowledge work, and STEM.

Siddhartha Ahluwalia 5:53
And how did OpenAI reach out to you? What was your mode or positioning in the data space?

Jonathan Siddharth 6:02
So we were not a data company when they reached out. I think OpenAI at that time was looking for, I mean, I don’t know for sure, but they were looking for who has the best coders on the planet. And it is Turing. We have the world’s best platform for software engineers. And now we’ve expanded that to experts in every industry you can think of, financial services, retail, healthcare, life sciences, every function, software engineering, sales, marketing, finance, every role in the org chart in that function. I mean, Turing’s platform has the smartest humans in the world in every domain. I think OpenAI at that time was just looking for somebody who can generate really high quality data for coding models really quickly. And the fact that we used AI to source talent, vet talent, match talent, manage talent meant we could scale up really fast while keeping quality high. That was the key.

Siddhartha Ahluwalia 7:06
So they used your this engine to help them or they said, Jonathan, this is what we need on the data side for coding. Come and help us with whatever you have.

Jonathan Siddharth 7:21
So I wouldn’t be able to get into the specifics, but it was about generating coding data to improve the models. At that time, the models were at the level of writing like some relatively simple Python code, like relatively the kind of coding tasks that a human would do when interviewing for a role. Now we’ve scaled up our engine for agentic coding where you’re seeing what the coding models today are capable of building end-to-end systems like coding for days, sometimes a week, and we’ve evolved it. But it started with just, I think like they needed a partner that could help generate really high quality coding data that could improve the models. You’re evaluating weaknesses in the models, figuring out failure modes, and generating data to address those gaps.

Siddhartha Ahluwalia 8:28
And if you have to explain, let’s say, coding data to our audience, who many of them might not come from the AI background, how would you explain it?

Jonathan Siddharth 8:37
So I think it’s one way to think about data is just look at the LLM training pipeline today. So there’s pre-training data, SFT data, and reinforcement learning data. All of this is slightly different. In pre-training, you’re feeding the model the entire internet, and you’re also giving the model access to books, other sources of knowledge. So pre-training is largely unsupervised. So you’re just feeding the model data, the model is building, I mean, you’re basically building a base model that learns how to auto-complete. In post-training, think of pre-training as creating like, it’s almost like brain mass. There’s this raw brain mass, but you haven’t quite shaped it yet. And in that pre-training corpus, from a coding perspective, there’s also data from GitHub, and other sources of code. If you’ve acquired code assets, you might also feed that into pre-training. But the base model is only doing auto-completion of tokens. It doesn’t yet know how to follow instructions. In supervised fine-tuning, there’s this paper from OpenAI called InstructGPT, which talks about the recipe for how to do this, where you’re teaching the model to be an assistant. So in supervised fine-tuning, that’s where GPT-3 becomes ChatGPT. So you’re teaching the model to be helpful, honest, harmless, and you’re teaching the model how to have a conversation.

So humans are teaching the model to, you’re having conversations with the model, the model knows how to communicate with the user. Now, then there is reinforcement learning. And in reinforcement learning, you create RL environments, where you’re creating these environments where an agent is getting a reward when it does something correctly. And when it doesn’t do something correctly, it doesn’t get a reward. And usually, you also have to give the agent intermediate rewards as it’s trying to get to a solution. So the context of coding, there is a benchmark called SuiBench. For your readers, I encourage them to read it. SuiBench is a really good benchmark, which is a, it’s open, it’s largely open source GitHub repos, where you are, the models are being tested on their ability to merge pull requests correctly with the unit tests passing. So one type of coding data that’s helpful to people building models is, and Turing does this, where we are automatically crawling open source GitHub repos at massive scale. We are also acquiring code assets from startups and other companies. And we are running those repos and figuring out pull requests where the test cases don’t pass for today’s frontier models and agents. So we have to, and in some cases where the test cases don’t exist, we would create test cases. But you’d want data sets like this that are realistic, like reflecting real world software engineering, long horizon, where the agent has to work for hours or days to complete a task and calibrated at the right level of complexity. If you have tasks that are too easy for the model, there is no learning signal. If it is too difficult, there is no learning signal.

So there is a sweet spot of complexity that you’d want the RL environments to be at, usually like 20 to 40 percent, something in that zip code, like a Goldilocks zone. And you’d have to spin these up at massive scale. And Turing does that. So that’s an example of one type of coding data set where you’re evaluating the agent’s ability to merge pull requests with test cases that pass without false positives, false negatives. There are other constructs like this. There is a benchmark called Terminal Bench, where you’re evaluating… Terminal Bench is the kind of benchmark you need for a command line coding agent that has access to a terminal, like think Cloud Code, Code X, Gemini CLI, Grot build, agents like that. There, it’s an interesting problem where you give the agent access to a terminal and access to the internet. And you’re evaluating its ability to do complex tasks in software engineering, machine learning, cybersecurity, scientific coding, etc. In theory, if you have access to the terminal on your computer and the internet, you could do anything here on your computer. And there again, you’re evaluating the agent’s ability to do this type of knowledge work. But with Terminal Bench, you don’t just evaluate the output, you evaluate the state of the system.

Maybe you might have to install something, you might have to write code, you might have to create some documents or PowerPoints or spreadsheets. Yeah, so that’s another example. Terminal Bench is like another benchmark. There are many others in coding. There’s MLE Bench, which evaluates an agent’s ability to write machine learning code that satisfies some objective. The way to think about reinforcement learning is if you’ve, I think the example that everybody remembers well is AlphaGo and AlphaZero by DeepMind. So AlphaGo was DeepMind’s Go playing program that was originally trained by learning from export humans. AlphaZero was their program where it was not trained from humans, it just played against itself. And obviously, in a game, you get a reward based on whether you won the game or you lost the game.

And depending on whether you won the game or lost the game, you can figure out which moves led to you winning the game or losing the game. You can have intermediate rewards. You can imagine a similar construct for coding or for knowledge work, where the reward can be whether you did the task well or not. But you have to be smart about intermediate process rewards, how do you reward partial completion to get to the task? And the task could be writing a program that can automatically vet a finance analyst, maybe that’s a task. Or it could be create a board deck. Or it could be evaluate this investment opportunity and write an investment memo. It becomes a little more complex when the task is not binary, like with code or with math, it’s relatively more binary, easy to verify. But how do you verify the quality of a board deck? You have to have a good rubric-based evaluator. So, yeah, lots of interesting problems. But reinforcement learning has been one of those fascinating areas that, like deep learning, has been a very rich vein. The thing I find fascinating, Siddhartha, is Ilya Sutskever, the chief scientist at OpenAI, I saw in a podcast that apparently his research program for OpenAI in 2014 was train a generative model on the world’s knowledge and do RL on it. And this was in 2024. And more than a decade later, it’s still the formula that everybody’s largely following. I remember this Sam Altman quote of, do more of what works. And I feel like deep learning works and reinforcement learning works. And I think all the labs are scaling this up. And we are constrained by algorithmic research, compute, and data. And companies like Turing advance the data pillar, NVIDIA advances the compute pillar, and of course, the labs advance the algorithmic research pillar.

Siddhartha Ahluwalia 17:55
And from 2014, say, OpenAI started roughly during that time frame. So, 2014 to 21, 22, the world never knew about how fast AI is accelerating. But suddenly, 22, 23, until now, every year, the acceleration is 10x more. Why is so much development happening at this pace?

Jonathan Siddharth 18:22
So, I think like, I feel like up until 2012, we were in the pre-deep learning era, we were in the machine learning era, right? Where, I mean, I did machine learning at Stanford, like the, where you would, there were all these different algorithms you would use. Neural networks was one such algorithm. We hadn’t really discovered that deep neural networks trained with lots of data and lots of compute do magical things. We hadn’t yet discovered that. 2012 was a milestone when ImageNet came out. The ImageNet breakthrough came out where these convolutional neural nets exceeded human level performance in certain tasks. And after 2012, the world just doubled down on deep neural networks, lots of compute, lots of data. And we started using GPUs for training. And I think there was a small group of people that knew what was happening. And there was that group that bet on deep neural networks. I think the OpenAI team was one of them. And I think it’s one of those things where exponential growth is like, exponential improvements are like hard to fathom. And between GPT-2 to GPT-3 to GPT-4, it’s like, as you kept scaling up, these models just keep getting, kept improving. It was reported that the current series of frontier models are largely in the trillion parameter realm.

Now we are starting to see 10 trillion parameter models. And I think when these models come out, I think we’ll see it’ll feel even more magical. Some people say like, intelligence is log of compute, like you’re just…

Siddhartha Ahluwalia 20:23
You’re just a computer.

Jonathan Siddharth 20:25
Yes. And the scaling laws are continuing to hold, meaning bigger model with more data, with more compute, means that the models smoothly keep getting better. It’s like when the pre-training loss keeps coming down, it’s like the model’s performance on also all these, a variety of intelligence benchmarks seems to keep going up. So I think it’s just people discovered the power of, I think OpenAI led the way and everybody has discovered that the scaling laws are holding. And I think it’s the world’s energy being more focused on a recipe that works. I think so we’re seeing faster progress.

Siddhartha Ahluwalia 21:12
Cool. But since now, Anthropic started in 21, 22 roughly, but in three years, it’s very astonishing how much they have been able to accomplish. And today, across enterprises, or even mid-market startups, they have become the default medium for coding. How has such acceleration happened in only three years?

Jonathan Siddharth 21:39
I think with coding, coding is one of those areas where, because it’s verifiable, I think that there is a good path to using reinforcement learning to improve coding models quickly. And I think like the labs, many of the labs see that coding is really important. It’s important for a few reasons. One is, many believe that the fastest way to get to AGI is to automate AI research itself. And to automate AI research, you need a really strong coding model. Since AI researchers are implementing research papers, trying out new ideas, you have to write a lot of code. So if you can accelerate your ability to write high quality code, ideally figure out what experiments to run, you’ll, of course, accelerate. So there was a lot of energy that went into improving coding. Second, when the models improve in coding, they have out-of-domain performance. They also improve in other tasks that have nothing to do with coding for reasons we don’t fully understand. My hunch is that when the models improve in coding, they also improve in tool use. And that helps these generalized computer use agents like Codex, Cloud Code, Gemini, CLI, etc. So I think a lot of people prioritized coding. And as a result, the coding models improved. We are probably the largest provider of coding data to all the labs. And before we expanded into other types of knowledge work, we really focused on coding. So part of it is the focus and the importance to the lab. So it just improved more quickly. And I think some of these benchmarks that we had, I feel like there were a lot of academic where you’re testing intelligence rather than the ability to do real work. And it was needed at the time. Like in the last few years, we needed to know how smart is AI.

So we were happy when AI wins like a gold medal in a math Olympiad or a physics Olympiad or wins a coding competition, stuff like that. But I think there were also some really good benchmarks for coding. I do think SWEBench was a good start. Now SWEBench is saturated. Now you need more complex coding benchmarks. Turing is creating many complex coding benchmarks. And those also help the labs hill climb on something that was more economically valuable.

Siddhartha Ahluwalia 24:34
So one of the interesting things is that you mentioned everything that you give to a model is almost like a, which you shared offline, is almost like a coding problem. Can you give that example that you gave me offline?

Jonathan Siddharth 24:50
Yes, I was saying like one reason coding is so important in AGI is a lot of problems are reducible to coding. For example, if you’re asking an AI a question like what’s the difference between AI investing in Silicon Valley versus AI investing in India, a good model would come up with a plan. The plan might involve querying CrunchBase, querying PitchBook, downloading some data, figuring out, okay, let me break down AI investing by sector or by category. Let me write some Python code to analyze the data, to visualize the data and present it to you. So it wrote code as part of its execution. So we all know that a decade ago, Mark Andreessen spoke about software is eating the world. What he really meant was software is eating all types of knowledge work. And now AI is eating software. So AI is eating all types of knowledge work.

It most tasks people do involve data analysis, involve some type of writing code. For example, like I’m coming off of a leadership offsite with my team. And one of the things we do in the offsite is figure out exactly where to bet on. And it’s actually a data analysis problem. You’re looking at which segments are growing fastest, which segments are not growing as fast. You might want to analyze data from your own internal systems. And that’s an example of, again, where coding is kind of embedded in the loop. If I ask my head of FP&A, what are some areas in Turing we should invest in? What are some areas we should reduce our investment in? That’s, again, a coding problem. A lot of problems in life are math problems framed the right way. And code is a way to express and solve math problems. And I feel like if you can solve coding, tool use, reasoning, and multimodality, you have the keys to superintelligence. Those are the building blocks. If you can solve those four, you can automate anything that a human does in front of a computer. And it’ll give humans superpowers. For example, as an investor, you might be interested in analyzing what are some trends in investing among top investors. That’s a coding problem. You might be curious, how is my portfolio at Neon different from Sequoia? That’s a coding problem, coding and data analysis. Now, every human has the benefit of a giant software engineering team in the cloud that’ll envision whatever they want. It’s actually even more than that. When you think of you having access to a giant team of software engineers in the cloud, the onus is on you to also ask the right questions. But now, given we’ve also baked in humanity’s best knowledge from strategy consultants, analysts, data scientists, software engineers, product managers, CEOs, CTOs, it can – I mean, you could literally ask your AI agent to come up with a strategy for how to build a portfolio that maximizes returns. Of course, it may be wrong today, and it’ll probably be wrong for a while. But that’s where you could exercise your human judgment to correct it. Basically, coding is the key to superintelligence. You solve coding, you’ve solved almost everything.

Siddhartha Ahluwalia 28:45
So then, with this thesis, the edge in most fields would – the human edge in most fields or knowledge working fields, for example, investing is one of the fields where judgment counts. And public market, the data is more visible. In private markets, you don’t discover – data is not visible, mostly, especially at the earlier stages. So, according to that thesis, the human judgment becomes less and less important. Because if AI superintelligence can make those judgments where to invest, then everybody has access to that information.

Jonathan Siddharth 29:27
So, I actually disagree. I think human judgment is going to become even more important. I think what’s happening is the floor is going up. The – for example, if a software engineer is working with a coding agent that has now written code for a week, you do – I mean, reviewing the code that got written needs work. You have to know where to look, what to check. Of course, AI will also help with that. And in your point about investing too, I don’t think these models are still at a point where human input is not needed. I think of these as productivity accelerators. Today there are things that you’re using tools for to accelerate your ability to do work. I think these frontier models are the ultimate knowledge work accelerator. So humans can solve problems at higher and higher levels of abstraction. I think we used to think about humans as having, in the Andy Grove high output management sense, maybe your span should be seven to eight people that you’re managing. Now I think in the limit over the next decade, every human might run seven to eight companies as the loop from idea to, I mean, as you can go from prompt to company in hopefully like a single step over the next decade.

Siddhartha Ahluwalia 31:10
Yeah.

Jonathan Siddharth 31:12
I think humans will ultimately still be key to knowing what questions to ask, knowing what problems to solve. It’s just that now going from idea to execution has been democratized. In the past, if you were a plumber or a doctor or a lawyer, you might have an idea for a cool startup, but you’re bottlenecked on, I have to raise some venture capital, I have to hire some engineers, I have to hire a salesperson, a marketing person. There was high friction. Now it’s like much lower friction. I’m very bullish on what this means for humanity. I think we’ll see more entrepreneurs, lots and lots of companies getting started. I think larger companies today will be, companies in the future will be a lot smaller. I think humans are just going to be, I mean, we are more productive today after personal computers, after the World Wide Web, after Google. And now I think this is the next step. AGI is going to just make humans 100x more productive.

Siddhartha Ahluwalia 32:28
So one limitation that I see right now is, let’s say, if I have a high agency teammate in my investment team, on a daily basis, they’ll think on what problems to solve. I think models are not there yet.

Jonathan Siddharth 32:47
Not yet, but they will be.

Siddhartha Ahluwalia 32:51
And you believe that that is a near-term future?

Jonathan Siddharth 32:54
Absolutely, absolutely. It is, even now in theory, you could get that behavior by looping the model as an agent. There are ways to get that from the models today with the right scaffolding, with the right harness. Absolutely. For example, I have a project at Turing that I run. I have a few agents running around to help me run Turing. The way I improve that code base is to also have an agent suggest to me, how should I improve the code base and to do it. So it’s in a self-improvement loop by itself. You can imagine applying a self-improvement loop to yourself, to your company, and letting it run. Obviously, it may not be at a point where you trust it enough to keep running it constantly. But even that is huge. I mean, even your human analyst might come to you for approval. I have this new business plan. I think we should do this. This will require this much investment, but it will give us this much in revenues. You will do that, right? So I would argue that it’s already there with the right harness, and it’s going to get even better, especially with these bigger and bigger models. I think of what we have right now is intelligence as an API. And it’s a very unique type of intelligence, but it’s already superhuman in many things. It’s subhuman in certain things. And as long as you build around its jagged intelligence, you’re good.

Siddhartha Ahluwalia 34:35
Understood. So today, software engineering is one of the works that is getting replaced globally. Companies, if you observe, people are replacing or reducing their team size in software engineering.

Jonathan Siddharth 34:54
So I think there’s a few things happening. I think these models have democratized coding to the point where every software engineer is now insanely more productive. Now, if it’s a company that has product market fit that wants to scale, suddenly your team is now 10x more productive, and you can grow faster than ever. If you’re a company that doesn’t have product market fit, or you’re treating R&D almost like a cost center, then yes, you need fewer engineers, and you can probably do that. But it’s also the case that engineers are now able to create a lot more than ever before. It’ll create some challenges with distribution. If you create a lot of apps, how do you get users? How do you make money? Those are still important problems. But I feel like at Turing, for example, we are scaling our software engineering team. We want really strong engineers because we see this massive opportunity. We are significantly more capital efficient in terms of how we can scale. So I think the impact of it is dependent on the company and where it is in their life cycle. You are correct that if you’re an engineer that is not using AI, not even an engineer, if you’re in any type of knowledge work, if you’re not using AI highly, like I’m talking, you should actually be the mindset of agent first, human second. Turing operates this way.

Siddhartha Ahluwalia 36:47
What do you mean by that? Can you share examples?

Jonathan Siddharth 36:50
So every knowledge worker at Turing, either in coding or any other domain, I want them to use AI to do the work first. So agents should create, humans should steer. Humans should create prompts, skill files, give the agent access to the right tools, access to the right knowledge sources. Humans should create the scaffold and AI should create. For example, if you’re creating a board deck, the investor, let’s say my head of investor relations, he shouldn’t create the board deck. He should give the model access to the right sources of knowledge, access to the right tools, write a skills file to apply consistent formatting, give the right instructions on how to create the board deck, use Codex or Corework or Gemini or Grok to build it. And then when there are mistakes in it, prompt the model to improve it and then keep improving the skills file so that next time it’s even faster. Basically, the agents create V1 of the work. Humans are verifying and iterating. Humans never create V1. There’s an interesting post from OpenAI on how they used Codex for one particular software project, which was agent first, human second. Humans were not allowed to write code directly. They can only like-

Siddhartha Ahluwalia 38:35
Prompt agent.

Jonathan Siddharth 38:35
Yes. So I think like knowledge work, that’s what I meant when I said agent first, human second in how you do knowledge work. And I think it’s a huge productivity unlock across every function and every workflow in an organization. Think of every function, software engineering, sales, marketing, finance, people recruiting. Think of every role in the org chart of that function. And every role is a composite of workflows. And every workflow has an artifact that you create. There’s a clear definition of done. At Turing, we are in this unique position where we generate data to help the models improve in every workflow, in every role, in every function, in every industry. We create data and oral environments to improve the models. And we also go build with it in the enterprise. So we know exactly where these things break. So that’s why I have the somewhat unique vantage point into seeing where this is going. And it’s super exciting.

Siddhartha Ahluwalia 39:47
And do you think the difference between today, services and product company is blurring out?

Jonathan Siddharth 39:53
Yes.

Siddhartha Ahluwalia 39:54
Can you share your perspective on that?

Jonathan Siddharth 39:59
So I think what’s happening is there’s two trends that are happening right now. One is the models are becoming agentic, where the models can accomplish long horizon workflows for every type of knowledge work for every type of coding project, the models are able to do it end to end, right? And if you can control your computer, they can do everything you can in front of your computer, right? With the humans verifying. So on one hand, the models are becoming agentic. And in theory, if the models have access to the right sources of data, you kind of don’t need software in the middle, the models can do it end to end. For example, imagine you want to create, here’s a fictitious conversation that I may have with an agent. It’s not quite fictitious, because at Turing, I’ll soon be able to have this conversation. But my conversation to the agent might be, hey, I want to create a hiring rec for a researcher in reinforcement learning for code. The agent will tell me, oh, that’s great. Here, I created V1 of a JD. Does this sound good to you? And I’d say, you know, this sounds good. No prior experience needed. So let’s say, okay, done, I’ve made that change. And then it says, the agent says, hey, by the way, I looked up comp benchmarking data, and I came up with these compounds for the role. I also checked with the finance agent, and this is in budget. Does it sound good to you? I said, yeah, that sounds good. Let’s go for it. And add a geo constraint for the Bay Area and Bangalore. And then the agent asks me, hey, can I update the website with this new role? Yeah, go for it. And by the way, in the future, it may not even ask me to remember this once next time it’ll do it itself. And it could say, I’ve also created, would you like me to create a LinkedIn ad for this role? I say, yeah, do it with a $1,000 budget. And then it could then say, I see that your calendar is, you have time on your calendar, you have like 10 hours a week in your calendar for recruiting, can I use those slots? I say, yes. And then when somebody emails me for this job, it could set up the meeting, it could even do the first assessment, could assess talent on certain things. Now, in this workflow that I mentioned, I did not talk to anybody in finance, anybody in HR, anybody in recruiting, anybody in the website, anybody in our engineering team. I did not talk to our performance marketing team to run ads. It was just the agent. And there was no SaaS software involved. There was no Greenhouse. There was no Asana. There was no separate vetting software.

The agent kind of whipped up whatever it needed on the fly. There was literally no human involved. So there’s one trend where the models are becoming agentic. So legacy SaaS is going to have some challenges as the models become more agentic. I’ll share, think of this as a dual pincer. It’s a pincer movement. That’s the top-down pincer. The bottoms-up pincer is every human can now write software. My EA wrote software because I have a crazy schedule. I work a lot. I travel a lot. I’m always on. So it creates a lot of work. So my EA wrote software to manage her EA team. And that software also automatically keeps track of all demands on my time, prioritizes it based on what’s most important for me. So nothing gets dropped. Now, she could have used Trello or Jira or Asana, but she didn’t. She built her own task management system. And she built something that was more custom for her than any of that software. It’s the era of infinite software, custom software. And there is an alternate universe. I’ve heard pitches for these types of software to help EAs serve their execs better. But probably wasn’t venture backable, or for whatever reason, it didn’t take hold. But she just built it and used it. So we’re not buying it. So you can see that superintelligence is eating SaaS, and it’s eating services, right? It’s as the models become agentic in this dual pincer motion, like one, every human now creates software that they wouldn’t pay for, but it’s just a fun weekend project now. And the models are becoming agentic. I think these two pose some unique challenges where I think the key is going to be who has access to the data layer. I think the data is going to be key. And in the enterprise, I think that will be particularly important.

Siddhartha Ahluwalia 45:23
So according to this theory, then all workflow SaaS would let’s say next five years would not be required, then only the system of records will be required.

Jonathan Siddharth 45:36
I think it is a possibility. I’m not saying it will necessarily become true. I think it’s a possibility. I think the I think the workflow SaaS providers will have to think deeply about about their moat. I think right now their moat is a little bit of controlling the data. Like if you’re a CRM company, you have access to this data, but a brand new company that’s starting today may not use a CRM or some of these workflow tools that way. They may just start either with custom software with them creating it themselves, or they’ll use the frontier models in clever ways while managing context, memory, and data themselves.

Siddhartha Ahluwalia 46:25
So help me explain a company which is starting new and scales quickly. How does it build its own system of record?

Jonathan Siddharth 46:34
I’m not saying they may, they’ll build. So they could. I mean, it’s all of it is just software, right? And if in the past, like writing enterprise grade, secure software was hard, and it would have taken a while. Now it’s become easier. So they will just use the models to write what they need to write. And the benefit for them is custom software, software that is fine-tuned for their specific workflows. Like we are used to, because the cost of software R&D was quite high, especially for ML AI software, it made sense to go outside and use something. Now you can build it. I mean, there was a whole industry around, you know, implement, you know, like the, I don’t want to name specific software, but you’re like implementing your own customization to some legacy SaaS software, right? Though you don’t have to.

Siddhartha Ahluwalia 47:40
The entire GSI industry is based on that.

Jonathan Siddharth 47:43
Yeah. There’s an interesting thesis that’s going to be validated. Like it’s kind of unclear how it’s going to play out, which is on one hand, we could be in a world of, I’m going to call it the no fine-tuning world, where you have a frontier model, like a trillion parameter or 10 trillion parameter from OpenAI, Anthropic, DeepMind, Meta, XAI, Amazon, Microsoft, somebody, a frontier model. And all you need to do is manage context and memory, give it access to the right tools, and let the model cook, right? So that’s one way, the no fine-tuning camp. All you have to do is be smart about how you’re managing context and memory. You don’t have to touch the weights. That’s one world. Let’s call it the no fine-tuning camp. There’s another camp, which is, hey, you don’t need these giant models. You could, with a half a billion to 10 billion parameter model, fine-tune it on your proprietary data, distill your proprietary human knowledge into the LLMs, automate your proprietary tool calls. It’ll be faster, cheaper, more accurate. So that’s one school of thought. Let’s call it the fine-tuning camp. I think it’ll be interesting to see how it plays out.

What I’m seeing more and more is there’s a lot more people going into the no fine-tuning camp than a couple of years ago for very high value enterprise use cases.

Siddhartha Ahluwalia 49:20
Can you give examples?

Jonathan Siddharth 49:24
I mean, I have a system at Turing that helps me run Turing, and this is a system that I built on top of one of the frontier models, and I didn’t have to fine-tune it. All I had to do was intelligent management of context, memory, access to the right tools, access to the right sources of knowledge. And you can imagine in asset management, in financial services, there are workflows in the day-to-day of a fund controller, of an asset manager, where the frontier models are actually doing very, very well without any fine-tuning. I would start more specifically with what’s the workflow that you’re trying to automate and what’s a verifier for the quality of the artifact that that workflow is creating. And step one should be to just do it with a frontier model and see how well it does. And most times you’ll see that you don’t have to touch the weights, but there are certain workflows like customer support, like an invoice-to-pay system, some of these more basic stuff, where I think a smaller open-source model that’s fine-tuned on proprietary data might be faster, cheaper, more accurate. But for really high-value use cases like discovering alpha in investment opportunities, you want the most powerful model. And you’ll probably see that that powerful model with good context and memory management will do very, very well. And I personally don’t fall into any specific camp. I think it’s much more specific to the workflow that you’re trying to automate. Some require giant models that you don’t have to bother fine-tuning, but have good context management. Some require smaller models that you can, some you can make do with smaller models.

Siddhartha Ahluwalia 51:51
And for our audience, if you can share, let’s say, in very simple terms, training and inference has been the key. And let’s say, the last few years, there was huge investment in training part. But now inference is also coming up well. So probably if you can share what has been happening and where is it going in both these areas.

Jonathan Siddharth 52:16
Yeah, yeah. So there is like, there is scale-up in compute for a few different things. There’s pre-training compute. So I would break training into two parts, pre-training and post training. In the past, it was a lot of the compute went into pre-training. Now a lot of compute goes into reinforcement learning in post-training as well, especially after O1 came out and DeepSeq came out. So there’s a lot of compute that’s needed for still for training. But what we’re seeing is as these models have gotten so much more capable, people need so much more. Like I’m compute constrained, like for how much I want to create agents, run agents. So it’s, so inference compute is skyrocketing because demand is skyrocketing.

Siddhartha Ahluwalia 53:16
And what are the use cases for that?

Jonathan Siddharth 53:19
All types of knowledge work, all types of coding. For example, what’s something in your workflow that you do most frequently in your workflow?

Siddhartha Ahluwalia 53:33
I think search for company data. Let’s say, for example, if I’m evaluating a cyber security company, I would try to find details about that company, mostly details about the competitors and where the industry is heading. And I would then try to find customer usage data to verify how are customers using. Is the customer usage growing every month or not? Because for us, I’ll give you a simple example. We come at pre-seed at zero revenue. So it’s very important that after investment, two or three years, a company can reach 10 million ARR. So all our effort is to validate and validate or disvalidate in that direction.

Jonathan Siddharth 54:22
Yeah. Can I, I’m just brainstorming on the fly with you. But I would, can I give you a use case for you to be compute constrained, inference constrained? It would be to run a deep research agent in a loop that is looking for cyber security companies all the time. While we’re talking, that agent is running, right? Maybe on a Mac mini somewhere safe. It’s running. It’s pulling interesting companies that it thinks are good, but then you’ve given it a budget. So it goes and spend some money on GLG or like some expert network. It recruits some humans. The agent spins up zoom and does a conversation with that human does interviews with them, gives you a summary of the interview results. And in some cases, it may play with the software itself. Cyber security. I don’t know if it is possible to do it well, but it might be, it might test the software by itself, or it might recruit some other humans, give them access to the software, collect their feedback. Maybe it recruits some CISOs. As long as you give it enough money to go hire people, research the web, go hire people, talk to people, run experiments and come back. This agent may come back to you every week and give you a summary. Hey Siddhartha, like here are the top three companies in cybersecurity that I think you should pay attention to. These are the most hyped. These are the most, these are the most underrated, but where maybe there is more investment alpha. How much would you pay for something like that? It’s literally helping you discover your next investment opportunity. It’s like your associate.

Siddhartha Ahluwalia 56:05
Yeah, absolutely. I’ll pay the cost of an associate.

Jonathan Siddharth 56:08
Yes. And that’s, that’s, that’s why you’d be inference constraint, right? Because to do that, it’s making a ton of LLM calls. It’s like running in a loop, right? Like that. I feel like if we kept talking, we can come up with even more where, and that’s why like inference is skyrocketing. It’s because there’s so much model capability overhang. The models are capable of so much more, but humans are still not extracting the fullest value from these models, especially in enterprise. In enterprise, I think first mile at last mile is still a problem. Like the average enterprise is still not getting the best value. And at Turing, that’s one of the reasons why we focus on enterprise deployments. Even if the models did not get any smarter, there’s so much more value to be unlocked. But of course the models are going to get a lot smarter.

Siddhartha Ahluwalia 57:01
Yeah. Because I think all the money in the world is getting concentrated on the models.

Jonathan Siddharth 57:06
Yes. I used to, I used to like a few years back, I used to think about why do we need so much compute here? Now it’s clear to me. Like I think humanity is going to be reasoning bound and compute bound to solve some of our biggest problems. And especially coding enterprise and STEM are so important. If you accelerate coding, you’ll accelerate AI research. If you accelerate knowledge work, like we’re going to create so much more value in the economy, like we’ll move the GDP of the world. And if you accelerate scientific discovery to discover new materials, new sources of energy, more efficient robotics, there’s just so much more innovation that will unlock. So I think like it’s going to be an exciting decade or two ahead more.

Siddhartha Ahluwalia 58:07
The biggest fear right now among founders and the community is that are models going to eat up all the work? Then we discussed about cybersecurity. I mean, if models eat up the domain of cybersecurity, then what is left for entrepreneurs to create? Similarly in software also.

Jonathan Siddharth 58:26
I think we’ll keep building on top of the models. Sam Altman spoke at a Turing event a short while ago. And one thing he would say is there are two schools of thought. There are two camps. One camp that’s betting on a world where the models are getting better. There’s another camp that’s betting on a world where the models are not getting better. You create all these other structures to fill in the gaps. I think like I personally would bet on a world where the models keep getting better. There are certain things you may do today thinking that the models are not going to get good at this. I would not do that. I would just think of intelligence as continuing to improve. I think like humans, the best founders will discover big markets. Like my advice to founders is pick a big market, markets that don’t exist, don’t care how smart you are or how hard you work. Pick a big market. Pick a market with weak competition. Ideally not competition that’s other big tech companies or other AI labs. Pick a market with weak competition or analog competition. For example, Uber competed with taxi companies, analog competition, not a fair fight. But companies like Airbnb competed with hotels. Again, analog competition. So pick analog competition or relatively weak competition. Don’t don’t compete with the frontier labs or big tech companies. That’s just not a good strategy. And third is the I think the basics of higher, really smart people who are highly driven. I think that always holds. I would think about, I would think about coming up with ideas that are betting on a world where intelligence will be abundant, and the models keep getting smarter. I think the hard problems to solve will still be, how do you get distribution? How do you generate revenue, not short term revenue, but long term revenue? I think those problems will still will still remain. And I think I would pick on problems that require inherently human, a human touch, like maybe problems more in the physical world, rather than in the purely digital world. Maybe problems where human trust, human connection are important.

Siddhartha Ahluwalia 1:01:29
Can you give examples of some of the areas where, let’s say, if you have to invest personally, because which are the kind of companies that you would invest?

Jonathan Siddharth 1:01:40
So I’ll give you a set that I think is necessary, but not sufficient. It’s a subset. I haven’t thought deeply about expanding it. I would bet on companies that are an input to superintelligence.

Siddhartha Ahluwalia 1:01:53
Like?

Jonathan Siddharth 1:01:54
Turing’s an input to superintelligence, like the models need data, they’re always going to need data for a long, a very, very long horizon. I would bet on companies that are either an input, like, and what are the key inputs, compute, energy, data, you need high quality research talent, but the labs would hire that themselves. Those are inputs, they are fuel to superintelligence. I would bet on companies there. I would also bet on companies that are betting on a future where intelligence is widely distributed. Turing’s enterprise business is a bet on deployment that intelligence is already there. Let’s just build the right systems around it so enterprises can unlock value. For example, if Neon were a Turing customer, we’d build that agent for you so that you have an associate GPT or an associate cloud that can do these. So I’d bet on companies that are inputs to superintelligence or an outcome of superintelligence. In a world where superintelligence exists, you’d need it. I might also bet on, I mean, robotics clearly will be huge. And it’s, I think it’s a matter of time.

Siddhartha Ahluwalia 1:03:15
Would you invest in domains like cybersecurity or would you think models would take up that domain?

Jonathan Siddharth 1:03:20
I would invest in domains of cybersecurity. Turing is investing in cybersecurity to make sure that agent decoding is safe, both for offense and defense.

Siddhartha Ahluwalia 1:03:36
But with anthropic launching models like Mythos, there’s a huge debate that would they try to take up whatever is going on, on vulnerability management, and then slowly start getting into endpoint management and everything else.

Jonathan Siddharth 1:03:51
Yeah. I think it’s, I would not build a, when I said I would invest in cybersecurity more in, I mean, we are investing in cybersecurity at Turing to help these models keep our systems safe, help these frontier models detect vulnerabilities in a safe way, and help the coding models write code that is more secure. I meant in that sense. But you’re right in this, the same reason I gave against legacy software, legacy SaaS software also applies to legacy cybersecurity products too.

Siddhartha Ahluwalia 1:04:37
Or I would say entrepreneurs not stop thinking in terms of products, but start thinking in terms of problems.

Jonathan Siddharth 1:04:46
Yeah.

Siddhartha Ahluwalia 1:04:48
That’s what you are hinting at, right? Whatever problem horizon that feeds into model or becomes an outcome of models. But to contest you, when I say the work that you are doing for an enterprise, the model are coming or will come in that kind of a work. So how do you make that defensible and growing?

Jonathan Siddharth 1:05:10
Yeah. We are building on top of the models, oftentimes with intelligent context management, memory management. For the enterprises, oftentimes they want a system that is relatively model agnostic. You’d want to make sure that you have the right auditability, governance, traceability, verifiability. You want a human in the loop. You need a workflow that’s constructed on top of the models. We would not build our own model. We don’t want to compete with our customers, but we are building around the models so that enterprises can unlock the fullest value. And in the real world enterprise, there is still a lot of last mile messiness, where there is a gap between what the model is capable of and what an enterprise is actually able to extract value out of. And it’s non-trivial. Even at Turing, it requires work to make sure that you’re operating in the right agent first, human second way.

Siddhartha Ahluwalia 1:06:18
Yeah. So you’re better. The model will not come into the last bit of stitching together everything.

Jonathan Siddharth 1:06:27
Yes. And part of that bet is you’re seeing like… I mean, Turing is hiring an insane amount of forward deployed engineers. You can see OpenAI, Anthropic, and all the Frontier Labs also doing that because it’s hard. It’s hard. And that last mile is also not fully digital. In some cases, you have to be in the right rooms, talk to the right people, observe the right workflows to make sure that you’re doing something that helps people. You have to manage change inside those organizations. You have to train people in new workflows. It is complex. And you’re also not white coding in the enterprise. You have to make sure that systems are secure, maintainable. They are built with the same coding standards as the rest of the code base in that enterprise. So there is quite a bit of last mile engineering to do. So what we are building on the enterprise side is a Palantir for AGI. We built our platform that makes it very easy for us to build and deploy agents, which is highly model and agent agnostic. It’s actually agnostic at every layer of the stack. It’s agnostic at the layer of the cloud, the model provider, agent orchestration layer, memory. It’s very, very modular, which is helpful for the enterprises so that they can use the right models and agents for the right task.

Siddhartha Ahluwalia 1:08:13
Thank you so much, Jonathan. It’s been an amazing conversation. I learned a lot in this process of discussing brainstorming with you. My last question is, if you see the world different in 10 years, what would your prediction be for 2035 or 2036?

Jonathan Siddharth 1:08:40
Companies will look very different. Jobs will look very different. Education will look very different. I think it looks almost unrecognizable from what exists today. And I think it’s going to be a golden age of productivity. I feel like most of humanity’s most challenging problems are intelligence constrained in terms of us being able to throw more intelligence at it to solve the problem. But imagine a world where intelligence is so abundantly available so that we find cures to diseases faster. We’ve mastered some crazy interstellar travel. We’ve discovered new materials. I think it’s going to be an exciting future. When we look back 100 years ago, 200 years ago, sometimes you may wonder, how did people live without computers and all of these tools and gadgets we take for granted? I think this will look archaic.

So I think those three things will look very different. Enterprises will look different. Jobs will look different. And the education system will look totally different. And I hope maybe we’ve also extended human lifespan with better knowledge of human biology and better drugs and better elimination of sort of common reasons people die.

Siddhartha Ahluwalia 1:10:29
Thank you so much, Jonathan. It’s been an amazing conversation with you.

Jonathan Siddharth 1:10:32
Great. Thank you for having me.