Siddhartha Ahluwalia 1:05
Hi, this is Siddhartha Ahluwalia. I’m your host at Neon Show and Managing Partner at Neon Fund, a fund that has invested in some of the best enterprise AI companies between US and India corridor like Atomicwork, CloudSEK, SpotDraft. Today I have with me Rohit, founder of Portkey.

Rohit, welcome to the Neon Show. So lucky to have you today.

Rohit Agarwal 1:25
Great to be here as well. I’ve been a listener of the podcast and it’s, I didn’t imagine I’d be on this side of the camera. So this is pretty cool.

Siddhartha Ahluwalia 1:32
No, no, congratulations.

Now Portkey has become a default gateway for a lot of AI traffic to pass through globally. I think daily, what, 500 billion tokens pass through you?

Rohit Agarwal 1:44
Yeah, we’re now processing almost 500 billion tokens.

And this is growing like every week, there is a 30%, 40% increase in traffic flowing through the gateway. And that’s what we set out to do. So it’s almost like, can we capture a percentage of the world’s AI traffic through our systems and then start to add value on top of it?

So it’s been a fun journey.

Siddhartha Ahluwalia 2:05
Yeah. And you recently closed a 15 million round evaluation.

And like, we’d congrats on that.

Rohit Agarwal 2:11
Thank you so much. Yes.

Siddhartha Ahluwalia 2:12
So let’s imagine, we are explaining all the AI landscape, token landscape to a beginner in the world, right? So how would you share the AI model, AI world, AI tokens in very layman language?

Rohit Agarwal 2:28
Yes, sure. I mean, I like to think of the world in analogies. So if you think of compute, so you had a compute layer in the old world, then you had an operational layer on top of this compute layer.

And people would build applications on top. Now I’m super simplifying it, but that was like three layers in my head. I think with AI, it’s something very similar that’s happening.

So you have your large language models, which are becoming the intelligence of the compute of the system. They’re obviously a very different kind of compute because they’re intelligent. They can think, at least they have the perception of thinking, and they can do actions as well.

On the other side, on top of it, you have the operational layers, which is companies like Portkey, which is helping manage and operate traffic to these LLMs. And then people are building applications on top of it. Now, an application is, I think, where a lot of people are interacting with AI today, which is where ChatGPT or Claude or all of these come into effect. There’s a new class of these applications called harnesses, which are becoming very popular.

So look at Claude Code, Cursor. They’re essentially harnesses in a way that they’re applications, but they help you use the underlying models a lot better. And I think that’s how I define this type.

So users are mostly interacting with applications. A lot of these applications are now powered by Portkey, which is then using LLMs under the hood to sort of solve these AI first problems or solve problems in an AI first way.

Siddhartha Ahluwalia 4:06
And today, 24,000 organizations use Portkey in any form?

Rohit Agarwal 4:10
Yes. Yeah. So we’ve had a very successful open source product.

I’m quite proud that we were the first ones to call it an AI gateway when everybody else was calling it something else. We’re like AI gateway is the right term for it. And this is what people would use.

We’ve got a very large open source footprint as almost half a half a million installs of the open source system across companies. But then we have organizations that also buy into the commercial product. And that is a very wide base as well.

We’ve also been able to have a lot of enterprises that are now deploying Portkey. It’s interesting, a lot of these enterprises now run almost all of their production traffic through Portkey. And that was the goal.

So you land with an enterprise with one team, they start using you, they realize the value quickly, and we’re able to scale it up to like, let’s move 95% of your traffic through Portkey. And that’s when value just starts to exponentially increase to them. Because you have one place to manage control, get visibility on all your data.

Siddhartha Ahluwalia 5:15
And can you share the use cases why these enterprises are using Portkey?

Rohit Agarwal 5:19
Yeah, I think it’s two emotions, right? And we started with this thing.

So the two emotions people are very worried about in the AI world, especially larger enterprises is, I don’t have visibility on what’s happening. It’s a black box, it goes wrong, it hallucinates, it can cause problems, my agents can break a lot of these things. The second is I don’t have control on the system.

So who’s using how much, which team is actually finding value? If I’m going to spend $100 million on AI next year, where is this money going? What is my ROI?

These are the two key problems that’s plaguing people. And that’s when we started, we said, okay, either you can build your own systems internally for all of these individual things. Or here is a way where you attach an AI gateway in the middle, route all your traffic through it.

And then we become that tool that helps you one manage all of this traffic. So you know, route to different LLMs, connect to MCP servers, get visibility on this traffic, which team is spending how much? What is your accuracy like? Is your application improving or decreasing? Which team is finding the most value from claude code? So all of these use cases now become immediately apparent without you having to do anything additional on your system.

And I don’t think this is a new thing. This has always existed in the world that you had gateways and gateways have had like a value in itself. And we’ve taken that model and said you can do a lot more operationally with having a gateway in the middle, which is obviously routing and observability.

But then also this acts as your area for security. Where do your firewalls live? They live on the gateway.

Where does governance happen? Happens on the gateway. Where does compliance happen?

Happens on the gateway. Sort of tried to collapse all of these things into a single plane, which says this is a low latency, high value add. It’s like you insert it in your system, nothing changes.

But then you now have a lot more control and visibility on all your traffic. So that was what we started Portkey with. And we’ve sort of stayed true to what we wanted to do from the very beginning.

Siddhartha Ahluwalia 7:25
And where did you get this insight of starting the gateway?

Rohit Agarwal 7:29
Yeah. So we had been working on AI, both Ayush and I at our previous company.

So I was building AI at Freshworks for customer support. Then both of us were building AI for content marketing at Pepper. And during the course of building, it was fun building applications, but production was very hard.

Siddhartha Ahluwalia 7:47
Why?

Rohit Agarwal 7:48
You have these constant problems, like I’ll tell you examples, right? We’ll get user reviews of your AI doesn’t work.

And you have no idea why it doesn’t. Like it’s working for 90% of the people have great NPS, but then it’s not working for you. And I don’t know why.

I can’t even fix it. We had an attack one day where we ended up spending our month’s AWS bill in just one day on open AI, because we had a bot attack happen on us. And that was insane.

Like we couldn’t have done anything to it. And we were scrambling to add captures to our system, etc. But it wasn’t very effective.

Siddhartha Ahluwalia 8:24
You could, if there was a gateway, you could have just closed the gateway.

Rohit Agarwal 8:26
Exactly, right. But we did not.

So we were almost building our application, and then building these backend systems. So we were very heavy users of retool. That time, retool is no longer very popular, but we were heavy users of it, like in 2020, 21.

And we had built 40 applications on retool just to operationalize our production application. And then we’re spending more time doing this than actually building a product. So when the ChatGPT moment really happened, I think one thing clicked for both Ayush and I that the same way DevOps allowed enterprises move effectively to the cloud, you had a solid foundation, it allowed you reliability, you could move to the cloud, that same operational platform needs to exist for AI as well.

The same challenges. In fact, when I talk about control and visibility, look at Datadog’s old websites, or Cloudflare’s old websites, everybody was just about control and visibility that cloud doesn’t give you, we will give you. That same emotion, it was uncanny that we were using the same words that these companies had used 10 years back.

But then I think in tech, everything’s a cycle. So I think with those two core insights, there is the ChatGPT moment, everybody in the world is going to build with it. And we’ve seen real value come out of this.

Like we had completely missed the crypto wave, both of us just didn’t believe in it. And I think that’s still a fantastic world, just I don’t understand it. But on the AI space, we had seen real value that created, like this is not a fad.

Everybody will adopt it. I think we were still earlier to the market. But then we said we need this operational platform layer, the boring pieces, which will allow enterprises to move effectively to the cloud.

And we’ve, we’ve now seen that happen with over 50 enterprises that have come on to Portkey was struggling with their POCs. But then within the first month, you start that you see the usage start to scale up very quickly, because they’re not running into the operational daily hassles of running.

Siddhartha Ahluwalia 10:29
And who are the other players who are recognized within the market in the same domain?

Rohit Agarwal 10:33
Yeah, I think it started with open source companies. So there are like open source players, YC startups, so light LLM is very popular. I think lots of YC startups came about, Helicone was one.

I think that there was a time in 23, when a lot of companies were trying to do similar things, where they were saying, oh, we’d want to be Datadog for the AI world, etc. We were probably the first ones to call it an AI gateway and then go deeper. But now I also see large companies, everybody who has a DevOps play is building an AI gateway, like brain trust announced a massive funding down.

They again said, oh, we have an AI gateway now of our own. Cloudflare announced its own AI gateway, IBM has one, every API gateway company has it. So it’s become one of those core architectural patterns that people are like, this is standard reference architecture.

If I’m building my AI systems, I need my LLMs, I need my databases. I need my operational platform, which is the control plane, like Portkey. And then I need my application harnesses on top.

So this is almost like a very simplified enterprise AI architecture. But those are almost the three layers of it in production. In development is a very different set of stories.

But in production, this is what we’re starting to see emerge as the pattern that’s giving enterprises peace of mind saying, okay, now I can go invest heavily in AI and find value as well.

Siddhartha Ahluwalia 12:06
And why do you win in your category?

Rohit Agarwal 12:09
I think one is just our love for building a platform product.

Both Ayush and I come from that mindset of we just understand that mindset and we can keep going deeper and deeper into what does a platform really mean. That’s helped us in terms of being able to build a really performance product, being able to build a product where we can still push 30 releases a week, and nothing breaks because we build the backing systems to do it. So that mindset of how do you scale a platform is there.

And we really enjoy doing it. So I think that definitely counts for something. We’ve been lucky we’ve made more right gambles than the wrong ones.

Because in AI world, it’s like every other week, you have to take a bet on will this work or will that work? So I think we’ve been able to take some of those right bets through the couple of years saying, we’ll not build an SDK, we want to build a service. That was a very big irreversible decision.

Siddhartha Ahluwalia 13:04
What’s the difference between the both of them?

Rohit Agarwal 13:05
An SDK could be like, so you have the AI gateway that’s sitting, but this is now separate from your application. We could have also built an SDK that sits within your application, and then your application and every application can use it.

Instead, we said no, it has to be a centralized service, harder to sell, because now you have to sell it to the company and not to an individual team. But we’re like, no, this is where the market will move. And we were sort of like, so all of the SDK companies became gateway companies as well.

The second bet was that we want to stay on the production side of things not on developer side of things. Because we had seen the DevOps world that production companies make money and are and see just much higher adoption than developer focused companies. So that again, was I think a bet that paid off well for us that initially, we had slower adoption.

So 24 was slower adoption, but last year was very fast. And this year, we’re seeing it move even faster, because more and more companies are going to production, their production volumes increase much higher than development volumes. And that’s why that massive spike into 15 trillion tokens being used through the gateway.

Siddhartha Ahluwalia 14:19
And you control LLM endpoints. What are LLM endpoints?

Rohit Agarwal 14:24
So an LLM endpoint is like any other API.

So this is an API request that you make to OpenAI. And then OpenAI is giving you a response back. So that’s the standard chat completion endpoints.

So that is an endpoint. So that is the API endpoint, I would say. Now, this is obviously gotten more and more complex over time, because these APIs have thinking, they are streaming responses back, there is voice to voice, there is multimodal, there’s images being generated, all of this stuff.

So that is where it starts to get even more complex that how do I not just do text, but I want to do more than text, I want to use PDF files, I want to use documents. Now, how do I do this with the same level of simplicity as a chat completion request? That’s where I think Portkey really comes in handy.

That’s one thing, one API, and it will work with everything else.

Siddhartha Ahluwalia 15:21
Got it. So do you believe that LLM Ops Infra would be a bigger opportunity than DevOps and at what scale?

Rohit Agarwal 15:29
I think it is almost to say will AI be a bigger opportunity than the cloud? I feel yes. Because it is almost like the cloud is a place where you’re hosting applications. But with AI agents, or the agent, these harnesses, they are going to dominate a lot of work happening in a business.

And if that starts to happen, operational management of these agents is definitely a big thing. I can look at all of the large companies, all of the enterprises, both customers and vendors, they’re preparing for this agentic future. Because you see it happening, like even at Portkey inside my small company, I can see the massive productivity gains we get by just deploying agents the right way.

And we might be a little bit ahead, but then everybody is going to do this within this year or next year. So will AI spend surpass cloud spend? I 100% believe that, which means that the AI ops spend surpasses DevOps spend, it’s bound to happen.

Siddhartha Ahluwalia 15:25
But today, the overall, I think cloud revenue globally is like AWS is north of 100 billion. And combined GCP, Azure, everybody else would be another 50 to 100 billion.

And what are the AI spend today globally?

Rohit Agarwal 16:45
I would say just the AI labs might end up doing more revenue than this over the next two years.

So can Anthropic or OpenAI be their own $200 billion revenue companies? Totally.

I would imagine cloud and this is all thinking hypothetical, I’ll probably come back in three months and tell you something different. But I feel cloud vendors will be the compute providers for the LLM labs. And then the LLM labs are definitely going to make a lot more revenue, because they’re using compute the right way that people expect to happen.

Siddhartha Ahluwalia 17:20
So today, you know, there’s a lot of uncertainty in the market on what cloud is doing. Where do you think it stems from? How much of it is real?

How much of it is hype?

Rohit Agarwal 17:33
I would say a lot of it is very real. So and I’ll tell you where the hype factor is also coming from.

So right now, I don’t think everybody has clearly understood how can they harness an LLM properly? So how do I use it properly?
Siddhartha Ahluwalia 17:48
Can you tell the meaning of the word harness?

Rohit Agarwal 17:50
Yeah, sorry. So like a harness is the so the framework that can use an LLM effectively. So you can start with just one LLM request.

But the LLM request can also take actions. So then you define a software function that says, oh, if the LLM responds with a message, okay. If the LLM responds with take an action, then take that action.

But after that action is taken, run this in a loop. So that’s the simplest harness, which is make an LLM call, make a tool call, and then make the loop again, which is just a continuous loop. And that alone is so much more powerful than just a simple Q&A.

Because it’s now taking actions, it’s telling you next things to do, it has context and all of this. Now, people who are able to use this effectively, are finding exponential value from this, which is why when you say open claw is just killing the market, and people are creating autonomous employees from it, I think it’s happening in real life, because people are being able to code it in that way that they use the intelligence and use it for their own purposes. It is all hype, I think that’s coming from people who are not, who are still in the Q&A mode, who are still like, I’m going to ask it something, and then it’s going to give me an answer.

And for a variety of reasons, there are people who don’t want to give it more control.

Siddhartha Ahluwalia 19:16
Yeah.

Rohit Agarwal 18:12
But I think you have to, it’s a mindset shift once that happens, saying I will turn on agent mode, and I will see what happens. And it’s okay if things break. I think that’s when you really start to see that you can do things that used to take hours, now can be done in minutes.

Siddhartha Ahluwalia 19:34
One of the companies that we invested in Sagepilot, their ex, they worked with you.

Rohit Agarwal 18:34
I know Prashant.

Siddhartha Ahluwalia 19:40
Yeah. I think they were one of the first companies that we invested in, that said, they’re building autonomous AI employees.

And all that time, like this was six months ago when AI employee was not a buzzword. And today, everybody’s starting to, what do you mean by AI employees?

And specifically autonomous AI employees who can execute tasks on their own, can take decisions on their own and probably with time can get better than L1, L2 employees.
Rohit Agarwal 19:03
100%.

I think that’s happening already, right? So if you think about frontline employees, who are like, you’re given a task, and this is a defined task, and you’re completing that task. Especially if this is knowledge work, LLMs have already gotten to a point where they are intelligent enough to do a lot of this knowledge work themselves.

So if you again, pull the right Lego blocks, saying this is my LLM, this is my memory store, which is my database. This is the way I want to run it. This is some set of rules you should follow.

Then you merge all of this together. And this essentially can start completing tasks for you. And an autonomous system that can complete tasks for you is equivalent to an employee.

So I wouldn’t say this is an employee. But I think as a term, it’s easy for people to explain saying, hey, you have 15 tasks to do, I have to fix five bugs, let’s take engineering, I have to fix five bugs, I have to create three new features, I have to review two new PRs. 80% of this work, I think can be automated with AI.

I still believe you need the last 20%. So there still has to be a manager of these agents who’s telling you, okay, do this, don’t do this. But outside of that, 80% of the work is still getting done fast.

Siddhartha Ahluwalia 21:25
And how fast, let’s say one example that when we were speaking to customers of SagePilot, or one of the customers of SagePilot, 50 crore revenue company based in Jaipur, they do clothing, clothing.

And they mentioned that they have reduced the team of 16 people in customer support and customer life cycle to three people. Amazing SagePilot. And then what has happened is, one of the customers just walked into one of the stores of that company in Jaipur and said, I want to talk to Rhea.

Because she provided me amazing support. And you guys are not able to understand me on floor. Please make sure you know I want to talk to Rhea.

Yeah, there is no Rhea.

Rohit Agarwal 21:04
Correct. Yeah.

I think that’s going to happen. I’m sure it’ll be interesting for SagePilot to probably have an instance of Rhea in store as well. Which is like, hey, you know, continue your conversation from..

Siddhartha Ahluwalia 22:19
In store?

Rohit Agarwal 21:16
Yeah, exactly. And I think, in some cases, especially with support, etc, the amount of context and LLM request can store in its context or in its memory is much higher than what you and I can do. So yes, we might be great in one of these tasks.

But just holding things in memory saying, you like me talking to you like this. These are your preferences. This is what I’ve answered before.

And now when you ask me a question, I can give you the exact right response. I think half of it is kudos to SagePilot to be able to build that really well. Half of it is also that AI today is capable enough to deliver on these things.

Siddhartha Ahluwalia 23:02
So as a gateway today, right? How big you think can it get in the next three, four years?

Rohit Agarwal 22:08
I’m thinking our goal is that can we get into significant percentage of AI traffic flowing through our gateway?

Siddhartha Ahluwalia 23:22
So how much does AI traffic flows globally, every day?

Rohit Agarwal 22:20
I mean, that’s very hard to estimate. But I would assume we are probably 0.5% of total AI API traffic today. Totally, our traffic might be higher, but we’re looking at AI API traffic, we might be like 0.5% of that. I think by the end of the year, my goal is how do we get to double digit percentages? It’s very doable, because all of this traffic that’s not currently moving through a gateway will need to move through gateways because you need, again, control, visibility, security, governance.

And if we build those right, and if we build the best architecture in which people can deploy it easily, and manage it and control it, then it makes sense. So I mean, that is where I’m like in terms of revenue, etc. I think the secondary story for me, we were discussing that revenue is a lagging indicator, it will come like if I’m providing value, it will come.

But traffic is very important to me. So I think we were probably doing a billion tokens a day a year back. Now we’re doing 15 trillion.

So that’s like a massive jump. And I see another jump happening this year. And it’s not just because we are an amazing company.

But it’s AI adoption is just growing that fast, that there’s very strong winds, strong winds for people to have something like Portkey as well.

Siddhartha Ahluwalia 24:44
And what do you think? Let’s say there’s a lot of talk around AI bubble also, what do you think?

Can it cause a bubble or something negative in the economy?

Rohit Agarwal 23:50
I think it’s everything. I like the Gartner hype cycle, right?

So everything has a hype cycle. And you had vector DBs that were like, oh, this is amazing. And right now vector DBs are like, this absolutely doesn’t work.

So don’t use them at all. So everything goes through that. I think AI has also gone through that multiple times and different, different things.

But there’s always been something new that’s come up, which is exciting and has completely changed. I feel that today, even if all innovation were to stop, there is enough fuel in the AI space to see you can still continue to pick up these Lego pieces and still keep building newer and better systems. But innovation isn’t stopping still, there is so much money that’s been pumping, so innovation will happen.

Now do valuations flicker over time can happen, because there’s so many things happening that the valuations can flicker, but I wouldn’t expect, I think this comes from personal belief, I have seen it create real value, I have used it internally to automate my life away. And if there’s value in work, then there is value in AI. So that’s how I’m equating.

So the bubble might be on the valuations, the bubble is not on utility.

Siddhartha Ahluwalia 26:07
How have you automated your personal life?

Rohit Agarwal 25:05
I have like, from the very beginning, we have been very much like I want to have a small lean, lean team.

So a lot of my tasks today in terms of, I think one annoying thing for me was sales follow ups. Like there are just these tasks, which I have to do. And they’re very important, like the customers waiting on me to send them a deck or to just reply to my questions.

And I’m probably having 15 of these calls in a day, and I’m just missing them. And it’s annoying for them. It’s annoying for me.

And I was going into this good cycle. So that’s where it started saying can I start automating follow up? They’re based on my meeting notes, which creates tasks, and does it for me automatically.

Slowly from then on independent to hey, can I start solving software bugs through the same assistant as well? That started happening. So my team started seeing me making more commits.

And they’re like, how are you on calls and still making commits? Because it’s automated, like it’s doing those commits, it’s running those checks, I review it and I’m committing code. We’ve started to use AI a lot for our contract reviews.

So I work with our team that’s reviewing contracts and turning around faster. Because we’re able to review contracts faster, we’re doing security questionnaires faster, because we have now a repository of all our answers. And we’re filling questionnaires from there.

So I think almost every part of the job, I’m trying to see how can this become faster with just AI. I was talking to like, we hav she was struggling with this automation where you know, in compliance, you still have to download a document, upload the document, say this is now completed, change dates, etc. And we were toying around with it saying you can take you can complete this in an hour, but it’s going to be boring.

Or let’s try to use Claude’s Chrome extension to do this. It took us an hour, which we knew that this is going to take the same amount of time as that. But then finally, it was just so fascinating for her that this extension once I tell it what to do, it’s just doing the right clicks.

So I’ve given it a set of instructions saying here are all the files, this is my compliance list completed. And it’s making the right clicks, it goes to the tab, clicks a button, uploads the document, says okay, this is saved. So okay, this is done next, next, next.

And she can be on a coffee break. Or she can be doing something else that’s more meaningful. While this operational mechanical job is being done by the extension.

Siddhartha Ahluwalia 28:39
So, so today you are supporting 1600 models. So how does you know, like the world works with your people choosing I want this model versus that model?

How does the trade off working? And it’s like, is it following Pareto principle, like 95% of the volume is the top five model?

Rohit Agarwal 27:57
More or less, yes.

The top five changes very quickly, which is what we..

Siddhartha Ahluwalia 29:05
What is the top five?

Rohit Agarwal 28:02
I think today top five would be. And I don’t know when this releases. But I think right now top five is probably Opus 4.6, Opus 4.5 sonnet models.

For coding, we see GPT 4 codex, sorry, 5.3 codex. In the open source world, minimax indicate who are popular, but I think there is newer models that keep coming up. But that is the broad set, which is like top of mind for us.

These keep changing very, very frequently. In fact, we launched a ranking website, right outside our office, we have this board of like a stock ticker, we have these models and their tokens as a ticker. Let’s say that which model is trending right now?

How many tokens?

Siddhartha Ahluwalia 29:47
On Portkey?

Rohit Agarwal 28:43
On Portkey.

Because we now are I mean, 15 trillion tokens. So we have a real subset of enterprise production traffic. So I think, okay, enterprise production traffic, we probably have a fair enough sample size of the world’s traffic pattern.

So percentage wise, we can see what’s happening. And I think with Opus 4.5 and Opus 4.6 were the first models we saw the uptick in number of tokens was just insane. Like forever, you would have GPT 4.0 mini or GPT 5.0 mini as the leaders in tokens, not in cost spent, but in tokens, because these were the workhorses. For the and these are also cheap models. For the first time, we saw 4.6 Opus, which is the costliest model there is right now, is at the top of the charts in terms of number of tokens. Yeah, which is another indication of people are no longer worried about cost of AI. So they’re worried about return of investment, which is like, okay, I’m happy to spend $200 $300 a month on claude code. But the value that I get from it is much, much higher.

So it’s no longer about I want $20, or I won’t spend more than $25. It’s about, oh, can I augment my employee with another additional $500, which is probably 10% of their salary and then do so much more with it.

Siddhartha Ahluwalia 31:07
Which part of the world you are seeing the fastest AI adoption?

Rohit Agarwal 30:05
Right now, it is for sure, software development. Like software development for sure is the space where maximum adoption is happening. claude code has driven a lot of that, claude code, cursor, open code, all of these IDEs. That’s the top right now, but I can start to see this seeping into every other space.

Siddhartha Ahluwalia 31:32
And what are the next second, third, fourth?

Rohit Agarwal 30:29
I would say, so operationally heavy tasks, so legal, finance, uh HR. These are three.

So, operationally heavy tasks. So, legal, finance, HR. These are three where I see there are lots of things happening.

Obviously, customer support. I think even before programming, customer support was like the first target.

Siddhartha Ahluwalia 31:49
So, customer support was already the first?

Rohit Agarwal 30:46
Customer support was probably the first. I mean, if you see companies like Sierra and Decagon and all of these, they were built on the Q&A world, which is even if you want just answers, the AI models are great at it. So, I would say customer support, customer success, these came even before and even content generation.

So, content generation, customer success, customer support were first. Programming, software engineering, second. And now we’re seeing, so legal was also there, but I think now legal is adopting it really well.

HR, finance is adopting it really well. And I think after this is like every other space. I don’t have that much insight into manufacturing, etc.

But I see our customers use it very heavily in that.

Siddhartha Ahluwalia 32:36
Then my next question would be, what are the domains where you see the most usage on Portkey? Like, for example

Rohit Agarwal 31:37
Yeah, I VC, obviously, again, software development, engineering teams using it is the highest.

Knowledge work is the second, which is large consulting teams or large business teams that are preparing proposals, etc. So, all of that is we see that happening quite a lot. We see some game development happening by AI to Portkey.

But I would say more or less on Portkey, the way enterprises are operating today is you probably start with one or two teams, which are usually engineering teams. But then it just is like everything on every AI use case within your organization is going to move into Portkey.

Siddhartha Ahluwalia 33:24
So today, globally, if you think macroeconomically, right, what’s going right for AI companies and what’s not going right?

Rohit Agarwal 32:32
In my opinion, I think just the fact that an LLM model can understand what you want to do and give you an answer tailored to you is I think going very right for them. And that intelligence is there at multiple levels. So you can add more and more structure on it to really get that intelligence out.

So that is what is causing all of this to happen at very massive space. Like when OpenAI says that we want GPT 5 to be building GPT 6. In reality, that is happening.

We’re seeing software being written like that. There’s so many companies coming out and saying 80% of all of my code base today is written by AI. And that is true.

And that’s what we’re seeing everywhere as well. So I think those things are going very right for them. I think where everybody is now fighting for is, what is next here?

How do we append the current incumbent model? And what are new use cases we can take this into? I’m sure at an underlying level, there is a data center war and electricity war and a GPU war as well.

But again, I don’t have visibility on that.

Siddhartha Ahluwalia 34:46
I think where in my view, what disturbance some of the AI companies have caused macroeconomically is because they don’t want their underlying cost models, like the cost structure to be revealed publicly. They keep on growing and growing and they don’t want to go public. That’s my hypothesis.

Rohit Agarwal 34:02
Yeah, I think you might be very right. Even the application layer, your margins may not be great or are great.

But it’s very opaque. So we don’t know are these good businesses or is it just good technology?

Siddhartha Ahluwalia 35:22
So then what’s happening is anything that Claude drops today, because people don’t have visibility into Claude, but they have visibility into public companies.

Even if ServiceNow is growing, they’ll just dump ServiceNow stock.

Rohit Agarwal 34:32
For sure, for sure. Claude launches a security plugin and then security stocks are down.

Siddhartha Ahluwalia 35:43
Which doesn’t make sense today. It’s just not that everything that Claude is launching is going into production.

Rohit Agarwal 34:46
Yes.

Siddhartha Ahluwalia 35:52
And they’ll keep on launching because the pace of the company is going so fast. So everything like today, cloud launched something on HRTech.

Rohit Agarwal 34:54
Yeah.

Siddhartha Ahluwalia 36:00
And they’re saying they’re slashing the prices of Workday. It doesn’t make sense to me. And this is what I think is causing macroeconomic disturbance.

Rohit Agarwal 35:05
Yes.

Siddhartha Ahluwalia 35:05
And a series of these macroeconomic disturbance just compound so fast. I think that these have led to bubbles.

Rohit Agarwal 36:19
For sure. Yes. I think the good thing and from a pure product technology perspective, I like product technology. But what’s happening is that ServiceNow and Salesforce and Workday and all of these companies are being pushed to innovate.

Like in other cases, they’re like we have large go to market motion. There’s a revenue that we can easily protect and you don’t have to. But because they see these drops, which are like 20% drop in your stock price because of an announcement.

So that is just forcing them to say, we have to look at this agentic way of doing things. And I think in general, it’s just a better way. Like nobody likes going to work there.

Nobody likes working in Salesforce. And I think nobody knew how to even fix it because over time your systems do get complicated. But I think these are ways where you can have agents take away some of that complexity and just do things in the background for you.

Siddhartha Ahluwalia 36:09
You might be right there because everything that Claude drops tends to make the world more efficient.

Rohit Agarwal 37:20
Yes.

Siddhartha Ahluwalia 36:16
Somehow the inefficiency which gets controlled by stock price tends to improve.

Rohit Agarwal 37:28
Yeah. I think stock price has become more of a forcing factor for teams to relook at their strategies, which is good. I think people are going to, this is going to force them to use it more.

Also helps Anthropic in another way that every time, obviously cloud adoption can increase, but their API adoption will also increase.

Siddhartha Ahluwalia 37:47
Yeah. And I think specifically what I like about the AI is, the best of the founders are technical people at heart.

Like earlier in SaaS phase, like you built in SaaS phase, right? Your first company, Framebench.

So you would require a very strong go-to-market founder paired with a technology founder and somehow the go-to-market founder would override that technology founder. That is not happening.

Rohit Agarwal 38:14
Correct.

No. I think this time it is purely, you can have tech founders working with go-to-market teams. Although on the flip side, the other thing is still that because building software has become so cheap, you need strong product thinkers or systems thinkers, rather than just strong executors.

Siddhartha Ahluwalia 38:33
Yeah.

Rohit Agarwal 38:34
And on the other side, you need somebody who can understand distribution well, maybe they’re not great salespeople, or maybe they’re not great marketers, but they need to distribution really well. So, I mean, we just had our first VP of sales and right at the outset, he’s about how can we use systems to change the way sales is done. And I think that’s how almost every other good sales leader is thinking that it’s not about the typical AI math anymore.

It’s about how do I get distribution fast enough? How do I use AI in my systems? I think that is where it’s sort of moving.

Siddhartha Ahluwalia 39:09
But let’s say it’s only possible in AI way today that you hired your first VP sales only after having 15 million and crossing X digit in ARR or whatever the number is.

Rohit Agarwal 39:21
Yes. Yeah.

Siddhartha Ahluwalia 39:22
So here it was not possible?

Rohit Agarwal 39:23
Not possible. Yes.

I mean, founders being able to do sales to this extent, yeah, wasn’t as possible earlier. Now you have so much help with the different AI tools. And you can learn so quickly about what is a quote?

And how do you do a proposal? And how do you pitch to an enterprise? And how do you review their NDAs?

Otherwise, these were usually I could imagine these were like multi week processes. But now it’s about, I get a response from a customer and I have no idea what they’re talking about. Like Claude tell me about this.

And then probably, I understand what they’re saying. And I can respond to them.

Siddhartha Ahluwalia 39:59
So, what do you think the edge Claude has over other models that has been able to move so fast and not just fast, but with the right direction?

Rohit Agarwal 40:11
Yeah. I think it’s just one of those things that maybe they’ve gotten the tone very, very right. I think this is talked about quite a bit amongst others.

I don’t think other models are any less capable. But it’s just that the maybe from the training data, etc. But the way it starts to interact with you, you just feel more confident in its response.

And the way it’s understanding what you’re trying to say. I think that’s been the biggest win for Claude overall. Otherwise, everybody’s still saying that Codex is a better model for code generation today.

But people still use Claude to think, brainstorm, plan. And then once the plan is ready, it’s like, okay, now Codex implemented. Codex is much better at writing code.

So, I think these niches will keep evolving, but this model is slightly better at this versus that. But that very natural human feel that you like, that I think Claude has just been able to crack really well.

Siddhartha Ahluwalia 41:07
And in your opinion, has Claude evolved or is evolving so fast that there was a parallel market created for white coding? Is it eating into that?

Rohit Agarwal 41:15
100% Yes. I think Anthropic is the reason why coding even exists today in the world.

Siddhartha Ahluwalia 41:20
But now it’s killing that market also?

Rohit Agarwal 41:22
It is killing that market. I think there is that market is now finding its own niches.

And Anthropic clearly realized that if 50% of my API revenue is going to come from white coding, they will launch code and be like I can still build the best white coding system there is. I think which is right now Claude code is killing everything else.

Siddhartha Ahluwalia 41:40
Do you think then it’s a risk for founders to expose everything to Claude because learning so far from your own systems and best practices?

Rohit Agarwal 41:48
So, it’s almost impossible to do that in a way. You want to use the latest greatest models because your application can’t be worse off than the next person. Which means either you become your own AI lab, which is very hard to do in today’s world.

So, you have to rely on an open AI on Anthropic. So, I think it’s almost an existential threat forever that whatever you do can be copied by these labs. But it’s the same case again, AWS could always copy your product.

Or if you’re in a Salesforce app store, Salesforce could always roll in your product. So, just about can you stay ahead of it and find your niche in which they don’t want to get into or they can’t get into and then keep growing there.

Siddhartha Ahluwalia 42:30
For example, Anthropic dropped, you know, some APIs and you know, specific use cases for legal.

And then suddenly, all the Harvey’s and liberals of the world.

Rohit Agarwal 42:43
Yes, yeah. I mean, that’s bound to happen.

Harvey is now trying to build its niche and I’m guessing which is also right for everybody else’s. I can look at your data specifically, I can tune for your brand for your firm. I think that is where these teams are trying to go in deeper.

There. Yes, you can buy a generic product. We are using the same APIs under the hood.

So, it doesn’t really matter. But we can then provide a layer on top which Anthropic can never do.

Siddhartha Ahluwalia 43:15
And why is that? Why Anthropic can never do that?

Rohit Agarwal 43:17
I mean, in their top three priorities, killing Harvey will not be their top three priority. I mean, they’re like, how do I make the next 100 billion in revenue?

Yeah, yeah. So, I mean, that is their thought process. So, then Harvey can always be like, okay, then my top goal is to build the best legal AI system. So, then that’s how Harvey is.

Siddhartha Ahluwalia 43:37
Don’t you think today every startup for VC dollars is competing with Anthropic? Because then a VC can argue Anthropic can still go 10x.

Rohit Agarwal 43:48
Yes. Yeah, absolutely. Right.

And I’ve heard this in conversations where a VC would be like, do I invest in startups, which is like a risky business, and I’m still getting a 10x return? Or do I just put my money in Anthropic and I’ll just get a 10x return?

Siddhartha Ahluwalia 44:05
Without doing any work.

Rohit Agarwal 44:06
Without doing any work. So, totally true. And I think founders probably need to have a story on why me versus an Anthropic now.

And I think that’s the story we’ve had to tell earlier in the marketing, why me versus not just the stock market. Like stock market is going to give me 20% return year on year, and you’re promising the same, then I don’t want you. The startups have always said that, oh, I can do 10x and 50x and 100x.

And this is the vision for it. That’s the same story people are having to do now just against an Anthropic.

Siddhartha Ahluwalia 44:41
And so, AI has also taught us that everything has become fungible.

Like yesterday, Cursor became a $30 billion company, and suddenly it’s lost. I don’t know anybody today that is using Cursor. Every developer is switching to Claude. What is happening?

Rohit Agarwal 44:58
I mean, these are those inflections that will continue to happen. And it’s very rocky seas.

Like we are damn in the middle of all of these seas, and we see these big giants growing really, really quickly. And then suddenly something changes, and then they have to re-innovate. I think that’s just the nature of the market right now.

Till there is no stability and calmer waters, which I don’t see for the next couple of years, it is going to be like this, that teams, you can only choose to grow really fast, and then see what happens. But ultimately, yes, you are at the mercy of a lot of the market factors that are prevailing around you.

Siddhartha Ahluwalia 45:37
So currently, the consumption that is happening through Portkey, can you tell us which top three geographies are the highest?

Rohit Agarwal 45:44
I think I wouldn’t know exactly those numbers. But I’d say US is by far very high for us. I think also our majority traffic currently comes from US because US started productionizing AI a lot sooner.

And I would say even in that, teams in SF did that a lot faster, just because of their proximity to an open AI engineer or an anthropic engineer, where they’re able to exchange information quickly and build production ready systems faster. So that happens, I think, I would say most of our traffic is US. We do start to see a lot of traffic from India as well.

But I think India traffic is still a little bit behind in terms of quality of traffic, which is, are you using the latest models? Are you moving traffic very quickly? I think I don’t think that is happening, except for some of the main companies that are coming across. We see companies in Australia doing really well.

Siddhartha Ahluwalia 46:39
So your would be US, second India and third Australia?

Rohit Agarwal 46:42
I would almost, third would be probably rest of the world because it might be so smaller percentage.

But like just anecdotally speaking, we’ve seen Australia companies just think about this really well. Like some of the largest banks in the world, the largest fintech companies in the world, they’re really thinking about what does an AI first bank look like? And that is very interesting to see how that’s happening, not in mainland in San Francisco, where Silicon Valley Bank is doing this. But like, you know, the largest banks elsewhere in the world are also in a meeting.

Siddhartha Ahluwalia 47:16
Then what are the modality? Right? Is it what numbers voice comes in?

Rohit Agarwal 47:22
Voice for us has not been very high.

Siddhartha Ahluwalia 47:25
Why?

Rohit Agarwal 47:25
But I think that just because we started as a text gateway first, and then we did voice later on. And voice is one of those things that’s growing very fast for us. So real time API’s, live API’s, voice to voice, that is a space that is growing very quickly for us.

I feel in India voice has taken off a lot faster than text. In other geographies, text and other modalities still dominate. But I’ll say this year voice will definitely take over in a big way.

Siddhartha Ahluwalia 47:53
Okay. And what do you think about, you know, then opportunity for startups? And the question is, whatever you build, will be small as compared to the larger labs.

And every day a founder will get a question that, you know, you might get eaten up by the larger models. So where does the MOAT come in? Does it come the application layer?

Does it come that infrared layer like you are?

Rohit Agarwal 48:18
Yeah, I don’t really know. It is.

I tell you, for us, the way we think about it, and maybe you might have better advice here. But I would say the way we’ve thought about this one, there has to be a core belief that you can stick to. And you can enjoy building in that space without really worrying about, oh, there’s 10 other companies that can come disrupt us.

Like, hey, we’ll navigate that journey. We are small, obviously, but we can focus, solve this one single problem really well, because we enjoy it. Having that insight, enjoying that problem, I think helps quite a bit.

Distribution helps a lot. Like thinking about what is your tactical market entry? How do you grow your base?

How do people discover you? I think that is becoming more of a MOAT than anything else. Because if you don’t have distribution, then you really have nothing.

Building software, as I said, is like the cost of building your software is zero. But thinking about the right things, taking the right trade offs, and then having a good distribution plan in place. I think those three are probably the biggest MOATs that people have right now.

Siddhartha Ahluwalia 49:20
In my view, where a startup can at least have a path to 100 million in revenue, and this is still theoretical, it’s yet to get played out in practice, just from observations from our portfolio company executing is institutional knowledge in your domain will surpass what models been built for that domain. Because the institutional knowledge in your domain, for example, let’s say legal, or pharmaceutical or construction is much more nuanced. Then, and as I said, right? Anthropic’s goal is not to become the best construction tech company.

Rohit Agarwal 49:58
Yes, I agree. Yes.

Siddhartha Ahluwalia 50:00
And also what happens is, as these OpenAI and Anthropic partner with let’s say, let’s say McKinsey or Accenture for delivery. So you don’t know who’s the person leading in both these organizations. And these people would eventually jump to some other ship.

Rohit Agarwal 50:19
Yes, that’s right.

Siddhartha Ahluwalia 50:20
So, for your institutional knowledge, your network with your domain is what you can really compound.

And a lot of the providers in those in these domains, it’s the pharmaceutical domain, right, are fearful of large labs also. So, so they would want to maybe just buy, you know, they would want to still give a contract to OpenAI and Anthropic, but they will not place all their bets on them because they might fear that tomorrow, Anthropic could become a pharmaceutical company.

Because building a drug is also intelligence.

Rohit Agarwal 50:59
Yes.

Siddhartha Ahluwalia 50:59
At the end of the day, if they have access to the same salt, same paper, same researchers, why can’t then Anthropic to grow a stamp, start a drug development arm.

Rohit Agarwal 51:12
True. And just acquire a company that does drug development and be like, okay, now we’ll compete with everybody else.

Siddhartha Ahluwalia 51:18
Everybody.
For example, top five, think of five top categories in drug market, let’s say fat loss or Ozempic like category and just ship out faster, better.

Rohit Agarwal 51:31
Yeah, could be totally.

Siddhartha Ahluwalia 51:32
We have better intelligence.

So, larger companies would be fearful of them. So, they will not hand over everything to them. So, that’s our belief and thereby a founder’s ability to become the go to voice in their industry becomes critical.

Rohit Agarwal 51:53
I agree. Yeah. And I think a more subtler point you mentioned that I’ll bring out is that as founders of early stage companies, you can be a lot more hacky in these things.

Siddhartha Ahluwalia 52:04
Yeah.

Rohit Agarwal 52:04
You don’t care about OpenAI copying you or ripping you off. You’re like, okay, there’s already 100 companies that are going to rip me off, doesn’t matter. So, you can be a lot more scrappy, hacky in your way to get there.

And I think that is a very big advantage that larger companies don’t. Like, I as a founder can be sitting here talking candidly with you about everything that’s happening. I’m sure if somebody is a billion dollar company, they have to have media training and be like, you can say this, but you can’t say this, you can’t be controversial.

Although in the age of Twitter, I don’t know how it goes, but that’s a big advantage. As a younger founder, you can just grow a lot faster. And I think as we start to hit that 10-15 million revenue, is when you will have to really start thinking about, okay, what is now my path to get to 100?

I think even then it’s probably faster, easier as a founder to do it versus a larger company, pivoting their company to doing this now.

Siddhartha Ahluwalia 52:59
Yeah. And second thing is that, let’s say if you are getting today, I think since you have proved and you are now processing 500 billion tokens per day, but earlier in your journey, you would still be getting questions, what is Anthropic builds a gateway? What was your answer to that?

Rohit Agarwal 53:18
I mean, we still get it, right? What happens if AWS builds a gateway, Anthropic or OpenAI builds a gateway?

Again, I think initially our answer used to be that this is, do you want to capture the 5% revenue or do you want to capture the 95% revenue? Most of these larger companies were like, we’ll focus on the 95% revenue, which let us go the first two years, which is like nobody’s, yes, NAI gateway is really important, but what’s more important? Should I build my own LLM or should I build an AI gateway LLM?

Obviously do that. Today, it’s just about we’ve built depth in the space quite enough. And we’ve built enough cross provider support, which people really value.

Siddhartha Ahluwalia 53:59
What do you mean by cross provider?

Rohit Agarwal 54:00
We have like large enterprises that want Portkey and one big advantage is I’m not logged into a Bedrock or a Vertex. Tomorrow, if I want to switch my traffic from Vertex to Bedrock for whatever reason, on Portkey, I just have to come in and switch a toggle and Portkey will direct all of my traffic to another provider.

Now that enables them cloud neutrality or AI neutrality that tomorrow when OpenAI launches its latest and greatest model and Anthropic, I need to shift my models there. I don’t need to re-architect all my systems. I’ll change my prompts.

And I’ll come to Portkey and be like, hey, Portkey, send my traffic to this new model. And that’s it. That leverage is what I think people like that I now have control over my own systems.

Tomorrow, if not claude code, everybody starts using some other coding product. Then do I build the same set of guardrails there? That’s all narrowing to enterprises.

Now, because all of that’s built in this layer, as long as we keep up with the AI chaos, the enterprises are sort of protected. That’s where I think that’s where we’ve won. I don’t think it’s because of anything else.

We’ve built a solid simple product that helps enterprises just take care of that chaos. Yeah, but that’s the longer answer. I think the shorter answer is just that focused on this space only.

And we’ve told people that if you’re only relying on one cloud, one AI provider, you don’t need us. But if you have a strategy that you want AI to be the centerpiece of your company, then now it’s reference architecture for you to have an AI which needs to be neutral and needs to be out of order.

Siddhartha Ahluwalia 55:45
So, you know, according to you, what are the things that you predict will happen in the next 12 months that possibly become your product or GTM roadmap for the company?

Rohit Agarwal 55:59
So, I’m hearing almost the software world is getting divided into two. Either you can be an AI lab, which is the intelligence, you’re building models, you’re doing new stuff, or you have control over this traffic so that you are doing the harness, application, ops, all of this. So, this is all I think, where the world is going to move into.

And this is where the two places of investment are going to happen. For us, just being able to sit in production traffic, and prove that we can be in production traffic for some of the largest companies is already confidence enough to large companies. Now, our goal is that what do we start to provide on top of this traffic back to you?

What Cloudflare did back in 2012 or 2015 was, I see all of this traffic, I can identify security incidents happening on one area of the world and solve it for everybody else on my network. So, every time I’m getting more traffic on my network, everybody benefits from that. I think Portkey is we have a similar thing there.

And for us, now it’s just going to be about if we have traffic, we keep increasing traffic. And we just we are about to launch our enterprise gateway 2.0 in the open source world, which is like if you’re running claude code on your machine, you can drop our tiny gateway daemon on your machine. And it has the full capabilities of Portkey without actually needing a server to deploy etc.

All connected. Now, these are things that help us grow traffic, but then add value on top of it to say, okay, let’s take more of your operational stuff. So that is what is very exciting to me.

That if you’re if an enterprise is making a billion API requests a month, this is amazing data that you’ve collected. Now, how do you use this data is going to be very interesting. So like we’ve heard cases of people saying, I’m correlating my claude code data with my GitHub data, and finding out team productivity, and finding hacks that I can share with the rest of my organization.

And hey, if you do this, you write code better, faster, cheaper. So then everybody use that in that intelligence on usage patterns will be super useful. Now, it’ll be interesting as I say it, I’m thinking AI labs would love to get this intelligence and build it into their systems.

And it’ll happen on both layers. I would want my own intelligence systems here. And then the AI labs will keep improving the models.

Siddhartha Ahluwalia 58:29
So do you believe a world exists where the current SaaS companies cease to exist?

Rohit Agarwal 58:38
In the current form, they they, in my opinion, they should cease to exist. Like I shouldn’t have to go to Salesforce and then click on feed, you can enter something and then click Save.

And then there’s a loader and then it saves it. I think that world is over. This year, next year, you are going to be like, spoke to this person, this is what happened, update my record.

I think that is the world that is like very real, will happen. It’s interesting to see what happens beyond it. Maybe there was a note taker that already knew it and updated the record.

And it’s giving me signals on do this. So I think that is the world we’re moving towards. In the current format, SaaS companies will just become systems of records.

So you had the database, which got obsolete because of a SaaS platform, which is just crud on this database, but interesting thread on the database. Now SaaS platforms, I think are the new databases, which is systems of records, system of execution will be more, it is the genetic systems.

Siddhartha Ahluwalia 59:45
And how big you think like apart from you know, for companies to build the opportunity to build a system of agents?

Rohit Agarwal 59:53
It’s huge.

Pick any industry which people are building software for. Now you have the opportunity to say, I can not only replace software, but augment employees working on that software as well, which is in itself a much larger TAM. So if you think about customer support is a, I don’t know, but let’s take $50 billion time.

So customer support itself, this customer support software is $50 billion. I’m sure customer support as a market is probably $500 billion, or maybe $5 trillion.

Siddhartha Ahluwalia 1:00:25
Yeah, 100x of that.

Rohit Agarwal 1:00:27
Exactly. So 100x of that. So now the opportunity in customer support for agentic is not $50 billion, it is $5 trillion.

And so that is the thing that is exciting so many people, which is why you see senior exec, large companies lots of money quitting to start companies, because they can clearly see, there are just massive blocks of opportunity available. And we just market grab at this point.

Siddhartha Ahluwalia 1:00:51
So now building has become so easy because of cloud and cloud and a number of tools.

What is really hard now for founders? What is hard in the market to execute?

Rohit Agarwal 1:01:03
Yeah. I think finding a good problem, and then telling the right story to have people try it out. I think growing the business is the harder problem. If you go to Twitter, there is, I’m sure there’s thousands of people who are building products and showcasing them.

It’s very hard to break out of that noise. Everything you build is like, oh, I can replicate this in a week. And there’s a engineer sitting at an enterprise saying, well, this is my weekend project now.

So one, how do you prove that this is not a weekend project, this is deeper, and we’re constantly going to think about it. And how do you break free of that noise and reach the right people. And that is what at least I spend most of my time focusing on.

So who are the people I need to hire in my team that will get me to break this clutter? It’s just about that.

Siddhartha Ahluwalia 1:01:58
And because of, you know, Claude making it so easier to build, are you seeing a difference in the behavior of enterprise because a lot of AI traffic goes to you on build versus buy?

Rohit Agarwal 1:02:11
Yeah, insanely, like, everybody is now saying, let’s build first instead of buying first. And it’s driven by two factors. One is obviously Claude is enabling engineers to build faster.

So anytime something needs to be done, not just engineering teams, but even the sales team is like, can I just build it because there’s always some small sect in your team that is creative and inquisitive, and they’re using Claude and they’ll build something. So I think that is happening for sure. The other side is also I think there is budget pressure on companies, where if you look out and I’m talking to my friends who are in, like traditional SaaS right now, they’re like budgets are just non-existent.

Like if you’re going and asking for a 100,000 budget or a 500,000 budget, except AI, there is no spend happening in any other software category, which means there’s one side, there’s pressure on I’m not buying a software tool. And on the other side, it’s so easy to build a software tool, that the build versus buy equation is flawed, right at this point. So everybody is like, at least the V1, can I code it myself?

And then you’ll see if the V2 needs to be done as well. And we see it so much like so many companies come to us saying, oh, we have a gateway that we built. But now it’s hard to manage server shifting.

So but I think we’ve gotten to a point where we can be like, okay, yes, we are the once you hit scale, then you come to us. But otherwise, starting up in this space, if I were to start a gateway company now, could be insanely hard, what would I even say? What is my narrative?

So there has to be a wedge that you deeply care about and keep talking about. Without it, I think it’s hard.

Siddhartha Ahluwalia 1:02:48
So if you were to start today, what kind of company then you would start if not a gateway company?

Rohit Agarwal 1:03:59
I think so two factors, right? One is what is the market saying? And then what is my interest?

So my interest has always been that something on the platform layer, something that systems something that is a small thing that fits in and list grows up. So I think that is. So in my mind, I would still pick one of these problems and try to do it deeper.

Or I would solve like I have a lot of fun building my personal assistant. And it’s like an agent that can build itself. So like, even when I’m talking to you, I can just send it a note saying this is what I thought.

And then it build it and it will add to it. So those systems are interesting. So what can I do with that, that Siddhartha can have in his pocket or everybody else can have in his pocket, which is personality.

I think those are interesting. It’s again a hard problem to do. So it’s not been solved right now.

Otherwise, I think security or some of these vertical plays are very interesting again. So how do you go one vertical and just go solve a problem cleanly? And you were talking about can you pick a vertical where you have deep domain knowledge and you find people with deep domain knowledge and just build in that space.

I think those would be interesting.

Siddhartha Ahluwalia 1:05:13
For example, in your case, you cared about all the AI traffic and you built everything on the gateway that could possibly be built.

Rohit Agarwal 1:04:15
Yes, exactly. And my goal was just that once I have this AI traffic in place, now how do I make sure there is value that I can keep layering on top, layering on top, layering on top.

Siddhartha Ahluwalia 1:05:29
You become irreplaceable.

Rohit Agarwal 1:04:26
Irreplaceable, yes. There is one level of irreplaceability already because it’s very hard to get in. Do you really want to put a third party system in your production traffic is already a hard decision for people to make.

Once you’re in there because you’ve proven to a large extent. And then my goal is that once I’m in this traffic, I will make sure that you like me so much and I give you so much value back that you continue to grow with me and I can layer revenue layers on top of it.

Siddhartha Ahluwalia 1:06:00
Today, 180 million of customer spend goes through Portkey. How do you classify this spending?

Rohit Agarwal 1:05:03
It’s just so this is all AI API traffic that’s flowing through us.

Siddhartha Ahluwalia 1:06:11
Like a purely LLM calls?

Rohit Agarwal 1:05:08
LLM calls. Yes. So it’s purely LLM calls.

And that is what makes this traffic valuable. Like till today, and I get this question a lot, right? Why is this different from an API gateway?

API gateways have existed forever. You’re doing the same thing. One, I think the technology is very different.

But let me talk about the other part, right? Which is every API request is literally that has a dollar value to it. Like an Opus request with 10,000 tokens is probably 23 cents.

And this is you’re making 1000 requests, which means this is $23 worth of API requests that you’ve made today, or $2.30, or whatever that is. That is what makes this gateway a lot more valuable, because it’s not an API request that is so minimal in value that it doesn’t even matter. Like a GET request to the DB request, you don’t even think of it as cost.

And now this is like, it’s 23 cents, it’s real. And I’m showing you it’s 23 cents. So which is why you being able to manage this traffic is important.

And that’s where I think we’ve been able to grow bigger. So that is what our spend is. So we keep an eye on how many requests are flowing through the system.

We also like to keep an eye on how much spend is going through the system. And I think we were talking about this earlier, the more Opus spends happen through Portkey, the LLM spend through Portkey increases. And we’ve almost like I think on 10x in LLM spend in just four months.

And that’s because we’ve seen…

Siddhartha Ahluwalia 1:07:40
From 18 million to 180 million.

Rohit Agarwal 1:06:37
Yeah, I mean, almost that would have been the case. Maybe not 18 to 80, but like maybe 30 to 180.

But that has happened because our top two models used to be GPT 4.0, GPT 5 Mini, claude Sonnet. This used to be a top three for a long period of time. Now our top three averages checking between was is Gemini 2.5 Pro, Opus 4.6, Sonnet. So Sonnet still is there. But then you have two extremely costly models at the very top, which is, which is, I feel this is not talked about enough, but there is a change happening in the system. We’re not focusing only on cost, you’re focusing on value of these models.

Siddhartha Ahluwalia 1:08:26
You focus a lot on MCP gateway, like, yeah, what is MCP gateway? And why does it matter to you so much?

Rohit Agarwal 1:07:27
Yes.
So MCP, or I would say, tool calling is what enables agentic workflows to work. So the model context protocol says that whenever an agent, so let’s start with LLM call started with, I will make, I will ask you a question, you give me an answer. Simple Q&A.

This was one request in response back. Then teams added this concept of tool calling, which is, hey, I can’t give you an answer. But you should probably make a call to this function, which will then give you an answer for this question, which could be this function could be within your code, it could be in a third party application or any.

This concept, I think Anthropic built a superb system on it called MCP, which said now MCP servers can have capabilities, which are like tool calls in itself. And now LLMs can say call this tool, call this tool, call this tool.

This is where I think the world changed in the last six months where people were like, it’s not just Q&A, but now my agent can take actions from it can update a Google Sheet, it can send an email, it can do those things. And then you can get into complex things as well. I think for us, it was like, as we’re looking at AI traffic, this is a very core part of the agentic traffic.

And I think almost six months back, we started building the agent story for Portkey, that we’re not just sitting in LLM traffic, we want to sit in all of agent traffic. And I think this year will be this big shift for us, where we announced an MCP gateway, we’re going to announce our agent gateway very soon as well, which is agents communicating with other agents through a central governed platform again. So this is almost part of our journey saying that we’re capturing your LLM request, we’re capturing your vector DB, your knowledge bases, we’ll also capture tool calls, we’ll also capture agentic requests, which is one single point for all your AI traffic.

And that’s going to be the journey. MCP gateway was like, one of the fastest adopted products, or I’d say modules that we have seen. People are just really excited about deploying it being able to use it.

Because the same thing, you can’t have LLMs in production till you have a good system in the middle. You can’t have an MCP server in production, because security team is coming and telling you, Oh, what if this MCP server is malicious tomorrow? How do I shut it down?

Because I now have 1000 engineers, all using this MCP server on their cloud code. I can’t go to everybody and say turn it off. What if there’s a tool call which suddenly starts breaking my systems?

I can’t do that. These are all things which are like massive risks to enterprises, which is why like no, no, no, no MCP. But when they see MCP gateway, they’re like, yes, this will give me peace of mind, and help democratize MCP to all of my organization.

And that’s going to happen. The MCP, the protocol is also evolving very quickly. So just about again, we’ll take care of the MCP chaos as well and let you go deeper.

Siddhartha Ahluwalia 1:11:39
Yeah, Neon and Portkey did together a MCP meetup at this place only.

Rohit Agarwal 1:11:44
Yes. And it was so popular. I mean, people after that walked away with so many good, interesting things around MCP and A2A.

And that was a fantastic conversation.

Siddhartha Ahluwalia 1:11:54
And, you know, my last bit that I want to focus is a lot of your initial GTM was community driven. Like how did you build that? Because that was also new to you?

Rohit Agarwal 1:12:05
Yes, yes. I think it was just this constant fire of we just need to keep talking about this. It was also a little bit of why is nobody talking about this?

This is important and nobody is talking about it, which just meant that we talk to more people, we go and talk about our story more. We had very interesting things that people wanted to talk about, like AI gateway was a very new concept. Semantic caching, when we were talking about it was very new.

It took on it was popular. And I think it was a basic need that we had to build distribution very quickly. And our goal, our way of doing it was embed ourselves in communities, be genuinely helpful, and then talk about new things that we are thinking.

So it positions ourselves as, hey, these people really know their thing, because they’ve seen it live in production. And I think that positioning was important for us before we could do anything. So releasing something in the open source world, making sure this became popular, making sure we maintained it properly.

I don’t think there’s enough teams in India, that understand that. And I think even we did not. But we work with a few people who’ve done it before.

And we are very, very focused on creating that community, making sure people like the product. And it just happened. So partly luck, partly just that constant hunger of we need to be able to tell our story. Because without it, nothing works.

Siddhartha Ahluwalia 1:13:36
Open source shaped your GTM. Can you tell us the vision of being open source?

And how did it shape your GTM and your all decision making?

Rohit Agarwal 1:13:44
Yes.
So we have never done open source before, like, I don’t think our team has experience in open source, or what do you do with this. But it was a chance meeting with Parag Agarwal, who now runs Parallel, who was the ex-CTO of Twitter. And I was sitting with him.

And I was generally chatting about the things. And I had a question to him thing, should I just open source bit of this? I don’t know what to do.

And why should I even do it? And he said that there’s only two reasons you should open source. One, it is giving you so you’re building a product that you can’t build in isolation, and you need the community to help you contribute, etc.

And second, it helps you build developer trust. So don’t think of this as a distribution channel or a marketing channel. It builds, you want to build solid developer trust on your product.

And that flipped our switching, we want to do both of these. We don’t know how to do open source, but we want to do both of these. We’re building a gateway which will have, and today we support 2300 models.

And this was probably 50 then, like this is going to increase, everybody will want to do new things. I can’t solely say I will build everything, I will need to have the community help in building. And second, we want developers to trust the technology, like all core technologies have to be trusted.

So we decided that we’re not an open source company, because we just don’t have the DNA. But we want to open source the core technology. And I think we learned the name for this was open core, where our core technology, which is the gateway is open source.

Take a look at it, see our architecture, compare it to competition, and then realize why are we better by what do we do differently, the control plane will be commercial. So that was the thought at the very beginning that this part, which I feel should be democratized, everybody should just be running this, like, even if you don’t want work, you don’t need anything, please just use this without it, your life is going to be so much more harder. That part, we said, Okay, this will be open community driven, we will try to build community as much for it.

That’s what we spend time. Control plane is where we said there is commercial value, we are still a business, we want to run a business. So this is where we will monetize completely.

And we still in our commercial product that open source gateway powers the control. So that helped us where we were not having two different efforts on, you know, oh, we want to make money, but we also want to open source, it was very clear that this piece, we will not monetize. But the value on top of it is what we will.

Siddhartha Ahluwalia 1:15:19
And, and today, can you share some metrics on your open source?

Rohit Agarwal 1:16:28
Open source? Yeah, it’s been fun.

So I think we have the vanity metric GitHub stars, like I think almost 11,000 GitHub stars now. I think daily, there’s 1000s of companies that are coming starting deploying the system. We have almost half a million Docker installs, which are live right now.

And I think this is I checked it a couple months back. But yeah, lots of SDK installs that way. So it’s one of those popular systems.

And while we don’t have too much telemetry on it, like here and there, we find out glimpses of teams that are using Portkey open source in the background to do everything. But for example, recently, when we made our series announcement, the CEO of XR was like, Hey, we use Portkey internally, like, oh, nice. So XR internally uses Portkey as their Portkey open source as their LLM gateway.

We have Snorkel that started with this where our open source gateway was their core AI gateway internally. So that is just hearty to see that I run into people at conferences. I’m just chatting with them and they go we use Portkey the open source version internally.

Like that’s pretty cool. And I think as a product founder, yes, I would love like the money. But I’m so much happy that the software that my team is building is actually getting used and people are liking it and coming up to me and talking about it.

And that is solid validation. Again, I think it depends that you have to really enjoy that part of the process. It’s painful, but then you do the open source, then you have fun.

Siddhartha Ahluwalia 1:17:01
Thank you so much, Rohit. Thank you for sharing the journey candidly, your learning candidly. And I wish best for Portkey to become a 100 million ARR outcome in the future.

Rohit Agarwal 1:18:15
Thank you so much. I mean, yes, I think that’s the whole goal that it’s almost a community. And we’d love to keep partnering with Neon.

I think that meetup that we did was appreciated by many. And I think we should just do..

Siddhartha Ahluwalia 1:17:24
Absolutely love to do more with you on different topics.

Rohit Agarwal 1:18:34
Absolutely.

Siddhartha Ahluwalia 1:17:29
Make the partnership work.

Rohit Agarwal 1:18:36
Totally. We should do that.

Siddhartha Ahluwalia 1:17:32
Thanks.

Rohit Agarwal 1:18:38
Thanks Sid.