Name: Scaling Agentic AI: How to Accelerate Innovation Without Losing Control
Uploaded: 2026-08-18T06:05:15.101Z
Duration: 1 h 2 min 56 s
Description: Scaling Agentic AI: How to Accelerate Innovation Without Losing Control

Transcript for "Scaling Agentic AI: How to Accelerate Innovation Without Losing Control": Hello. Hi, all, and welcome to today's webinar on scaling agentic AI. My name is Vinay, and I will be helping support today's live session. We are excited to have you all join us. In this exclusive session, our experts AI expert speaker will explore how to scale agentic AI. Sorry about that. Our our speaker will explore how to scale agentic AI effectively by extending your existing zero trust foundations. But before we begin the session, a little bit of housekeeping. On the top right corner of your screen, you will see the live chat option that you can use to interact with your fellow attendees. In the docs section that is located next to the live chat, we have attached a couple of ebooks, blog articles, and solution briefs that are relevant to today's session. On the extreme right, you will also find the q and a box. We we encourage you all to leave your questions in that q and a box as we progress through this webinar. We'll make sure to address all your questions towards the end of this presentation. Also, a quick reminder, this session is being recorded, and we will share the recording with you within forty eight hours. Now please join me in welcoming today's speaker, Jeremy Jaurad, senior product and GDM specialist at Cloudflare. Hey, Hey. Jeremy. Hi, Vinay. Please take it away. good morning, everyone. Good afternoon. Thanks for the intro. So, hopefully, we, we have some people, on the audience. Okay. I can see some. Hi. Hi. Cool. Thanks. Thanks, guys, for interacting. It always feels better when, you know, you know, you have people on the other sides, ready to hear about what we are sharing. So thanks for that. So, like Vinay mentioned, today, we're gonna talk a bit about agentic AI, and this is a topic that has been, I think a few times already, a theme that we covered a few times in the webinar. But today, we're gonna look at another side of it. We're gonna look at, of course, the security aspect, but we're also gonna look at other aspects. And one of them, which is very important, which is cost. And some of you might know that two weeks ago, we had, an innovation week. So we have this concept of innovation week in in Cloudflare. And during innovation week, we talked about agent weeks and how the agents are changing a lot of things in how we we use software, how we develop software. So we're gonna talk a bit about that and some of the announcement we made during that week as well. Quick background on me. I've like Vinay mentioned, I'm a product specialist. I look after, zero trust mostly, but as you know, we move from, you know, security for human to security to nonhuman. So there's a lot of impact on AI, and that's why today, I'm helping to cover this topic from, the AI as well as the security angle. So let's jump into it, and let's have a look quickly at what happened across the last six months. And I think even more than six months, mostly a year, where we heard a lot from all the CEO, all the c suite. I'm sure in your own companies, you you you hear a lot from the executive as well that AI is mandatory. Everything needs to be focused on AI. And like we used to have a few years ago, cloud first initiative, we now have AI first initiative. And all these mandates are starting from the top to engage everyone in the organization to start being proficient and leveraging all these new technologies that AI is helping us with. So why why this why this happen? This happen because we are human. And as human, we are a bit resistant to change. You know, when we are used to do some things in a specific way, changing all of these, the process, the thinking, the rationale, it's not that easy, and it takes time. And that's why this messaging, used to come really from the top to encourage people to, bring the change in the organization at all the different layers. And this was necessary. Like I said, we are all human. We are all a bit resistant to change. We had to do this. And it was not super easy, and that's why in many organization, you realize that people were starting to put metrics around that and maybe leaderboards and maybe did shout outs to people that were using AI or that were creating new applications really to encourage innovation and and share all the wins inside of the companies. So this was not easy, and many organization managed to do it quite well. And then they moved to the other side of the problem, which is what we called, the culture of token maxing. So what is token maxing? Token maxing is basically using as much as AI as possible, consuming as many token as possible without caring so much about the outcome. And this led to a lot of interesting announcements, from very large company that were very early in the AI journey. And you can see here, Uber, for example, that announced early in 2026 that their whole AI budget has been totally consumed in a few months. I think this was, very early in the year, and you realize that because you encourage people to do a lot of these things, to use AI a lot, they're actually doing it now, and they realize the benefits of doing that. But by doing it, by trying to be at the top of the leaderboard, by trying to use all of this, they kind of sometime lost track on, what was the real value. And, also, all the processes were not really developed. It's a new thing. It's a new way of working. So there's a lot of optimization that can be made. So this was and still is a large problem around AI cost or the cost of AI. And Cloudflare understood that very well, and that's why we're gonna talk a bit about this topic today. You if you follow the Cloudflare blog post, you know we already discussed a bit about that, in some of our blog post. That was, in May when we announced that now we are able to fix the AI deal by imposing quota, per user, per team, depending on the models. And, obviously, we're gonna improve that more and more, using a lot of, semantic driven, you know, model choosing. So a lot of things still coming up that way to really help you to keep that under control. But costs is definitely a a a problem that we are trying to fix. There's another big problem, of course, which is the overall security. And the overall security is an ongoing battle. When we started with security, you might have seen other webinars when we talked about the AI security suites. We talked about how to make sure that when a user is going to a website that is AI powered or has a chatbot or has, or using Gemini or or ChatGPT, how can you, have visibility and control on what they are putting inside of the prompt? But now we move even faster to another type of problem, which are agent security. What happen when you let agent do the work they are supposed to do, but at the same time don't have all the right controls around it? And this is really something that happened, again, recently that was in, in July, so just last month. I'm sure a lot of you heard about this, OpenAI and Hugging Face incidents. So, without going all the details, there were some agents, as part of OpenAI experimentation that managed to come out of, sandbox or containments, environment, and, unfortunately, went to create an outage on, on Hugging Face, which is a very famous, marketplace for AI models. And it's very interesting when you go into the the details of that, the timeline. The guys did a very good job. If you go into the Hugging Face blog post, you will see on the right side, there's this anatomy of a frontier lab agent intrusion that goes from, start to the end. So just a few weeks and explain everything that happened. The first thing that is really, really marking is the scale of that because all these agents are running much, much faster than what a human can do, and they can also evolve. They can start to adapt to some of the conditions. One of the findings over there that's very interesting also, exposing that, Black Hat session, from OpenAI directly is that, initially, the agents in that attack didn't have a way to communicate. They were isolated. And what they did is they managed to find a single common vector, which was a repository. So something that they can use when they do their work to download packages, and and, supposedly, only one way. But what they managed to find is a way to go and write inside of that repository. And what you see in that screenshot is that, actually, some of these agents managed to use this as a message board. They started going into the repository and started writing. And you can see here, something called, like, agent six soft trace, upload the found. So they were trying to communicate with each other by that single, I would say, components that was not supposed to do that at all. So it's very, very interesting. We are learning a lot. And when we the more we learn, the more we realize that we need to be very careful about these things. There were already a few papers out there from the cybersecurity agencies explaining how we need to be cautious about, how we we do, AI, how we use AI. OpenAI, at the end of this, explanation as well as this blackout session, was saying that they were repurposing some of their engineering efforts from research to security. Right? So we we don't have a full, understanding of all the AI security, but they are basic principles that we can still already apply, which are the zero trust principle. We just need to fine tune that to nonhuman, which is different scale and different speed, but these principles are still valid. And I would like to, highlight a quote from, Nikola. Nikola is someone that has been working with Cloudflare, quite a bit back in Europe. Nikola is working for, Swiss Post as the head of zero trust and agency k I architecture. And it's very telling, this quote basically says that, the velocity, the speed at which a developer can build, a full app or a full feature in minutes is the same force that will allow an agent to potentially kill an algorithm, kill a production, or exfiltrate data, and that already happened in the past. So very telling quotes over here that tells us that we need to be very mindful when we use all this technology. It's very exciting. It's very powerful. But as you know, you know, with great powers comes great responsibility, so we have to take care of that. So before we move further, quick question to, all of you, and feel free to access, the poll on the right side, on the top right side. So the question is, what is your biggest challenge when moving for, agency KI from pilot to production? Because we have been doing a lot of pilots, but now is the time really to push forward and and put that into production and get outcomes. So is this more like security and access control that was mostly your concern or is mostly your concern? Is it more around data privacy and compliance? Is your focus cost control? You are you have been able to put that into production, and now you realize that because of the scale, the cost is becoming very difficult to analyze and and, constrain. Or is it integration with existing system? Because, of course, AI on its own, it's purely, you know, AI inferencing. You can create text. You can create data. You can create images. But the real power is when you start to integrate with existing system, to actually move to that agentic AI portion. So feel free to let us know. I'm gonna give you maybe another another minute just to get a few more votes in. Okay. So the top biggest challenge that every one of you have voted are security and access control. And I think it's very relevant when earlier, I was mentioning about zero trust. Right? Because we zero trust is all about security and access control, and there are definitely things we can do, around that to help secure AI environment. Some people also voted for integration with existing system. So that's the second I highest vote. We're gonna talk a bit about that as well. There are ways to integrate with existing system and do it in a secure manner or at least in a controlled manner. Data privacy and cost control are very similar. They are actually equal number of votes. Interesting to see cost control, lower on the voting scale today because it's usually quite a a big concern. Maybe today, the audience, doesn't have to pay some of these bills. That happens sometimes because it's the finance team. Maybe they would be more aware of this. But interesting. And thanks for the the the feedback. Thanks for everyone that voted just now. So let's go back to, the conversation. And for people that might be new in, in Cloudflare, what is actually Cloudflare, and why are we talking about all these things? So Cloudflare is a special place, of course. Cloudflare started as a CDN and a and a and a WAF and an application services company, and we build a huge network. That huge network is covering almost all the world. And we do have a lot of capabilities, around application services. Like I said, zero trust and also AI. We have a full, development platform that enables you to do all this. So it's the perfect place when you start thinking about agentic AI in terms of developing, this application, but also securing this application and distributing this application. So that's why this topic is extremely important, and some of the early decision foundational base of Cloudflare, makes a lot of difference right now when we talk about this. You can see over here in terms of performance, we are fifty milliseconds away from almost any Internet connected user in the world. We are highly interconnected, which is very important when you wanna secure and distribute all this data. You don't wanna compromise, performance, for security. And Cloudflare, which is also very special, operates its own purpose built physical infrastructure. We are not relying primarily on, people, and and, technologies like AWS or GCP to run our infrastructure, which is also very interesting. It gives us a lot of, differentiator when we go into this, project. And let me talk a bit about how Cloudflare went through that journey because we learned a lot through, the last year, and that will lead us to, the topics that we wanted to talk about, cost control and security. So when Cloudflare started with, AI, in May 2025, so a bit more than a year ago, I would say most most likely, like a lot of you, started with, a Gemini or a ChargeGPT. Basically, your clients where you can go, have access to an LLM, and start automate some task, ask questions, draft some documents. But everything that that Gemini is doing or that ChattyPT is doing is really generating text most of the time. There was no interaction at that time. It was more like a fault partner, but the human was still very important to to do this task. Then we start to realize that we actually need a bit more than that. And especially when you go to engineering, when you go to people that needs to, create code, review codes, push this into a GitHub, things like that, have more, I would say, complex workflow, we needed something else. And that's when Cloudflare started to look around OpenCode. So open code is, a client that helps you to basically have a diversity of models. You could choose your models. You could use Gemini. You could use Claude. You could use specialized models for specialized task, and you could really create all this workflow together. That made the real transformation from, generative AI to a more agentic AI. That was really the shift. Now the challenge when we arrived there was that OpenCode is not the easiest tool to use. You need to install something. You need to, make sure you have dependencies. It's a terminal user interface, which means black and white. You know? You can imagine when you go to, salespeople or maybe your finance people, people that are, usually less technical, this is not the favorite way of working. It's a bit more complex. So came the last phase or, at least for now, the last phase of the AI journey for Cloudflare, which is the internal development of of something we call Cloudflare OS. And Cloudflare OS basically brought all the learnings that we went through, the use of Gemini, the use of OpenCode, to really create a comprehensive AI workspace in a browser, nothing to install, with dedicated context, dedicated skills, basically everything that you need as a user to accelerate your productivity. So what is Cloudflare OS? You might have heard about this, a lot recently, maybe in the news, because of agent tweaks, of course. So Cloudflare OS has a few different components. Mostly, what people will be interacting with most of the time is the central AI agent workspace, right, where you're gonna be able to query, you're gonna be able to, create routines, create workflows, and, eventually, you can deploy sub agents or apps that will be doing customized workflow. You would be able to set specific time as well. So think about something that you can, preconfigure. Go to bed. You wake up in the morning. You have all the results, and you can even have notification on your phones and things like that. So this is where the user will spend most of its time. But at the back of this, there are a few very important components. One component is the content library. This is where the harness really, take all this power and this value, where you create a set of skills and content to preconfigure or, help set a very specific way in which the agent has to, frame his thinking, deliver his results, present, some artifacts. And this is really where, Claflya spend a lot of time to understand what the what the human, what the employees were doing, and put that into specific constraint for the model. Of course, you have to go to an an AI inference model. So you have to, make available different models for people to use for different perspective. Let's say if you wanna do a code review or if you wanna do an image generation, that might be a different models. And all of this has to go through a specific layer of security and control, which is AI gateway. I'm gonna come back onto this. But the last portion I wanna talk about is integration with third party. Right? That was one of the question just now and one of the, highly voted answers. How to integrate with, external party? And that's mostly via a protocol that we we call MCP. And MCP, has been around for some time now and has been making this very easy to integrate with by speaking the same language. It's basically a protocol that AI tool, and back end application or MCP servers, can help to, understand and talk the same language to basically exchange information. Again, we don't want this to go, unmonitored and uncontrolled, so we have an MCP portal over there that will help us, realize that. And all of this Cloudflare OS has been built entirely on Cloudflare. You remember when I was talking about the developer platform? Cloudflare has a lot of primitives. Some of them you might be familiar with. Cloudflare workers is has been around for some time. It's basically our compute, but also durable object, which is, helping to get, durable, storage and context memory if you want, dynamic workers, Cloudflare sandboxes, a lot of this component to help you with, the security aspects of of that and compartmentalizing, all these different agents. We talked about security. Remember? We talked about applying zero trust to all of these agentic AI. That's why you will find here access. Access is our zero trust network access. So everything, authentication, least privileged access, everything is managed over here. So, really, we're using the same pillars, that we had for SASE and for SSE for everything agentic AI. And then at the back, AI gateway and MCP server portal. We're gonna talk a bit more about that. But you have other things. You have browser run to help with ARAG or what we call now AI search, making sure that an agent can go and browse a website or browse a PDF with a dedicated browser. Right? You an an agent is not using web the same way as a human is doing, so he needs a different type of browser. And storage, we have plenty of different storage, for backups, for, key value pairing, for analytics. So a full completely full environment to to develop all this platform. The biggest news that happened two weeks ago is that we released Cloudflare OS as an open AI workspace for the enterprise. This is now open source. You can use what Cloudflare is using internally that helping all of us to achieve our day to day work in your company, and you can make it your own. You can build integration with many different, applications via a concept of gatekeepers. You are able to, leverage different models that you wanna use. So this is very, very powerful, and it can run, of course, on Cloudflare platform. It has been designed to wanna to work on Cloudflare platform, but you can also, leverage that on your own environments because it's compatible with Workers d, which is our open source version of of workers. So very interesting, announcement two weeks ago. Created a lot of, market buzz. We had a lot of people talking about it. So I hope that today, you get a better understanding of what Cloudflare OS is. Of course, it is not an OS as Windows or Linux, but it is an AI OS to really help you to do all the different task and and workflows that you wanna achieve by leveraging an AI model. So let me give you a quick overview of how it looks like, for Cloudflare. And this is the version that Cloudflare is using internally. So you can see that the colors is a a bit different. But this is the almost the same engine. It was the the v one version. Here, you can see all the different skills that have been created, for different people, the salespeople, you know, if they wanna create an account plan, if you wanna create a battle card. This is all the context, management that I was talking about earlier. So you really get that brain that knows how to create content, how to reason, how to think. And that is very useful for, the LLM model because now you have to try less and achieve more. What do I mean by that? I'm sure you you you in the early days, you know, you type a prompt. You don't get the right results. So what do you do? You type another prompt and another prompt and another prompt. That's called prompt engineering. Right? You try to refine your prompt to get a better outcome. But every time you do this, you actually consume more token. You actually increase the bill. So when you do context management, you create more context as a skills so that when you ask the prompt the first time, you have a much higher chance of getting the results that you want. This is really a difference from a traditional generative AI plain, I would say, Gemini or ChargeGPT environment compared to an open code kind of approach with skills, and and all this, added knowledge available, from the get go. Not just this. You remember we talked about cost tracking. It is very important to pinpoint where the cost the cost is coming from. So how many messages, what's the total cost, what are the models used. So this is something that today in Cloudflare, we are able to do at the user level. And this is very life changing because now you can you can get better visibility. So that's the beginning of having better controls. You can have daily activity trends. You can see if Frontier models are a bit more expensive, are more used than, Openweight model. We're gonna come back to this. But this is really key, and this has been asked by, many of our clients to really get that, analytics kind of dashboard, from the FinOps perspective, the financial operation. So this is how it looks like from a more architectural point of view. You have a user, a Cloudflare employee, or any, places where Cloudflare OS is deployed, connects to that web page, have access to, that chatbot's, or or homepage that I was showing you just now. And at the back of this, you have all the skills. You have all these elements, that are creating the context management. And then if you remember, there are a few different components over there. So, AI gateway that is connecting back to all the LLMs, the MCP server portal, the and the remote MCP servers for connection to external third party, and as well as the ZTNA, the access components, to ensure zero trust principles are respected. So before we go into a bit of the details of that, and we talk a bit more about the components, Quick poll over here, to see from your perspective, what do you think is the capability that would help your team the most when scaling agentic AI safely? Do you think that today you are lacking a bit of centralized policy enforcement? So you are a bit everywhere trying to, control that on the different, attack surface or or different traffic flows, or do you think you don't even have visibility to do any enforcement? Today, what you need is really to get, monitoring in place, not then know what's happening, and then you will be able to enforce. Or your problem maybe or your biggest challenge is to say, I don't know what model to use and how to make sure that the proper guardrails are in place because there are so many of them, and they all have different way of interpreting the prompts. Or do you believe that you are more or less there, but you have too many hurdles, which means that you are too slow to experiment, you are too slow to deploy, and you need a faster way to do all of this? So let's look a bit at what the poll is saying. I'm gonna give you a bit more time. So the leading votes for now looks to be the guardrails for model and tool usage. Okay. Interesting. Why am I saying interesting? It's because a lot of time, people think, oh, but I'm already using an LLM, and this LLM has already some guardrails inside. But these guardrails are very specific to a model. And from a security angle, what we love the most as practitioner is consistency. Right? We don't want to have different guardrails for different model behaving differently. So I can totally relate to that guardrail for model and tool usage. Especially on MCP, we don't really have a lot of this, natively. The second one is observability and audibility auditability, sorry, which is fair because, again, if you don't know, what's happening in your network, it's very difficult to put enforcement. So that's, that is normal. That is one of the second highest voted right after we have centralized policy enforcement and finally, faster experimentation and deployment. Okay. I think it makes sense if we have more network and security people on the in the call today because this is usually a concern more on the application, cloud team, AI team. They wanna go faster. On the security side, we try to be more secure before we we go and and completely let this guy, run freely. Okay. Thanks for all the votes. Definitely, very good insights for, myself and for everyone. So thanks for that. Let's go back to the deck. And like we said, we're gonna go and walk through some of these additional, components inside of the OS, Cloudflare OS, and see, how they work. First components, we talked about integration with third party. This is done via mostly MCP, and Cloudflare is enabling you to create MCP server very, very easily. We have an agent SDK and a supporting platform at the back to very easily create MCP servers. These MCP servers will become the, translation, components between your internal services and your AI tools. Right? So what what happened in in that MCP server is you have, an API call at the back, okay, that will interact with the database or or data lake or any internal services that you might have. And on the other side, it will integrate with an AI LMM via tools. So it will tell the LMM, these are the tools available. Please choose what you need to execute, and the MCP server will be creating the API calls on the other side. So sometimes you use, SaaS, you use, third party, MCP servers because it's already available. It has been provided by the vendor. But sometimes when you have internal services, you need to create your own. And that's why it's very useful to have this running, in the same environment. When we launched MCP, jointly with Entropic, a lot of large company like Stripe and PayPal and all these, managed to deploy MCP servers on our platform and creates new applications, new features in a matter of days. That was very, very powerful. And today, MCP has been, given back to the community as part of the AI agenting foundation, and Cloudflare is one of the, founding member of that platform. So we are really supporting MCP. For those of you that are aware, MCP was just went through a quite a quite a significant revamp where it became a a stateless protocol and, a few of the authentication mechanism change as well. No more DCR and things like that. Cloudflare supported this from day one. Why? Because Cloudflare was involved in the development of, this new, standard, basically, or new version of MCP. So it's always good to know that when you go with a a partner to work on these kind of things, you stay up to date, and you don't have a lag, you know, between the new standard, the new version of this software that are changing quite a lot, and what is available as a product. Zero trust, we talked about this. Access or zTNA is the main component that will authenticate every single request. Going through MCP, going for an AI gateway, connecting to a Cloudflare OS, having access to specific, back end MCP server or internal services. A lot of these are relying on authentication standard, like example, OAuth. This is supported by default by access, meaning you don't have to reinvent the wheel. You can straightaway leverage that and enforce the same capability as you used to do for human now for these agents. Now let's talk a bit about MCP server portal. We have an MCP server. Why do we need an MCP server portal? Right. Because MCP server, like I said, could be your internal MCP server. It could be a third party MCP server. And what you wanna have is consistency. You wanna have visibility across all this MCP server, what kind of traffic is going here going there, and you wanna have controls over this. You might wanna block specific tools from being called, on specific servers because it's too sensitive. Example, if one of the MCP server over here are exposing a tool that allows you to go and change a record in ClickHouse or Snowflakes, that might be a problem. Right? Remember all these stories that we heard where an agent potentially make the wrong decisions, go inside, and make a wrong change. So we wanna have a little bit more control, on top of the, native protocol. So this is how it looks like. Before you access any MCP server with your agent, you have to authenticate that. So leveraging on our ZTNE platform, our OAuth authentication. You will have a list of all the MCP servers available at the back. So you are able to see directly which one have been authorized, which one are connected. And once you are connected, that's it. Everything is channeled through the MCP server portal, and you have now visibility and control. So security, check. Now what about cost? When we talk about cost, MCP is actually a big, token user. Why is it a big token user? Because the LLM has to understand every tools that is available to him via, the MCP server portal. So he's gonna go there, he's gonna understand this, and he's gonna consume token for every single tool. So Cloudflare has a lot of very smart, and brilliant engineers at the back, and they were thinking, how can we optimize that? How can we reduce the build? And they come up with something called code mode, that really helps you to reduce the token, usage. I'm not gonna go into the details, but, basically, what the whole thing is doing is reducing the number of tools from maybe 100 to two. Any MCP server will only expose two tools to the LLM. If you wanna have a a bit more detail, there's a blog post explaining all of this. Very, very interesting. Now second part, again, we are going by a big big on on security. We just announced during agent weeks a private beta for something called Riot Guard. So Right Guard is putting an even more fine grained access and security control for MCP, where you can now, put, metadata before an MCP agent is going to an MCP server and do a validation guard. So maybe you can have a a human in the loop for specific, high privilege action before anything happen. Very interesting to have this on MCP because it's not natively available. And finally, we have the AI gateway. So AI gateway is communicating directly with the LLM. That's where all your inference is is happening. So what do we do on the eight, on the AI gateway? AI gateway helps you to get options in models. Right? Because when the request comes to AI gateway, you can decide which model you're gonna use and redirect it to an external inference party or maybe, a Cloudflare inference solution or a model running on workers' AI. This is all through dynamic routing. I'm gonna show you that very quickly. And there's also a component of, caching and resiliency. So resiliency means if one of the inference server might go down or is not available for any reason, you are able to send that same request to another, model that is potentially available at the same time. And caching, of course, reduce the number of token, used because if the request was already seen before, you can simply, reply with the same answer without going back to the model and consume token and redo the whole thing again. So two steps, using AI gateway. Make the spend visible. Put AI gateway in front of your model, and expose it to the app. And look at what happens during a week, during a month, during a quarter. Get visibility on how much are you actually consuming, who are the users that are consuming all this, and stop, you know, building a case on, okay, this is where we can potentially optimize. This is where we're gonna cut. This is where we're gonna change things. And turn on catching as as well and and routing because this is giving you the opportunity to do that without any back end changes. You are future proofing your infrastructure by putting an AI gateway in the middle. And now anything happen in the future, you wanna change model, you wanna, change some of the, dynamic routing, you can do that at the AI gateway. No need to reconfigure your application, your direct API calls to the inference. It's very similar to, like, a network load balancer, you know, when we we used to do this, to connect back to the back end application. You put this in the middle. You have this layer of abstraction, and now you can do the routing as per your need. Another component when we talk about cost control that is very interesting to look at is different models. So you have what we call the frontier models that most of the people are aware of, And these models are usually closed model or proprietary model. Why? Because they are very expensive to develop, and, the companies behind them needs return on investment. They need to protect their intel intellectual property. So this model are usually completely vendor locked. And then you have two other types of model. On the other extreme, the open source model, which are fully downloadable. You can know the source code. You can, understand what was the training data, which apply in some cases. But the more interesting one are the one in the middle, what we call the open weight model. So the open weight model, usually, they don't disclose all your training data, but you can self host them and you can download the weight, which means you can basically run it on your own. And why this is so interesting these days? Because the price of the token, the cost of the token has been growing tremendously. For frontier models, it keeps on getting pricier and pricier. And at the same time, the development, the research that has been done, on the open weight model really lower the difference between a frontier and a open weight model. It's really, really close now the the type of outcome and response you can get in an open weight model compared to a frontier model. And the price, it's heavily different as well. Now the last, reason why this is really changing the the AI inference world is that on managed platform where you use your Frontier model, it might be AWS Bedrock or it might be Azure Foundry. Usually, these open weight model are not available. They are not available because they are commercial agreements or things like this. They take a lot of time to come up on this platform. So what you wanna do is you wanna be able to leverage this open weight model as soon as they are available because you can get similar results for a fraction of the cost. Why not? So how to do this in a most efficient way? Workers AI and AI gateway. Using these two things together, AI gateway gives you, the, dynamic routing so you can change where you are sending your request. And workers.ai is Cloudflare AI inference platform where all this open weight model run. And if you want a graphical, representation of the differences between the open weight model and the frontier model, this is how it looks like. So on the, horizontal scale, you can see the cost. And on a vertical scale, you can see, the intelligence index, which is, like, the the the power of that model if you want. And you can see that, Fable five and GPT five six so are not so far from k three in terms of, intelligence, score, but the cost is very, very different. Right? You are at 0.8 over here compared to maybe a a $6 over here. That creates a huge difference at the end of the day at the, organization level. So what you wanna do is identify a workload. Try. Put a routine task and send that to an open weight model. Evaluate how the outcome is different. It might not be so different. And that's when you realize that you can save costs and, at the same time, achieve the same outcome. AI gateway is giving you a very easy way to do that. You can continue sending some of them at the external inference with the Frontier model and some of this request to Workers' AI with some of the Openweight model. And this also prevents what we call the vendor lock in or single catalog lock in. Right? Once you start working with a single provider, you are stuck. And only what is available on these models, you will be able to leverage. Anything outside of that, you will have to reconfigure your environment. You will have to, basically change a lot of things. So let's have a quick look at how, the AI gateway is showing up in, the user interface. So it looks a bit like this. You, can see over here all the different AI gateway. So I have some gateways that are, no authentication, and I have gateways with authentication. Of course, this is usually what we put in production to make sure that we can, identify who is connecting to that AI gateway. Sorry. Just a quick check, of course. Security always. You just saw the turnstile, in action. So now we go into the analytics. So once you have traffic going through that AI gateway, you can see the number of requests. It aggregates all the token cost, the cache status, the number of errors. So you you really get a lot of visibility on all your different AI traffic, not just a single one going to a platform, but all across the board. A bit more analytics on on costs. And this is where it gets really interesting because you have logging of every single request. So So you can see at the bottom over here, we have all the different calls that are being made going through the AI gateway and the full compliance audit trail. You can see which model, have been used, the duration, the cost, the full requests, including the prompt, including the response. Everything is available here. And what is interesting, you can see down there, is that you also have some additional security components. Right? It's not just about performance. It's always about the combination of performance and security. So let's look at this. What do we have over there? We have guardrails. People just now voted, and you were saying this is the highly, I think it was the first one. The top concern was guardrails. So we do have guardrails here for, hazard different hazard category. It could be crime. It could be deformation. So that there are many pre preconfigured, guardrails that you can activate across the board. But because Cloudflare is, if you remember, a zero trust platform as well, we have a full DLP engine running at the back of this that we can leverage. So you can apply PII. You can apply, financial data, security insurance, like, source code detection. All of this can be applied on your AI gateway request, which is super powerful. And you can also build now what we call custom topic. So you can create your own, DLP topic based on natural language processing. So you can just write it as you would write an LLM, and we will create a policy based on that to catch any of these data flowing through an API sorry, an AI request. Very, very powerful, because very consistent. You manage this at one single place, and it applies across all your AI models. So that's the the DLP portion. And let's look a bit at the dynamic routing that I was talking about just now. So you can see here a very graphical user interface. You can use building blocks to do that or you can write the code over here. So in this case, what do we do? We start. We look at, who is the user? If the user is a c level, then we're gonna send them to a very expensive model. Right? Because c level usually wanna have access to the more expensive, models to get better outcomes. So that's okay. We can send it over to, let's say, a SONNET four model. But if you don't belong to the c level, potentially what we do is we're gonna send you to another model, which in that case is, Kimi. Right? Open weight model, more cost effective. There are also a lot more people that are not c level. Right? So, we just want to use the scale to our advantage. But first thing we're gonna do is we're gonna check if this user has used $100. This is just something you can configure, but $100 per week. If they use already $100 per week, then potentially we're gonna block that and we're gonna fall back and we're gonna decline that AI request. Or we could use another model, which is even cheaper. That's up to you to design all of this. But you can see the the flexibility that we have over here, leveraging multiple models, leveraging, dynamic routing, leveraging budget to really get, that whole AI interaction under control. Let me go back to the slide. I hope that was a a good overview of what the AI gateway is able to do. And we're gonna do a last quick last poll. So in this one, we're gonna ask you what is, where's your place, basically? Where are you standing in the agentic AI journey as an organization? Are you still in the phase of exploring use case, so you are not really there yet? You are still early stage, or are you running already pilot? You have isolated some use cases and some vendors you are doing pilots. Are you done with the pilot and preparing for a broader rollout, or you are what we call on the most mature side of things already deployed, agents in production, basically running at scale? So very interesting to see, where you guys at inside of that maturity scale. We're gonna let the few polls coming in. And at the same time, I'm gonna check the q and a very quickly. Okay. Can you DLP sanitize a non English, name? That's one of the question. So today for DLP, we are applying a block action. We are not doing the sanitization or the masking, but, yes, it can work in non English as well. We are using at the back, so we can apply, some of this, on non English language. That is possible, but it would be a block for now. From the observability and to monitor the data that user is giving to the model and the data that's AI model is able to create it. Okay. I we need to come back to this one. Can we use Google agent platform as the back end, but Cloudflare AI gateway at the front? Gemini model directly from GCP agent platform, not from Cloudflare inference model. Yes. This is possible. You can use any model at the back, as long as you can, put the API key inside of, AI gateway. So the model sorry. The client will connect to AI gateway. AI gateway will connect to the agent platform. So, yes, we don't have to use workers' AI. If you remember on the slide, you can decide if you send to an external inference or if you send to a workers' AI. It's definitely possible. So let me go back to the poll and see where we at. So we do have, a lot of people that says, running limited pilots and already deploying production, and then a few others on exploring use cases and finally preparing for broader rollout. So it seems like you guys, are really in the trend like we were explaining just now. We've passed the time of experimentation. Now we need to be in production. So, I hope you have all of this, covered. If not, I hope that you learn maybe some of these things today and you are, interested in going a bit further in some of the security and cost control around your production deployment. So we are almost at the end. I'm just gonna go back to the slide and wrap it up quickly. So today, Cloudflare is definitely, here for the change. The agentic cloud, the agentic AI, all of these new applications needs a new infrastructure. And Cloudflare, like I said in the introduction, Cloudflare build this primitive build this infrastructure way, way early in our development. So what we see right now is that some of the things that we used to have as a almost a muscle reflex. Right? We want to use a compute. We go to e c two, AWS. We wanna use a storage. We go to s three or equivalent in all the hyperscalers. Now we see a lot of people thinking about Cloudflare because we provide primitive that are way more suited to agentic development. Some of them are called durable object, artifact, the workers. Very interesting workers is not a container. It's not a full VM. It's an isolate that has plenty of benefits. I'm not gonna go into the details today. We should have a dedicated session to talk about this, but I really encourage you to look at durable objects and workers and workflows as a potential replacement of what we used to do in a traditional hyperscaler. And this is not just from us. Our clients are saying that today, it's very clear where the hyperscalers will be in ten years because the foundation that they have is not easily changeable. It would be very difficult for them to adjust to the new agentic world because they build it in a very specific way. Where Cloudflare, like I said, made very interesting decisions early stage that puts us in a very good position today when we talk about AI and agentic. Because AI needs a totally different scale, a totally different performance, totally different, security containment. And what we see in the market is that a lot of the company, very large company that started the cloud journey on AWS, Now we see some equivalents in the new world, in the agentic world, like Figma, Lovable, Entropic, OpenAI choosing CloudFlare for, some of their deployments for all the reasons that I just explained. The embedded, integrated platform for developer development and security as well as delivery of content. And if you are interested, you can have a look at what these people are are saying about Cloudflare. They are just in the month of June, all of this, released a lot of information about how they work with Cloudflare, how they leverage Cloudflare. And if you have more question, feel free to reach out to the team over here in, in Cloudflare as well, if you wanna have a deep dive on any of this. So I hope that was, interesting session. I hope you guys, learn some some of these things. Maybe, we we try to go a little bit more in detail in MCP server portal, in AI gateway. So, we do have five more minutes. So I'm gonna quickly have a look at the q and a to see if there are a bit more than that. We also have our own CICD based solely off Cloudflare, and it's been quite insightful. Okay. Not a question, but a praise. So thanks for that. Really appreciate that. And I hope you're gonna continue working with with us. Deploy Cloudflare, Workers' AI models differently based on their core approach to the compute storage optimization. Okay. So that's also a comment, but thanks for trying Workers' AI model. I think a lot of people are trying. It's it's totally fair. Right? When it's new, you try. You see if this has, benefits to your organization. But, yeah, we usually hear quite a good feedback when people start leveraging open model on Workers AI. How do we move from an existing ADK to Cloudflare? I am not so sure about ADK. Maybe you can clarify what you mean by ADK. Sorry if I don't know the acronyms. We have way too many acronyms. What else? Oh, yeah. Let me try to go back to this question just now. For the observability and to monitor the data that user is giving to a model and the data that AI model AI model is able to query and fetch, is there observability for these two? So is there an observability metric for the data that the user is giving to the model and the data that the AI model is giving to the user? Yes. So I think I get it. I think the question is, do you have visibility on the actual data that is going to a model and the actual data that is coming back from the model? So, yes, if you remember, just now we show the AI gateway. And in all the logs, you have the query and you have the response. So you can see all the data that it that is inside. You can apply the guardrails and the DLP and all of that. So I hope that I understood the question correctly and I answered the right way. If not, feel free to, put an additional comment. Do we have model, do open models have access to tools when run on AI gateway? Yes. So the tools is actually not called by the model. The tools is called by by your clients. So either you have AI gateway, you don't have AI gateway. The actual tool calling is is usually done by the client itself. So, yes, this works with or without AI gateway. Google ADK's agent development kit. So how to bring up the whole setup to Cloudflare AI gateway, or can we keep both and integrate them? Oh, Google, agent development kit. Have okay. So I think this one shouldn't be a problem because ADK will help you to, build the the application. But at the same time, you want your request to go to AI gateway. So we just have to, skip the direct API call to a a back end model and replace this with the AI gateway. So the destination will be changed, and potentially the credentials will be changed to make sure that you have AI gateway in line in the path of the query. So it should be possible. If you wanna have more, deeper conversation on how to do this, reach out to us. We can set up a time and and bring our expert to see how we can do that. ADK's engineering. We use this ADK stays in GCP. Front end is Cloudflare. Yes. I think Andre, also replied exactly the same way. Let me have a look at any other question that I didn't answer. I think we are all here. Cool. So we have two two minutes early. I see some people are dropping off. I know it might be lunchtime for some of you. You might have some meetings. I do get hungry also, so I'm gonna leave you for now. I'm gonna get my lunch before continuing the day. Thank you so much for, joining us today. Really, really appreciate, your interaction, the other polls, your questions. You don't know how hard it is to do a webinar when it's very quiet. So thank you so much for, spending the time with us and and be very active. And I hope to see you in the next session. Thanks, everyone, and thanks, Vinay. See you guys.