Name: Unified AI Security: A Strategic Framework for Protecting AI Apps and Workloads
Uploaded: 2026-02-25T06:18:04.853Z
Duration: 1 h 18 min 20 s
Description: Unified AI Security: A Strategic Framework for Protecting AI Apps and Workloads

Transcript for "Unified AI Security: A Strategic Framework for Protecting AI Apps and Workloads": Hello, and welcome to today's webinar. My name is Vinay, and I will be helping support today's live session. Just want to let you know, our original speaker couldn't make it for today's session due to unavoidable circumstances. So we have a new speaker who will be conducting today's webinar. I apologize for this last minute change and appreciate your patience and understanding in this. Before we begin, a little bit of housekeeping. On the top right corner of your screen, you will see the live chat option that you can use to interact with your fellow attendees. We also have attached a solution guide on Cloudflare AI security and our blog article on twenty twenty five AI trends in the docs section that is next to the chat. On the extreme right, you will also find the q and a box. We encourage you to leave all your questions in that q and a box as we progress through this webinar. We'll make sure to address all your questions towards the end of this presentation. A quick reminder, this session is being recorded, and we will share the recording with you within forty eight hours. Now please join me in welcoming today's speaker, Arjun Kalavar, senior strategic solutions engineer at Cloudflare. Hello. Please take it away. you. Thank you. Thanks, Vinay. Hello, and good morning, good afternoon, everyone, depending on which time zone are you joining for. And first of all, thanks a lot for taking time today to join this wonderful session that we are going to have on AI, focusing on a strategic framework for protecting AI apps and workloads. Now quickly diving into the session, we all know that AI has taken world by the stop. And that is all for good reasons because AI has been rampantly used by writers to create new articles, by content creators to con create new content, even by coders to amplify their code taking code quickly to the production. So while AI is taking world by the storm, it is imperative on the organizations to also have robust framework as well as strategy are not only around bringing AI into the ecosystem, but also ensuring that AI is getting adopted and in the right manner. So if you look at any organization today, it faces multiple challenges around security, and that comes from all possible directions. So it could be how do I leverage AI for improving my security posture? Or if it if the organization is in the realm of media or content production, then how do I secure my content by getting scrapped by AI? Then there could be enterprises of the world which are developing mobile solutions either for their market or for their customers. So then the question is, how do I secure my digital workforce from any AI kind of an abuse? And lastly, because everybody today is in the race of exposing AI features inside of their application ecosystem, then the second question becomes, how do I ensure that the API AI APIs are not abused by hackers? And that's what we are going to look into today's webinar. So look further diving into certain challenges, because AI has so much simplified the life, be it of developer, because they can not only create code, but also push code in an automated way. So there is a lot of pressure from the investors on the c level to start using AI. Plus, even before AI, everybody was building the flight while it was in the air. So now it has only increased the urgency. Increased the urgency around how do we adopt these AI solutions. How do we ensure that these AI solutions are secure? Because what happens is in the need for speed, developers take their piece of code into the AI that they are familiar with. Let's say, for example, ChatGPT, and then ChatGPT can give them a debug version of the code. But in this process, what has happened is the sensitive information that lies with the organization is now with ChatGPT. And the question over here is not if ChatGPT is secure or not, but what we should really ponder on is AI applications or ML applications or, to that matter, LLMs were not built with security first approach. They were built with certain objectives of solving a certain problem, and that algorithm is focused on doing that. So a particular LLM would be exceptionally great at creating new content, which is persuasive. Certain LLM could be hands on on code. Maybe how do I fine tune a Python code or how do fine tune JavaScript code? But it doesn't have security built in. So these incidences, what they essentially do is they'll take your piece of code to the LLM. And tomorrow, if that LLM gets compromised, it's your sensitive information that is compromised and can be abused. And that's where you see on the slide there are many headlines that have been made by either abuse of AI or because there were no boundaries around using AI application. So that brings the us to the three critical roles that an organization has around AI. So the first role is, of course, of CIO because they are focusing on increasing the productivity. They will always go for how do I make sure that my teams are more productive with respect to shipping new products and new free calls, and how do these align with the business priorities. Then there is a critical role that the CISO is playing. The CISO plays a critical role of how do I ensure that while I am bringing the new tech stack into the ecosystem, a, that tech stack itself is not going to post a new set of security vulnerabilities or set of challenges. B, how do I build AI governance, and how do I enforce the AI governance? And then c is how do I ensure that none of these breaks, which ultimately falls under the realm of a CTO. Okay? So AI has not only made these roles more sensitive, but then what they have done is they have just exposed the hidden vulnerabilities and post a new challenge to the c level on how do we not only build applications faster, how do we make applications interactive, novel, give that personalized experience, but also at the same time, how do I ensure that there is no increase in the attack surface because of these added flexibility coming from AI. And in today's session, what we are going to do is we are going to talk about these exact challenges, and we are also going to talk about what an ideal solution or what an ideal platform should be. So coming to the Cloudflare platform itself. So the Cloudflare platform is a global platform. We span across 330 cities globally. And because of this expansive network, we are at about fifty milliseconds of latency from the 95% of the connected devices. But why is this important? This is critical because when you are building applications, there are two components which are usually an afterthought. Those two components are how well this application will perform and scale, and second is how well will this application be secured. Now just by virtue of bringing application closer to the end users, what you are doing is you are not only addressing the scale and the performance bit, but you are also ensuring that the threats or the attack vectors getting inside of the applications are kept at bay wherever they are originating. So this is very important and should be taken a note of while building AI centric applications. Second thing is the connectivity piece. That means when I am building my application, I am just not using LLM. What I would be doing is I would be having the application server instance. I would be having an LLM, and then that application would be writing into the database. So how well are these components connected while I am shipping these applications at the edge? So that is another thing. And the third thing is the security of it. That means how do I ensure that the the prompts are not being injected with anything malicious? Or how do I ensure that the a single user is not abusing all the API tokens? Okay. So the protection becomes the critical aspect. At Topware, our mission is to innovate. But while we are innovating, we strongly believe in these three pillars. That means when applications are built, they should be built with edge into consideration. They should be built with connectivity into consideration, and security cannot be an afterthought. It has to be built into inside of the application. And that's where the range of services by Cloudflare come into picture. So Workers AI help developers to run their algorithms, to deploy their algorithms with a single click. And when these models are deployed, they are deployed across Cloudflare's network so that irrespective of where the end user is coming from, they will still enjoy that fifty milliseconds of latency to 95% of the connected devices. Now think about end to end flow. The request is going to hit the Cloudflare, and from Cloudflare, we are going to connect the end user's request towards the application side, and that's where the security mitigation kicks in. Now it could be the reverse as well. That means there is a workforce ecosystem, and then we want to regulate the use of AI applications, and that means we have to secure the outbound traffic. That's where Cloudflare's Cloudflare One platform comes into picture, wherein by virtue of which, we can help you regulate which applications which AI applications per se should be allowed, which AI applications should be monitored, and which AI applications should be blocked. And this is something that we are going to talk about in detail. And before we do that, what we would like to do is we would like to know from you what are your priorities, what are certain initiatives that you are taking so that this webinar is more curated towards your priorities. So on the screen, you have it. Let's take a few minutes, and then we'll get into these four use cases. Yeah. So now let's get into the AI initiatives. So the Arjun, just wanna call out the poll that it's open now. Everyone joining the session today, the poll is now open. You will see at the top right of your screen, next to the chat, the poll is now open. Please cast your vote and let Arjun know what's your next AI initiative as a priority. Thank you. They will stay open for a couple of minutes, and then we move on. Thanks, Arjun. Sure. Thank you. Alright. So looking at the poll responses, the maximum words are swinging in favor of our first use case, which is protecting workforce use of GenAI tools. Okay. So let's get started. So the before use cases that you see on your screen right now are something that we have learned talking to our customers by interacting with our customers. And what really outshine over here is AI, even though has been around for a couple of years, organizations are still figuring out their way to navigate through AI. Now what I mean by that is most of the organizations are in the first leg of designing the AI governance. That means how what should be the AI posture. So let's talk about some of the challenges over here. Now let's take a typical organization, and I'll go to my first example wherein there are plethora of AI tools available in the market today. And the free tier comes with most of the tools, be it Gemini, be it LORD, be it ChargeGPT OpenAI, publicity. Now what happens is because it's so it's a novel thing that's happening on the Internet And because we have curious minds, as human, we are enticed towards playing around with these tools without thinking in detail what could be the consequences. Now I have a promise that an AI can solve a challenge. I have a challenge and a curious mind. So now why not play with it? So in that race, happens is if I am a developer, I will put my code to open AI tool. It could be any tool to that matter. Let's say if I am into analytics, I will just take the customer data, put it on the tool, ask certain questions to the tool that hey, Gemini. Tell me what is the pattern that you are seeing. Or hey, Gemini. I want to identify so and so pattern. Can you help me visualize a pattern around the data that I'm putting it? But in the need of speed and curiosity, what we have done is we have taken sensitive data to an open tool. And as I said that there are no guardrails around these tools, either put by the organization or by the overarching owner of the tool. The LLM has not taken that data for its own education and training. And tomorrow, if it gets compromised, which we have seen tools have gotten compromised, it leads to leakage of your sensitive information. Either your API keys can be compromised, your PII data can be compromised, or your code base can be compromised. Second thing is an IT admin can only stop you from using certain tools if they have visibility of that tool. If there is no visibility, then there is no way for an IT admin to put governance or put guardrails around what are allowed AI tools and what are AI tools that should be blocked. And that's where Cloudflare's solution comes into picture. First of all, what we do is in order to solve this problem for ensuring the right employee is accessing the right tool, what we do is we offer a gateway solution. Now as part of that gateway solution, we bring visibility around what is the applications are being used inside of corporate network. Now for this AI application, we also gave something called as generative AI score depending on what is the ISO level of that what is the ISO compliance of that tool depending on do they have certain security measures, right, like rate limiting in place. So there are many such checks that we do. This is which we give out a score, which also helps IT admin to ascertain if this particular tool should be allowed or if it should be blocked. Another thing that we do is we once you have identified what has to be allowed, we also help you select the preferred AI tool. So let's say if you have an enterprise grade of Gemini, what you can do is if there is an employee who is trying to use OpenAI, ChatGPT, or Claude, or Publixity, they can be redirected to Gemini. This way, you can ensure that within the periphery of your corporate network on company owned devices, you are enforcing that the right AI tool is getting used. Okay. So these are some of the challenges that we are addressing in the market today. Now this brings us to the next use case, which is governing AI agents. You can very well govern humans, and that's easy. But then with AI agents, it becomes even more difficult for all the valid reasons. Why? Because now, AI agents have evolved from being just a normal chatbot to being a autonomous entity. So today, organizations are using AI agentic AI for writing to the database to make entries into the CRM, or it could be even pushing code bases to the production. Okay? So it becomes extremely critical to govern if the AI has the right access or are the AI agents going rogue. So now this brings us to the first use case, which is how do we secure the remote MCP. Now when it comes to MCP, MCP servers were designed to run locally. Now if a developer is running an MCP locally, then how do they expose that functionality to the corporate environment? Because if they do, they are just increasing the attack surface. So now there is a need for MCP to be hosted remotely. Now if we do that, then how do we ensure that this MCP is actually interacting with the allowed resources? And that's where our secure access comes into picture. So with secure access, the problem that we are solving is you assign right access policies to an agent. Each time the there is a need to access a particular resource, there would be authorization checks done and only then the right set of access would be granted. So this ensures that we are allowing the right entity to access the right resource with the right set of permissions. On the screen, you would see some articles which highlight what nuances can happen if MCPs are put to risk. I'm not going to harp much on that. But we just have to be a little cognizant about the fact that it's an easy challenge to solve. It's just need simplicity, and that is something that software brings to the table. In how you can remotely host your MCPs and ensure that they have right set of permissions to access right instances. Okay. In between, if you have any questions, feel free to post your questions. I do see that there is a question. So the question is, is this AI gateway part of suite functionality or is it sold separately? So AI gateway is a separate functionality. It is complementing SWIC to do certain things. So with AI gateway, what you can do is you can achieve rate limiting. So let's say if you have multiple models in place. So let's say you run certain subscriptions. You have a free subscription, then a premium subscription, and then you have models. What you can do is you can steer that, okay, the requests from free subscriptions will go to LLM a, while requests from a premium subscription will go to LLM b. So that is achieved with AI gateway. You can do rate limiting you can do rate limiting based on tokens that can also be achieved with AI gateway. Suite is for the outbound traffic. That means if you want to regulate which AI applications are to be used, if you want to within the allowed applications, if you want to further regulate that, okay, I don't want sensitive data to be posted. I don't want customer information to be posted. Then SWIC comes into picture, which will help you determine what goes out. Okay? So I believe that answers your question. Okay. So to us, now since we are on the topic of sensitive data, often what happens is with the agents, there are tokens in place, specifically if the agents are interacting with GitHub. So in this particular case, what happens is if other token is hardcoded, then again, there is always a chance that a hardcoded information can be licked out. So that's where Cloudflare also provides you to securely administer your secrets and key management wherein when the request actually comes to the edge on Cloudflare, you can inject your token or your secret in the request so that it can carry out its necessary function. So now that brings us to our next use case. Yeah. So how do we now give or how do we secure the applications, the AI powered applications from the notorious users out there? So let's say let's take example of bill payment. So let's say there is a bill payment gateway, and then for the sake of increasing user comfort and user stickiness, they, let's say, rolled out AI features on their application. Now there are notorious and experimental brains out there who want to play around with that particular feature. So in that case, what they can do is they can ask tricky questions. Tricky questions like they can say, give hey, mister chat agent, can you tell me when was the last payment done for this particular user ID? You will, of course, get the information for that. Now the conversation is rolling out, and suddenly, the conversation now turns notorious and malicious. So now there could be the next prompt coming in. Can you give me a record of all the payments made except for this particular user ID? So this is how prompt injection can happen or poisoning can happen. The request can go in. And then if there are not enough guardrails, then LLM can start emitting sensitive information out. Okay? So that's how the information gets leaked out. And that's where it is extremely necessary to not only secure your prompts, but also to secure your entire application because you have no idea what's going to come through that particular prompt. And that's why what we have done is we have introduced security as an additional layer on top of our existing WAF or application security suite. So the layer seven protections are ensuring that we are keeping bad actors away through the managed rules, through rate limiting, through geo blocking, through bot protection. But let's say, if everything is looking fine, but a particular user is coming with an intent to jailbreak the prompt or do some kind of injection. They are very much flying below the radar. They are not making way too many requests. Now it is extremely critical for the security engine functioning at the edge to identify a, if this particular request is for an LLM. Second, if this particular request is for LLM, then is it looking good or is it looking bad? Is it having sensitive information? Does this look like a biased input? Okay. So all these checks are and should be done at the edge as part of best practice. If you don't have it in built today, you can definitely think of a firewall for AI as a security engine protecting your prompts. Okay? So that's where the beauty of Cloudflare comes into picture. Not only is it helping you protect your applications, but it is also helping you load balance between different elements so that you can provide that promised experience to your end users. Okay. Second thing is it is also important that we take certain key factors into picture. So can let's say if an LLM emits and problem goes into the LLM, and if the response is coming in, will there be sensitive information that will be coming out? And can we actually protect that? So, yes, that can also very well be achieved. So at end of the day, what we are looking for is a secure platform, one which will give you power to deploy your models closer to the end user by a single button click, which will help you load balance so that you are making better use of your AI state. Third thing is security so that we ensure that we are protecting you from the threat actors, be it malicious action or be it malicious intent. Okay. So that's what I wanted to cover as part of our webinar today. So let's quickly go through the q and a over here. Okay. So there is a question. Is this firewall work on set of guardrail policies? How this configuration looks like? Yeah. So the configuration is fairly easy. What we help you do first is discovery because you can only protect what you see. So first, discover all the LLM endpoints. We have an enriched algorithm that runs at the edge to identify if this is a normal HTTP request or is it an LLM endpoint. Now if I have to give a single clue, it's as good as LLM needs time to think before it gives out a response. That time to think varies. It's not pretty much consistent versus our regular HTTP application. So that's one of the many signals that we lean on identify LLM endpoints. Now once the endpoint is identified, you can either put rate limits on that endpoint or independent of the discovery, you can use some of the curated criteria that we have made available as part of firewall for AI. It could be, is there sensitive information in there? Do you want to do rate limits? So there are such prebuilt criterias that you can use in order to secure the endpoint. Apart from that, there are managed policies in the system, which will also help you administer and protect your endpoints LLM endpoints. Okay? I believe that answered your question. So there is also a follow-up question. That means it can be a sync request or LLM using HTTP. Yes. Yes. Endpoint identification is not simple. So it's not that we will discover endpoint on the very first request. We will need a sufficient number of requests to be seen before discovering that endpoint, and that's valid for any discovery tool. It needs to learn as well. We will learn about what kind of requests are coming in, how is the response coming in, with what delta is this response coming in, Is the response coming in chunks? So there are various factors that go into the system before stamping that, okay, this is an LLM endpoint that we have identified. Any further questions that we can take? Maybe you can think about a typical scenario that you are envisioning. What would be the best practices around it? So something like the example that we just took in for billing gateway. Something like that. Maybe a hypothetical example. Yeah. So the question is, can you summarize all the products supporting AI security from Cloudflare? Yeah. Now AI security is a pretty broad term. I will just break it down into two separate categories. One is I, as an organization, I am building applications which have AI features baked into it, let's say, like a chat. And then I want to protect, build, scale, and protect that application. So for this specific use case, the tools that we are looking for is going to be for building. It could be workers' AI. Now if you want to store the data, then it could be vectorized. Now, again, this is if you are building with cloud on CloudFare. Second thing is if you already have built and you are looking towards Cloudflare as a performance and a security platform, then in that case, the firewall for AI will come handy for you so that you are identifying and securing your endpoints. Now firewall for AI runs on top of our application security portfolio. Our application security portfolio complements firewall for AI because we just want to ensure that the application is secured from all possible attack scenarios. It could be a DDoS targeting that application. It could be a DDoS on, let's say, the lower layers of OSI model, like layer three, layer four, or it could be a layer seven DDoS. Then we also want to ensure that the right bots are accessing the application. Right humans are accessing the application. So that's where bot management comes into picture. Now all when everything is sanitized, then we come into enriched controls around protecting the prompt identifying and protecting the prompts, and that's where firewall for AI comes into picture. Now shifting the use case to a corporate use case. Now let's say if we are centric towards enforcing AI governance, so that's where SWIC will come into picture or Cloudflare one will come into picture, wherein what you are doing is you are ensuring that the right first of all, do a discovery of all the tools that are being currently used. Those could be SaaS AI tools that are used inside of the organizations, which probably IT admins are not even aware of. Now once that discovery is done, you can create policies final policies in terms of which application should be allowed, in what capacity these applications should be allowed. Similarly, for remote MCPs, if you are connecting these to the internal resources, then access comes into picture because you would want an agent to connect to the right resource. So that that's where authentication and authorization has to be enforced. And that's where, again, the access part of Cloudflare one will come and help you so that the right agent is doing the right activity. It's not that agent which was supposed to read the Jira is now deleting the Jira stories. Okay? Yeah. So there is a question around product overview sheet with basic facts. Yes. Those are available on our website. So you can look for firewall for AI. You can look for AI gateway, and you will find good documentation. The good part of Topflare is we believe in sharing best practices and the knowledge that we have incurred, and that will be documented in within Cloudflare blogs. It could could be documented within Cloudflare documentation as part of the datasheet, and all this information is open. Any further questions that we can take? Alright. If there are no further questions, then thank you for taking time out today. Have a good day, everyone. Thank you.