Video: Tech Talk: Running Cloudflare Containers In Prod | Duration: 3384s | Summary: Tech Talk: Running Cloudflare Containers In Prod | Chapters: Introduction to CloudFlare Containers (12s), CloudFlare Container Benefits (117.689995s), Container Improvements Overview (504.965s), Sandboxing with Containers (681.85004s), Containers in Action (1107.195s), Container Wish List (1705.54s), Future Container Features (1896.5249s), Questions and Resources (2467.165s), Container Integration Strategies (2508.18s), Container Persistence and Distribution (2929.59s), Closing Remarks and Q&A (3255.43s)
Transcript for "Tech Talk: Running Cloudflare Containers In Prod":
Alright. Hey, everybody. Really glad you are all here. My name is Mike Nowmich. I'm a group product manager at Cloudflare and been working on containers, since the time I joined. Really excited to talk to you today. We're gonna be talking about, Cloudflare containers and running them in production. And so, kind of a a variety of things. Let me go over the agenda really quick. And so oh, did that flip over? I think it might be sorry. One sec. Are we seeing the bullet point list? Okay. Cool. Sorry. There's a little delay. Anyways, so we're gonna be talking about, you know, just a like an intro of, containers, like, what are containers on Cloudflare? Just you know, if if you're used to them already, this will be a little bit of recap. But if if you didn't even know that Cloudflare was running containers, this will be a nice intro to what we're doing and why we think it's unique. Then, you know, we launched in June in in beta, and I wanna go over, you know, what's changed in the last four months, since we launched. And then we're gonna go over two use cases. So one use case is is becoming, like, really, really popular, and that's that's Sandboxes. And so we have some, really interesting stuff running in prod on top of Sandboxes. So I wanna talk about, like, how we set that up, dig into the code a little bit of what that actually looks like. And then we have, some friends here from New Harbor. Giuseppe, I'm gonna kick it over to him. He's gonna show you, their use case, which is totally different from the sandbox and use case. This still works on top of containers. We'll go over like, hey. What was the problem statement? Why were Cloudflare containers a particularly good fit for this use case, and then we'll talk about, you know, what's next. Awesome. So let's just get into it. Okay. So, why containers generally? So, you know, I'm a big fan of workers. They're super lightweight. They do a lot. They run everywhere across our network. They're super cheap. But sometimes you need to reach for a full container. And so this might look like, hey. I need to run, you know, Terraform. I need to run Pulumi. I need to run various tools that need a full Linux environment. I need to run different languages. I need to run resource intensive workloads that use four CPUs in parallel or that run for twenty four hours straight. Or I need to bypass some limitations of workers. Like, I need to eval some code, eval, or I need to, you know, make more concurrent requests. So all these reasons why you might wanna reach for this, you know, kind of fuller container. Or lastly, it might be, hey. I have some preexisting image that someone some vendor has sent to me, and I wanna run it everywhere across the globe, and I wanna spin it up and down programmatically. Really easy to to do this, on Cloudflare with containers. So that's just like why I would reach for a container generally. But we're we're not only telling you, hey. Use containers, but use containers, you know, on Cloudflare. So what is it that makes Cloudflare containers different? And there are a couple of things here because there there are, you know, a million container providers as you know. So to to kind of explain that, I wanted to show, like, the basics of what a a super basic container looks like. So at the bottom of the screen, you could see that there's this, you know, fetch that's happening that's saying, hey. Get me a container and pass a request to it. So this is saying from anywhere in the world, I'm gonna handle this like ingress, and I'm going to basically just call to a container. This gives you a little hook to start programming around your containers. But in this case, it's just a super simple proxy. And up top, you can see we've said, hey. Import this container class, and I define some stuff about my container. So for instance, I'd say, hey. Here's a default port, and here's a sleep after, which says, hey. When a request comes in, automatically boot the container. But if no requests have been coming in for ten minutes, put the container to sleep. So that's kind of like like a very basic instance of using us. Really easy to configure. There's also another file that just says, hey. Use this image and, you know, this instance type. But that's pretty simple as well. Alright. So that's what that looks like. And and why do we think it's important to have this kind of programmability and integration, with the the workers' ecosystem. And and that's because it allows you to use the right tool for the job, in many cases. So in this case, you know, I have some, some request that comes in. I can very easily, at the very top conditional, say, hey. If it's if it's meant for the container, send it to the container. If it's not meant for the container, just run some JavaScript and run some, you know, serverless JavaScript. Again, that's gonna run more at the edge. It's gonna be, even cheaper, even, you know, easy or, like, more lightweight to execute. And so I can run serverless. I can run containers and they're very easily integrated. And then the last line there with the assets is just basically saying like, hey. Don't even hit the container. Just serve some assets. And these would be, you know, again, cached on the edge. So in order to set up this architecture that's, like, really efficient and really easy to extend, it's just a few lines of JavaScript. It's not, you know, setting up, you know, Terraform resources and writing Kubernetes manifests to do all this, like, ingress logic and routing. It's again just a few lines of JavaScript that really allow you to to build around the container. And some of the other use cases we have are are gonna show even more of kind of the programmability here. And I think that one of the things that we've realized is that, you know, there have been a lot of container offerings that are really good for cattle. Right? If if you've been around the industry long enough, you've heard this phrase, cattle not pets. Right? You're supposed to treat all of your servers as these totally interval interchangeable, units, none of which are unique. But that doesn't work for every use case. Like, a lot of times, there are use cases where where containers actually need to be individualized pets. Right? Like, you need to be have some state in the container that you respect. You need to route requests to a specific container. And and a lot of the, tools that are out there for running containers are are actually more suited towards cattle and not pets. And and we wanted it Cloudflare containers to be to work really well for the, you know, the the pet case. So an instance of this is, you know, the Cloudflare editor. This is the old Cloudflare editor, when I'm I'm, like, editing a worker. But we're actually going and building a new one, and we're actually using containers for this. And and why are we doing this? The reason for this is that, you know, I have, like, things that I'm writing in my specific editor session that go to disk. I have, like, specific memory in the console that I wanna save and I wanna route to. So I have to route requests to a specific container all across the globe, and that needs to be really easy. And I wanna manage the life cycle of these things over time. Right? Like, I want if I wanna you know, we're using Versus code under the hood to do this. If I'm, like, spinning up a Versus code instance, I have to write to or route to the specific instance. And so with Cloudflare Containers, like, what does that routing layer look like? What does that orchestration layer look like? Something like this is super easy. Right? So my top line here, get session ID, is basically saying like, hey. Grab from the request whatever, you know, editor, header, header, or path name or auth token or whatever routes me to the right editor. I get the specific container on the next line with one line of code, and now I'm very easily, like, interacting with that specific instance of the the editor. I can say, hey. Destroy that specific instance very easily, programmatically, and and I can route to it easily. And so, again, the the routing layer, the the orchestration layer is just very, very simple and a few lines of code makes it really easy for these kind of stateful containers to to be, interacted with. And you don't have to write all this, you know, second layer of scaffolding on top of your scheduler orchestrator. And then lastly, I think that the differentiator for for for Cloudflare containers is, you know, the deploy the development experience should be really easy. If if you're used to workers, running locally is just Wrangler dev. Everything works locally. When you wanna deploy, it's just a Wrangler deploy. You don't have to jump through a ton of hoops to deploy something globally. And this is this is deploying to region Earth. Right? So you're not having to deal with many, many regions. You're not having to deal with, you know, hopefully, you don't even have to really think about registries even though we'll get to the cases where you might want to later. But the development experience and the deploy experience should just be single CLI command, and and it works. Alright. So that's that's a kind of brief overview of of containers on Cloudflare. And and when we get into some of these use cases in a second, yeah, I think you'll get, like, a better sense of what it actually looks like under the hood. But really quickly, I wanna go over so what's new since launch? And, let's get into it. So I'd say there are a couple major themes. One of the major themes is, scalability and sizing. So when we launched, we started with, you know, pretty small containers, you know, half of the CPU or four gigs of memory, and we bumped that up significantly. Right? So you can now use, four v CPU, 12 gigs of memory, 20 gigs of disk. This is something that throughout the beta process, we're gonna be pushing higher and higher. But, you know, just within, you know, the last few months, we've we've really opened up a lot of new use cases, and we expect this to, again, keep going up in terms of how large each instance can be. And then there's a question of, like, how many concurrent instances can I run? So, we've also pushed these limits up, you know, considerably for, you know, your average, paying user, and and we will push them up even more. So, right now, you could run, you know, a 100 b CPU simultaneously, 400 gigabytes of memory. That's just for, you know, kind of your standard user. We when we're talking to some of, the customers who are on this, please don't be, like, discouraged by those limits because, if you just talk to us, we could bump this up a ton. And so we have some users who are running, you know, tens of thousands of concurrent containers simultaneously, really easily scaling up and down. And so being able to hit 20,000, 30,000, pretty easily scalable instances is something we could do. So if you're in that camp and you're thinking, oh, man. I I really wanna be able to burst up, in the tens of thousands of containers if I need to. Just just talk to us and we can, we can configure it. Not something we've opened up for everybody just yet in the beta, but, something that we can do, behind the scenes. So that's like the scalability. I'd say that the other big thing is kind of ergonomics, as far as changes recently. And I I grabbed the screenshot because it actually highlights a couple of the cases in which, like, improvements have recently been made. So, for instance, rollouts. So I highlighted the little rollout bar here. You can now configure your rollouts to say, hey. First step, deploy 10% of my containers to the new version, then 20%, then 50, then a 100. So these are really now more configurable. They're actually much faster than they were on launch. The next kind of ergonomic thing is servicing errors really easily. So if you get any errors across your whole fleet of containers, we now service them at the top level, and there's a little button to say, hey. Jump to logs for the specific container. And then logs. You can now get, you know, truly live logs with a click of a button across any of your containers super, super easily. So it's it's really easy to come into the dashboard and and and logging is just set up. And you just configure observable equals true to get that. Alright. So that's kind of a lot of what's new. And then I wanna, you know, talk about this use case that we've seen a lot. And this is kind of our our first of the kind of things that I wanna dig into in terms of, using this in production, and that's the sandboxing use case. And so when we launched this, honestly, we thought we were gonna go and build, you know, kind of auto scaling and load balancing next. But we had so many people come out of the woodwork and say, hey. When we talk about pets, not cattle, we really are thinking about, you know, sandboxes in some way, and, like, isolating specific containers, for often, you know, AI agents or LLMs to execute code that may or may not be trusted. And so I wanna dig into the sandbox use case a little bit, because I think it's, you know, super important. So I'm going to, share my whole editor or my whole screen really quick, and then we can look at some of the code. One sec. Alright. Cool. So, in this case, I have some code, and what I wanna do is I essentially want to be able to spin up a container and execute arbitrary code on that container. And then I wanna be able to spin up multiple containers. Right? Like, I want control over this. And so, you know, this could work in Python or or JS, and we've actually built this sandboxing library on top of containers to make this really easy. This is all open source, and it's it's something that anybody could do themselves on top of containers, but we've, you know, just kinda provided these libraries that that make it easy. So, I have some JS code. You could see it's super simple. It takes the length of a string and returns the length of the string. So, like, presumably, this would be something that's, you know, generated on the fly, but I've just kind of hard coded a template here. When I want to route to a or, like, get a specific container, you know, I'd presumably be taking the actual ID from something like a header or an auth token. But in this case, I'm just saying, hey. Depending on the path I give you, spin up a different sandbox. And so that will, again, say the first request that comes in spins up the container, and then I can execute some code on it. And then in this case, I'm saying if I go to the slash JS route, I just pass in that that code to this run code command. And so this run code command is a really good example of the RPC calls that you can associate with specific, container instances. So in that container class that I showed you, you can write individualized, methods to say, hey. When I'm when I'm calling this on this specific container, here's the run code command, and it's going to do x y and z under the hood on this specific instance. So, this is the command, and I'll I'll just show this working. So I have a container up here. And sorry, the URL is probably small, but I'm gonna give it the ID demo, and I'm gonna say j s. And then, I believe I called it input. I'm gonna say input equals, hello world. And I run that, and now, you know, it's it's delaying a little bit because that was spinning that was cold starting a container from scratch, and then it took the length of held world. And it said, hey. You know, that was 10. For it to say hi, You know, that's only two. So now I'm able to, like, very quickly pass request to that now that it's cold started. If I were to spin up, you know, I'll say demo two as a totally new, instance, I do that. It's gonna load a little bit longer, and it's gonna spin up a new instance as well. And so that just executed. Again, a second container respond up on demand quite easily. Some of the other things that we could do here, you know, we could execute Python code as well with this run code command, with sandboxes. You know, we could actually exec, specific, commands. So in this case, I'm execing echo high within this container. I could also say, you know, command equals PWD, and it gives me the slash workspace. Like, so, yay. That actually, like, calls down to exact code in this container very easily for me. And then lastly, you know, we could even interact with files with write file or read file. And so when we have this use case, it it becomes really easy to, you know, build things that are, again, spinning up containers on demand. And what the the UI now looks like, you know, I had just spun up two different instances because I I passed it two different params to spin up different instances. And you can see, hey. These are running. I'm based in Seattle, so it gave me two locations close to me. One was in Seattle. The other happened to be in Los Angeles, you know, same general region. That's something we're also improving. So, like, more and more, we'll be getting closer and closer to you, as we kind of, like, prewarm things effectively. And then these things will automatically spin down after a preconfigured limit. And then, you know, I can grab it, come in here, get my logs and whatnot. So that's that's, like, generally what it looks like. And we built this sandbox library, which, again, I think is really important to show the kind of programmability here in terms of it's really easy to, you know, parse commands and then actually spin up these things on demand and and spin them down. And, you know, the the title of this is running, you know, containers and prod. This sandbox library is actually what we use to to run this build dot cloud cert dot dev tool. And so, you know, by coding, I'm sure you've seen a million prompts that look like this, right, for for various kind of, like, specialized, use cases. This is actually the thing, that that powers this build dot cloud for dot dev. And this is totally open source, so you could look at how it's built. But if I wanna say, you know, create a, vapor wave to do list app, I I run this. What this is now doing is in a very similar fashion to what I just showed you. It's it's saying, hey. Spin up an isolated instance of containers, execute code on it. And in this case, it's saying, hey. Get the LLMs code. And then it's doing things like, and over here on this, this side is some ones that I just built beforehand. Right? I'm able to, like, get a live preview and and pass request to the specific container. I'm able to introspect it and see the actual code running in the container. And this is kind of the the core building block, the sandboxing mechanism for this, you know, build that Cloudflare. Dev platform, which, again, if you just go there right now, you can start playing with it using containers under the hood. So there are all these use cases again where you have these individualized instances you want to interact with, and it becomes really easy to do so on containers. So I'll I'll link to this stuff in the show notes. This Vibe SDK is the thing that this is built on. Again, you can kind of go in and and see how this is working under the hood. Alright. So that's kind of use case one for me. Use case two, I wanna, bring up Giuseppe, from New Harbor who's gonna show off a little bit out, how you guys are using containers for, again, totally different use case, but still has these dynamics of I want some programmability. I want to, build around the containers. So can you be, like, go into, like, what was the business problem you're trying to solve? What were the technical challenges? And then, you know, what does that look like on Cloudflare? Yeah. Absolutely. First, I wanna say thank you for for having me here with you. Cloudflare has been foundational to to our entire company since we started it and almost our entire tech stack runs on Cloudflare tech. And so when containers came out, we learned about containers. It just was a natural extension to the point you made earlier about just making it really easy to do a bunch of new things we couldn't before. So my my goal here is I'm gonna share a little bit about New Harbor, who we are, what we do. I'm gonna then share power using containers live today, and then we'll dive into a bit of the tech details. And then, Mike, we can we can, dive into any questions from there too. So, New Harbor, helps organizations get basic security hygiene in place. Think like I have all of my multifactor authentication in place. I have all of my settings in place. So we we scan and integrate with a bunch of applications through browser extensions, and through APIs to make sure that an organization has everything in place, so they can make sure that their their the configuration of their applications are are are secure. Now one of the things one of the challenges that we had recently was how do we actually add and expand the scope of the security checks that we're doing. Right? So we're we're integrating with apps. We're using APIs. But what we wanted to do was we wanted to actually add additional checks for scanning on, email security and website security. So think like security headers on your on your website as well as your DNS records to make sure that things like DKIM, are set up properly. And so we looked and, obviously, there's some great open source tools out there. The two that we picked were MDN HTTP Observatory and Check DMARC. But the question as we were trying to figure out how do we integrate those checks, how do we actually take advantage of those open source tools to be able to display those results to our users was, how do we do it without a ton of overhead? Right? Like, all of these tools are in the different languages, have different infrastructure requirements where we'd have to spin up VMs. Like, there was a whole set of considerations that we had with regards to, like, we want to do this thing, but it's gonna be very distracting for our team. How do we how do we do it efficiently? And that's where kind of class like containers came in. And so, what we ultimately decided to do is run those open source projects in class like containers and integrate them directly into our app. And so I'm gonna demo for you now, like, what the end result is so you can kind of grasp what we're talking about. And then I'm gonna kinda walk you through kind of what's happening behind the scenes. So, let's hop over here. So this is oops. Sorry about that. This is our, you're not seeing that. Sorry. You're still seeing demo. Is that correct? Yeah. Well, not anymore. Oh, it was it was, lagging there. How's that? Good good now. Yeah. Perfect. So this is our new hardware security checkup tool. This is kind of a standalone application, not part of our core app. It's something think of it as like a report card where any organization could come and get a quick scan of do they have their website and email settings and and even Google Workspace settings set up correctly. And so this is all running on Cloudflare tech. It's running on Cloudflare Workers. But what we're gonna do here is we're actually gonna run live here. So I'm gonna put in my email here And, we'll get go ahead and run it, and we're gonna start a security scan. And so all I did was put in your your name and, website, and it's gonna, in the background, start on start running those scans. So these what's happening right now is those containers are in parallel in isolation, actually spinning up, running those two open source projects on the domain, and it's gonna actually then use the worker to process those results and and then share them back to the user. So you can see here, that was kind of one step. Everything happened in the background, and now we've got a our checkup results. So we've got a b score. We can see all of these different email settings, how my security is doing on my website. If I dive in, I can understand the check. This is a bit of the value add we're adding on top of those open source tools, like making it really easy to understand what even is an SPF record. But basically, what happened is we were able to just start that worker, call those containers, run those jobs, and then process those results. And it's something that we were able to do very seamlessly without really thinking about infrastructure. So I wanna pop back over, to the slides here. Let me know. It seems to lag a second here. So let me just know if the demo comes up. I'll re share just because it seems to tilt it into place. How's that? Okay. That should be good. Alright. We're done with transition. So that that that that should take care of that. So what's happening there is our security checkup tool, is running in a standalone worker. And as you saw there, the user will initiate the scan. And then once the scan is initiated, the worker is actually gonna call a couple of containers in parallel. Right? So we have one built for Check DMARC, that opens our tool for email security, and another one built with MDN HTTP Observatory, for the website security checks. And so in parallel, it's gonna fire up those jobs. Those containers are going to spin up in isolation, run the the actual command line tool in the container, and then send it back to the the main worker who can then display it to to the user. It really is as simple as this chart. And so I I wanna kind of dive into, like, what we're actually doing here just at a very high level. So this code here, these these these five lines here is in our main security checkup tool. Right? So what we're doing here is we're we're, we're basically just, binding to the durable object that is hosting the container. Right? So as simple as the check mark d o, we're also generating a random ID. Now this is a design decision that we had. This is not what you need to do. But our design decision was because we are a security tool, because we're accepting arbitrary input potentially, we wanna make sure that all of the jobs are as isolated as possible. And so our design decision was we're gonna, spin up a new container, do the job, and then that container is going to be destroyed. And I'll show you how we do that. But that random part is really important for us, and you may have used cases yourself. I think this was kind of kind of the innovative thing for us is if I had to do this anywhere else, like, in most other places, it would be incredibly difficult to, like, just get the the beauty of that simplicity. So you hear, we create the unique durable object and then what we're doing is we're calling the function. And the function in this case is, like, let's run the DNS scan on the domain. Right? So quite simple. Now what's happening? That's actually making RPC, call over to the durable object that is holding the check demark container. That durable object is actually going to now, receive that command. It's going to initiate a new and spin up a new container, and then it's gonna actually, make a, a fetch request into the server that is running on that container. And so then, that server will then in the actual container. Right? So this is the durable object code. The durable object will call the container and the container is what is running, the actual command line tool. And so all that's happening is that, the container is now receiving that request. It is understanding that run DNS scan means this command line tool. So check demark. So we're doing a mapping from the API that we have into, the actual command line tool, and then it runs that command line tool in a secure way runs the command in a secure way and then returns those results back to the durable object. And the final thing the durable object does is it destroys that container. Right? So you can configure Cloudflare to, like, time out after about ten minutes, for example, and the container will self destroy. For us, we're like for billing reasons and a bunch of other reasons, we're like, we don't need it. Everything should just go away. So soon as this job gets processed, container is destroyed, results are returned to the worker and that's it. Right? So, the simplicity of being able to do this without thinking about infrastructure was the biggest reason that we said this is the the technology that we needed to use to be able to power these checks. So I'm gonna run through kind of the three the three reasons. The first one I mentioned here, the the infrastructure is super simple. It's integrated with Cloudflare. You you can see I'm not really doing any fetch requests over the Internet. Everything is running parallel. Right? So, like, I can spin up I've got if I have a 100 different open source projects, I can just spin them up, like, without thinking about anything. And if I have a 100 users, I can also obviously auto scale up and down to zero. So the fact that I can parallel and auto scale across projects and users at the same time makes me not have to worry about the the, the scaling problem. And then the other part we obviously talked about was that everything is isolated. So I spin up a container, has the perfect image that I need, it runs the job, and then it gets destroyed. And so that makes it super simple for us to be able to have confidence in the, the segregation of the, customer data. The final thing which is kind of not related to the other parts, was we built a set of templates around Cloudflare containers. So two config files, one around the the actual Docker image and the other around the API to function mapping. And so I can spin up a new open source project in, like, five minutes with two config files. And so as we got more familiar with the tooling and as we were able to spend more time with it, we were able just to move faster and faster. And, obviously, as, like, a startup, we wanna be able to test things rapidly. And so what Cloudflare containers basically did was allowed us to test a whole new set of, security checks in in in basically no time. And I think that's the magic of the platform is it's, like, super easy to use both from a developer experience, as well as from, like, an infrastructure, side. Awesome. Yeah. Just, Debbie, I I was thinking, you know, we we have this, Vibe SDK, and, there's so many, like, CI tools that are kind of, like, a pain to integrate into your system. And I, like, I kinda wanna make, like, our our, CISDK or something just so you never have to deal with, like, spinning up Jenkins for some, like, random workloads like this. Right? Like, just, like, build it in two seconds, and, you're good to go as far as, like, batch jobs and programmability around that. A 100%. Right? Like and, like, happy to, to collaborate on the templates and things like that that we have. But, yeah, it it just the the underlying tech that exists there just makes it easy to make, like, really elegant design decisions. That just, like, otherwise would take us, like, days to even, like, contemplate. So, yeah, that I I completely agree. If if if a tool like that, I think, would have a ton of use cases. Yeah. And, you know, you had mentioned that kind of, like, you know, you're you're spinning these things up, like, and then spinning them down. I think that's it works for in my instance, like, we're keeping them up. We're routing requests to them, you know, potentially from around the world. Two very different use cases. Like, one can be super long running, and we keep it going as much as we want and addressable for many different use places. Another one could be, you know, short lived and totally locked down and and meant to be isolated. Another thing which I think I'll touch on a second is just like the isolation. I think the auth story is pretty good. Right? Like, you didn't have to expose any ports to the world, right, to to get this going. It's it's all within the the cloud for boundaries. So, you you know, you you didn't have to worry about, like, hey. I passed my, like, JWT token in, and that did it right off. It's just like, no. No. It's locked down by default. And another thing here, as far as, like, the the lockdown, and I'll I'll get to some of this, like, detail around this. It's like you can lock down egress as well. So if you had some workload that had to run-in the in your case, I think this is totally it wouldn't work. But, hey. I wanna run some something in isolation, truly in isolation that's like worker code. You you could just say, hey. Don't even talk to the Internet. There's some more interesting stuff I think we're gonna be doing, around that too. Cool. Just have any any, like, wish list items for, containers? Honestly, I think it's been incredibly magical to use. I think the thing that you mentioned there about providing some more templates would be great, just so that we're using your your your stack and and your syntax, in an easy way. I think the other we we have a a very unique use cases, which is sometimes we need to run Windows VMs, and so that's, like, really, really different. And so that would be, like, I would say, like, on a very long tail or but now I'm stretching here because of, like, how much runway we have with with what we have here. But I would say if that was, like, a super wish list item, it would be just, like, that would be one that would be one with, like, a super niche use case that we have. Cool. Nice. I I can't make any promises, on running Windows on Linux boxes, but, you know, noted. Awesome. Well, cool. Thank you. Also, just that'd be like, hopefully, we've got a link to New Harbor in there just so people could check it out if they're interested in the top level use case as well. So, we'll make sure to to link that in the show notes. Cool. So, yeah, I'm I'm gonna talk about, like, what's next too, with CloudForm containers because there's there's a lot of direction we wanna go into. And so the the first few things I'm gonna show are, like, literally coming the next, like, week or two. Like, these are PRs are open in review. You know, some of that I think might be merged today. And then I'll show some other stuff that's gonna be like next quarter, or after that. And then I can talk a little bit more about, slightly more long term here. Okay. So what's next? Faster cold starts. So right now, if you if you, launch a, like, a fairly basic app, it's, you know, around, like, a two second, a little bit more cold start. And if you're loading more into memory, that's gonna be longer. So we've got a change that should shave off, you know, around eight hundred milliseconds on average. So you're gonna be close to a second on the cold starts. And this is gonna be something we keep pushing further and further. So I'd say imminent improvements on the cold start front, but we should be getting sub second, you know, I guess, you know, early next year. We actually just hired one of the founders of Docker, Sebastian, who is actually, like, tackling this problem head on. And he knows, like, a little bit about containers. So, cool. So faster cold starts, external registries. So there was a question that the show notes entered in the chat, not show notes. You could tell I listen to too many podcasts. So, in in the chat around external like, how do I integrate this if I've already built my stuff elsewhere? So right now, when you push a Docker file, it goes up to our registry, which we it it's part of your account. We manage it for you. You shouldn't have to think about it. But we we will have some work for external registry soon with private auth. And so you'll be able to say, hey. I'm using ECR. This is my auth token to do that, or I'm using Docker Hub, stuff like that. And we're gonna be fleshing this out with, you know, Google registries and JFrog and all that good stuff coming soon, but these will be the kind of first two we land with, in the coming weeks. Then this one I'm super, super excited about because of kind of the implications on state. So FUSE support. So, hey. I have a file system, and I wanna interact with that file system, and then I want to, you know, come back to those files in the future. Allowing FUSE on the VMs, this is actually something that just got merged, is going to allow for, interactions like this. So, and there's a PR up for this. I could also find this and link to it, where we have this mount bucket command on the sandbox. And I can say, hey. There's, an s three compatible bucket. So it could be s three. It could be whatever Google's s three is or, you know, ideally, you're on r two, to get, you know, even better global reach and and cheaper egress. But you basically take an r two bucket, mount it as a part of your file system super easily. And then for all of these workloads that are, you know, editing files, hey, I download something from Git and I start changing those files or I'm, you know, I'm writing x y and z to disk. You can very easily then remount that bucket, come back to it in the future. And so this is gonna be a really important building block for these these stateful workloads. And and again, the the the few supports out there right now, so you could go and and roll this yourself. But we also are gonna be adding this, you know, kind of mount bucket command, to the sandbox, which should make it much easier to do so. New pricing. So I I wanted this actually to be the, like, the the official drop of the new pricing. A a lot of the feedback has been, you know, hey. You know, this you know, the pricing's fine, but it would be great if it were lower. And and, hey. It's it's gonna get lower in the near future. This is something I would expect next week will drop. I can't give the details that much on this other than, you know, pricing will be coming down and simplifying a little bit. So I have to be a little bit vague about that one, but, just stay tuned to the next week or two for that. And then simpler config. And so on the config front, you know, I kind of showed the workers config. There's also this this config where I I, you know, I'm defining how to route to things. I'm defining this state that's associated optionally with my container. And this is, you know, just frankly too confusing. And so, one of the changes we have coming in the next couple weeks is that you can drop the durable object, declaration, and so that's just implicit. And then I'd expect the next change to come, probably, you know, early next quarter, you know, after the new year where we could drop the migrations. And so, in the next, you know, few months, you'll just be declaring containers like this. Super, super streamlined. Obviously, there's more config that can go into containers if you're, like, customizing your rollout behavior or, defining a specific instance type or or whatnot, but, should just be much much slower and and lighter weight. So that's that's coming soon as well. This is coming a little bit later. Again, I'd expect this probably early next year. But, right now, you have to npm install this container class, and we are gonna just be building that right into workers. And so any updates that roll out in the container class, you'll just get for free. You don't have to worry about, oh, I'm gonna yeah. You don't have to think about node modules at all, which is always nice. And so when you're defining your container or you're defining, things like on start or on stop or on error or custom RPC methods, this is just gonna be this class is gonna be built right into the, into workers. And then this one I'm super excited about. And I was talking about Egress with Giuseppe a little bit. Like, you can currently lock down Egress or not. You can say this container doesn't talk to the Internet at all. But for a lot of use cases, you might wanna execute some code that needs to talk to to the Internet. So imagine you're running cloud code and, you want cloud code to be able to hit GitHub and you want cloud code to be able to hit your own GitLab instance that is that is, you know, has some auth token. But you don't want arbitrary code to be able to get your API key, and you don't want it to hit anything else. Right? So that's, like, really hard to actually configure. Right? You're gonna have to, like, go into your, you know, Etsy host file in the in the in the container and tweak things, then you're gonna have to figure out some auth system that exchanges tokens. What we decided is, hey. What if we added this outbound fetch, handler that essentially intercepted outbound requests? And in this case, I'm intercepting an outbound request. I'm attaching an additional header, and then I'm letting that request go. But there's all this arbitrary logic that you could put in here. Like, you could manually, you know, allow us, deny us certain sites. You can, you know, redact certain information coming from inside the container. You could add information in this case. So you have this control kind of outside of the container to to tweak, you know, outbound HTTP requests. And then also we're we're kind of exploring how do we, you know, lock down UDP TCP as well with a similar mechanism. So this this egress, layer again, I think it's super important for the the cases where you wanna isolate things, but they still have to talk to the outside world in world in some way. So, again, AI agents are are the main one where people go to because it's so untrusted. But really anything where where an end user is writing some code or or there's something untrusted and and you wanna be able to to lock it down in some way, this is gonna be really easy to do, with containers. And then, lastly, container to worker calls. So I can attach some, like, host name to my container, and then I can write arbitrary handling of any request to that host name. And so similarly, you know, you could intercept any outbound request, but you can say, hey. If I hit, you know, k v dot c f, it's gonna route me to the Cloudflare KVStore. Or if I hit other dot c f, it's gonna route to another container. And so this is gonna make it, you know, essentially really easy to make this, like, programmable service mesh on top of containers. So this is something you've done. This is in flight. I I don't expect it to land in the next few weeks, but I'd expect it, you know, early late this year or early next year, to land. The the last thing I'll say before we get to where to start is, you know, we were kind of expecting that we'd go, like, really kind of double down on the, kind of auto scaling stateless workloads as well, on top of Cloudflare. We we do expect to do that. I think that's something that we're really gonna be getting into early next year, where it can be, hey. I just deploy and this thing is, these stateless servers are are just placed all around region Earth. But that's something that I think, you know, we'll see a lot of a big push for early next year for these kind of, like, stateless workloads. And so if you do have anything, like, any use case where you're like, hey. Cloudflare has a network and they're you know, we run, for instance, in Italy, several locations. Right? Like, it's it's really hard to get, another cloud provider that has several locations in Italy and several locations in Brazil and several locations in, you know, Argentina, South Korea, like, all these different places. If you do have a use case where you do need to say, hey. I need to stick containers specifically here. Just just talk to us right now. We'd we'd be super interested in in talking to you. We're talking to a lot of people who have, like, media forwarding. So, hey. I'm I'm running some, you know, selective forwarding unit for, real time chat or a video call or I'm I'm processing data in such a way where we can't use Cloudflare real time. We have to run our own containers, but it has to be in x, y, and d countries. We're we're super interested in talking about that, but that's, you know, not something that's in the public product yet. But I I'd expect again next year, we're gonna push more in that direction. So lastly, where to start? If you go to this link, this is probably the best place, developers.cloudfire.com/containers. There's a nice little getting started, guide there. You could run through that. Should be super simple. And then as always on, you know, on Twitter, Mike Nowich, at Mike Nowich underscore CF. So, feel free to hit me up if there's anything else. Cool. I I think that's it for for our talk track. I'm happy to kinda open it up to questions. I see some in the chat. If anyone else has any questions, please just add them to the chat. And Giuseppe, yeah, if you wanna come back up here, just in case there's there's anything for you. Let me go through. Okay. So I see Carlos says, how could this work if I have my all entire infrastructure in containers and I wanna start testing with existing code? So when I define a container in the the wrangler dot JSON config, you can point to either a Dockerfile, so just like a local Dockerfile. And when you do wrangler deploy, we will build that Dockerfile and push it up to our registry. Or what you could do is you could take an existing Dockerfile and then just push it up to our registry. And I mentioned the external registry is coming soon, but, you know, existing containers, the actual content of the container, you know, you should be able to just take what you have. Right? Or we're not doing anything super crazy with the runtime where this has to be, like, x, y, and z specific type of container other than SDAMD 64. You know, ARM container is not supported yet, but I think that will be coming. And, other than that, your images should probably just work. Right? Like, unless you're doing something super funky. And so you should just be able to point those either Docker files or, you know, just mirror the image to our registry. So that's one thing. Another thing I'll say about integration is, we just launched something called Workers VPC, which allows for you to basically start talking to the Cloudflare development environment very easily, from a a private cloud. And so if you are on a private cloud and you say, hey. I'm running all my infrastructure over here, but it'd be really nice to kind of, like, reach out for some specific use case, but still be kind of, like, totally private. I don't really have to worry about auth. You don't have to set that up. It's now really easy to do so. So, that's an option. Let's see. I'm going through other questions here. Is it possible to manage traffic for container instances? Imagine new deployment and we wanna manage percentage of our traffic to new container. So the easiest way here is probably to do, like, a blue green deploy, type thing with, like, you know, two kind of container applications and then route in workers. You also have the ability to customize your rollouts and and not auto progress rollouts. So when you roll out a new instance, you can say, hey. Only wait for the manual, steps to roll out those instances, and I have to, like, hit an API call to be, like, progress to next version. And so you can say, hey. When I do my rollout, I'm gonna say, you know, hey. I have a canary that's, like, 10% and then wait till I manually can progress to the rest. That's actually how we do things internally here because we have our own kind of rollout system at Cloudflare, and we're we're dog food in containers all the time. And so, that's possible. We don't have, like, a multiple version set up for that, though. So, like, that that actually gets harder when it's like, hey. I have, like, four versions. I wanna a b c d test. There's probably not a great solution to that other than just, like, manually build out four container apps. Cool. Giuseppe, did you ever run into that as far as, like, rolling out new versions? I'm curious. We we have not since we've been able to just keep everyone on on the same one. I think more for us is actually rolling out different security tools and then rolling them out, which because of the infrastructure, that part is really simple. Right? I could spin up, hit the new container, but not so much on versioning. We've been we've we've kinda kept everyone on the same one. Awesome. Okay. That makes sense. Yeah. And the rollouts again are are really easy. Basically and I mentioned this, like, you say rollout, there are the various steps and you could see it progressing through the steps. That could those could be manual. And you there's also a an override to say basically, like, keep old instances up for this amount of time. So, again, with, like, a one liner from your CLI or in your config, you'd say, hey. Like, allow all instances to run for five hours before I kill them. The the rollout is gonna take longer, but it it it won't kill user sessions mid flight. Which is which is also why, for us, it's it's because we destroy everything in, like, less than thirty seconds, we it's, it's a bit easier for us to manage right now with this particular use case. Yeah. Makes sense. And, let's see. We have Dave again. If we wanna do, data processing pipeline with private medical data, should we take any special considerations into this setup? Is this a good environment for that? So, I think, yeah, private medical data in general should be fine. One of the things that, you know, we've talked to actually a lot of companies that need, like, FedRAMP compliance, which is like, you know, as soon as someone said FedRAMP, sometimes my eyes lays over, but it's it's pretty important for a lot of companies. And if you're one of those companies, we, are getting FedRAMP certified, I guess, not currently, but imminently. And so, that's something that I'd expect to be there soon. So if you're like, oh, man. This looks great. I wanna run sandboxes. I wanna run isolated containers in a FedRAMP environment. Please talk to us because that'll just, like we can get you in our queue just to to reach out to you immediately once that's that's ready. And and but I you know, for, like, private medical data, I'm actually not sure exactly the implications there if that's like, I I'd imagine it's not dependent on FedRAMP and and where we should be, you know, fine compliance wise. But, I'm probably not a great source of truth on that, honestly. Oh, there's a whole QA tab. Let's see. Is the sandbox package already available for Rust? I don't believe so. So if you're using a Rust worker, you can do Rust to JS interop, And so you could just be, like, running it in JS, and unlike most of your logic is in Rust. But the the actual package itself, just JS, unfortunately. Glad we have some Rust stations in here. When we do have stateless cattle, what options do we have for auto scaling outside, writing our own JS load balancer? Raymond, I don't have a great answer for you right now, and I it pains me that I don't. And part of the reason here is, again, I thought this would be something we were doing, like, two months after launch. And there were so many people who were like, I wanna be able to scale this this kind of PES instance as soon as possible. And so, I'm super committed to this long term, or not even long term, like, medium term. Like, it's it's queued up for us to get this out, early next quarter to, again, just say, auto scale across the world true. Hit this percentage of memory or hit this percentage of CPU, and then we'll you know, with some max instances you could set. But right now, it's it's very basic, you know, just like round robin, load balancing that you can write yourself. So I I again, it pains me that that's the answer right now. But, and and we're, again, very committed to that long term. But this this specific use case was just like there was so much heat on it that we we focused there first. Do Cloudflare containers support KVM? Not sure what that would mean in this context. I could talk a little bit about the the the runtime environment, and and that's basically just that we're, spinning this up on top of Firecracker VMs, under the hood. So every every container is actually isolated by Firecracker, And then within that, the actual container runs. And so if if someone were to, like, escape the container boundary, they're still isolated in in the VM boundary, which is nice. Daniel has a a question around IPs. I think this is also something that we'll get to early next year. And so, exposing I p v sixes per instance and then also having, we we have, like, a global load balancer working at l four layer, that we don't have the finishing touches on. And so I'd imagine early next year that comes out where I can just say, like, hey. I can send UDP packets to to load balance over my containers, but, unfortunately, not there yet. So as far as single IP addresses, this would just be something that, you would have to hit the hit the worker router and then it routes to. Chris, you mentioned persistent volumes. So it would be great if that could be persistent with the underlying durable object. Yeah. So let me talk about persistence really quick. I actually should have gone into this a little bit more. So I mentioned the FUSE file system. I think that's step one where it'd be like, okay. I wanna, like, mount the an r two bucket, and we'll have the the primitives for that soon. There's also this, idea that we're working on of, like, checkpointing specific, instances. So I spin up a container, and then there will be we're debating calling it snapshotting or checkpointing, obviously. But I can just basically say, like, save the full file system to disk and optionally save memory as well. And, then next time I address it by ID, I could just automatically load that in. And so that would be the entire disk rather than, like, I'm mounting some specific persistent volume. That we found when we talk to people is actually, for a lot of the use cases right now, what people want more than, like, a traditional persistent volume where it's like, I'm writing my Postgres data here. And so that's probably gonna be the next step. So the first step is, like, fuse file system, r two mounting. Second step is this, persistence, layer with with kind of checkpointing where I come back to my ID, and then we can quickly load your your, your disk back to the state it was previously and then memory after that. And I think after that, we'll get into, like, more persistent volumes that are are, like, you know, your, like, classical, like, really high, read, write, throughput. But, you know, that'll be kind of, later on. And then as far as the durable object goes, the kind of container to worker flow will you'll be able to talk to your durable object very easily that way. And so you could be like, hey. Write all this stuff to the the SQLite database in, in the DO. And so that will be an option too. So we're kind of attacking this from multiple angles and depending on, like, if you want full file system, what your persistence needs are, what your throughput needs are. Various of these options will be the you know, a specific one will be the right call in terms of, how you do persistence. Cloudflare workers are distributed across the world. How does that work with containers? Will users have different latency to performance depending on where they are and where the container is running? So the short answer here is yes. Right? Like, we can't get around different latency depending on where you're running. What we will do is we will pre warm a bunch of containers, in various places around the world. And then when a request comes in, we route it to the nearest kind of pre warmed container. Right? Like, if you haven't seen the ID yet, it will say, hey. Boot up the the closest container to you and and use that. This is actually something we're iterating on. At very low counts, we actually have some some times where, hey. If I only have, like, NACS instances of two, the container will be too far away. So this is one of the things that is is some improvements here are coming soon. But, basically, we will route to the closest container we have pre warmed. And then as we spin up new replacement containers, we will try to spin them up closer to where the traffic is coming from. And so, this, you know, essentially, if you're if you're spinning up a bunch of instances from this place, the first few should probably be in your region. If you spin up a lot really fast, they might be in farther distances from you. But then over time, it should, you know, be closer to where where the traffic is coming from. This is something again, like, honestly, we're not, like, perfect here yet, but we are, kinda iterating towards this. So it won't be in your absolute closest location, but it should be pretty close. Let's see. I think that's that's a good spot to end. I I got through most of the questions. I'm sorry if oh, wait. There are a couple more. I will try to answer these probably after async. Daniel, your your firecracker question, can instance be frozen rather than being destroyed? Yeah. This is kind of the the persistent stuff that I was talking about. It's essentially exactly that where we just freeze the VM and then resume it from a snapshot. So that's that's part of what's happening in, the in flight work right now. Fernando, your question about workers or containers, I would say, you know, try to use the worker, until you need the container. It's gonna be lighter weight. It's gonna be more edgy. And, you know, I'm the containers guy, but I also love workers. Right? And they're you're you're gonna be cheaper, lighter weight, and and closer to the end user there. Even though, again, containers are great, so I don't wanna undersell them. Anyhow, Robert, okay. This will be my last one. I'm sorry. And and the reason it's my last one is you called out Nomad, and I'm love to see a Nomad user here. I was actually the old PM of Nomad. So very familiar with Nomad. Props to Nomad. If you're sick of Kubernetes, and need something similar, that's a great option too. But if you have an individual container, app API running in Nomad or Cloud Run, support to Cloudflare containers. We just write a simple worker and forward all requests to the container. Yeah. It it should be that simple. And so if you follow the get starting guide, it should be really easy. Alright. Cool. I I've been rambling enough here. I will if there was any question I missed, please, I'll follow-up after the chat. And, Giuseppe, I just wanna say thanks again, for demoing. That was really great, and thank you to everybody for listening. Reach out. Let me know if you run into any snags, and, really excited to see where this goes. Appreciate it. Thanks, everyone. Thanks, Mike.