Name: Migrate from S3 to R2
Uploaded: 2026-03-17T09:46:59.898Z
Duration: 1 h 5 min 4 s
Description: Migrate from S3 to R2

Transcript for "Migrate from S3 to R2": Hello, and welcome everyone to today's live workshop. If you have been building web applications or designing data platforms, storing large amounts of data, especially powering complex pipelines that rely on object stores, you're in the right place. Today, we will be learning how easy, it is to migrate your existing data from an s three object store to Cloudflare's r two and leverage its unique features. But before we dive in, allow me to introduce ourselves. I'm Umkar. I'm a senior solutions architect at Cloudflare here in Singapore with focus on our, AI and developer platform products. We have my colleague, who is colleague, Timo, who is a senior solutions architect joining us from Munich. Timo has been kind enough to join and help us out with any questions that might come up. Now to make sure that our workshop Rajesh, is interactive, I will be asking questions. Watch out for the poll section on the right, where you will see questions pop up, when I bring them, and drop in your answers when you see anyone. All of those are MCQs, and you're not penalized for a wrong answer. So I encourage you all to try it out. Also helps me understand, how much of the content you're following. Okay? So let's warm up to this. Here's a survey question for you all and for me to learn about our audience today. Have you used the Cloudflare tool to support your storage needs yet? You can answer either with, yes, I'm running it in production right now, or, yes, I'm considering, but not sure, or, no, I'm I'm here to learn more. Either of those. I'll give you ten seconds to respond so I can understand what the audience looks like. I see some words coming in already. Thank you for being interactive. Okay. I see a I see a lot of words. I see that few of you are already using this in production. So congratulations on that. Few of you are considering this, but a majority, I think, about 80% of you are saying no. You're here to learn more. Give me this wonderful opportunity to introduce you to our tool and also show you how you can use this with some of your existing applications that are relying on s three. Once again, thank you for keeping that interactivity going. Also, feel free to use the q and a section to ask questions if if you have any. Okay? Let's move forward. So our agenda for today will be first to take a look at the relevance of object storages and why they are important so that we just set up that ground. And for those of you who aren't introduced to r two, we will run a quick primer on what r two is and what makes it different. Further in, we will focus on what s three compatibility means and why it is appealing as a developer. We shall then jump right into the meat of the topic, looking at our native migration tools with live demos. We will discuss a couple migration strategy that will help you also make that switch seamlessly. I'll also introduce you to the data platform capabilities on our tool that are in beta. And if you have again, if you have questions going throughout the workshop, please feel free to use the q and a section. Timo will be helping us over there. And I'll make sure I keep an eye out in between so that I can address your questions live if need be. Okay? So with that, you have the opportunity to follow me during the migration process that we will be conducting during the demo. However, it's completely optional. You might incur charges on either AWS or Cloudflare during this activity. So don't forget, especially if you're trying this, don't forget to delete your buckets after the workshop. That's that's basically a reminder. You can see the docs section, for a prerequisite document if you'd if you'd like to follow that. But, primarily, you need to have a Cloudflare developer account with ARPU subscription enabled and an AWS account with a s three bucket setup, especially with the access keys and secrets because we won't be creating those secrets on AWS during this demo. I'll show you everything that needs to be done on Cloudflare dashboard. Okay. So if you're comfortable with that checklist already, let's start with why does object storage even matter. Now we all know why object storage is critical, but let's use it as it's the it has become the backbone of modern web, which is scaling through a lot of unstructured data. And whether you're delivering high res media files to end users or hosting application binaries or even storing massive data lakes, object storages are where the data lives. The fundamental expectation from this object storage is generally that you put the data in and you read read data out. That is it. You don't want to worry about scale. You don't worry about the structure of the data. Just use it to add data and read that data out. However, while these object storages are cheap to store, they are expensive to read. The cost you bear is a function of two metrics, the number of read operations and the egress or bandwidth that you consume when you're reading your data. So, essentially, every time you're reading your data, you're paying a premium on it. And, usually, the egress or the bandwidth plays a may is a is a major component of it. This introduces limitations around how you operate with the object store. Especially, you develop a dependency on the platform. As your data grows, your data mobility starts getting restricted. Naturally, growing data sizes and frequent sharing of data starts adding pressure on your cost models, especially if your costs rely on how you're serving this data. Serving the data to a distributed architecture, especially if you want to leverage unique features across platforms, start getting expensive. What this essentially does is developers start spending too much time architecting around this egress cost. And at Cloudflare, being on a mission to build a better Internet, this problem motivated us to build r two. And we introduced r two as a method for saving large amounts of data, especially unstructured data in the cloud with s three API compatibility backed by our programmable global network, and most importantly, zero egress fees. Now the our two product is built for scale. You are able to store, each bucket is able to hold unlimited data. A single object can be of up to five terabytes in size, and you can in in a single account, you can create up to a million buckets. The network reach is, is basically influenced by Cloudflare's own global network. We have 330 plus data centers globally that allow distribution and low latency access to objects stored on r two. The number of those three thirty data centers keeps on increasing, by the way. And so that's a tough metric for us to track internally as well as we keep growing. I'd like to call out some key features and capabilities of r two, but especially bring your attention to, the 11 nines of design durability. What this level of durability means is that if 1,000,000 objects are stored, you can expect to lose one object every hundred thousand years, matching the durability standards of major providers. And this is something that that that we communicate to ensure that you're you're confident in the data that you're going to be storing on r two. I'd like to also explain how r two works. And one second. Yep. So I'd like to explain how r two works. So r two exposes three three ways in which you can operate with objects that are that you're storing on r two. One, via the s three API. Secondly, the CloudFare REST API, or the workers binding. Especially if you're building on the programmable global network that we talk about where workers are the core component, worker bindings allow you to natively connect with your r two objects. With either of these methods, when a client is fetching an object, that request is first routed to r two gateway that is nearest to the client. And the gateway then connects with the metadata service to fetch the object metadata. And the fetch to actual object relies on the tiered read cache. Checks if there is a nearest cache where the object already resides. If not, that request is request goes upwards towards a distributed storage infrastructure where that object is fetched from. Once this object is fetched, it stays in the tiered cache if this object gets requested within that region often. So you're able to leverage that native tiered cache every time you are using our tool to store your store your objects. I'd I'd next like to quickly go to the s three compatible API because I've called this out multiple times right now. And like I'd like to emphasize on what this means to developers. Number one, it's familiar tooling. So if you're familiar with the AWS SDK or AWS CLI, both o three, you are still able to use r two with with these SDKs. Your this basically means that you're also able able to work with this with minimal code changes to your applications. Many a times, the only change that you'd require would be only environment variables, and we'll look at this in action during our demo. When we talk about s three API compatibility, this is what it looks like on a high level. You are able to do object reads, object writes. You are able to do multipart uploads, basically creating those pre signed URLs. And you are able to do the bucket operations, like listing of the buckets, using the v two API or the or the legacy API. Some advanced operations, like object tagging, are not supported. And 99% of the use cases, you're going to use operations that are on the, on the top. If you would like to look at the details of which APIs are supported, you can visit the link below and get an idea on exactly what this is. There are a few AWS specific headers that are not supported by the s three API. But, again, those do not those do not get used in 99% of the cases that we have run into. Okay. So looking at the pricing metrics, how is r two priced? The the the pricing metrics go by storage. You're priced by the per GB stored against storage classes. So you have storage classes which are either standard or infrequent access. There's no no egress fees, so you pay $0 for egress. And in terms of operation tiers, you have class a operations. So every time you are writing to the object store, you're carrying out class a operations, which are priced per million. There's class b operations, which are fetching these objects, again, price per million. And then there's free operations, naturally, the delete operations. Again, if you'd like to take a look at the pricing, do visit the pricing page to understand details of what this really looks like. We believe that we have a competitive pricing available out there. In fact, if you'd like to have some fun, we have a r two pricing calculator that would help you understand the cost you will be able to save, especially against your egress compared to other cloud providers. So do give that a try. Now I have been speaking for quite some time, so that means it's time for a question. What are the storage classes supported by r two? And we covered this just two slides ago, so this is a very quick memory test. Let's see how many of you have been following me. The options are hot, cold, famous, infamous, or standard, and infrequent access. I'll give you ten seconds for you to vote on the right answers. K. I see the words coming in. Okay. Excellent. So majority of you have guessed this. Right? Or rather not guessed, remembered this. Right? Standard and infrequent access. That's those are the storage classes that are supported by r two. Okay. So let's get closer to our topic for today, which is migration from s three to r two. How can Cloudflare help you migrate from your existing s three compatible storage or s three into r two? There's two tools that we make available for you. And we have some affectionate names for both of them. The first one, SuperSloper, which allows you to quickly migrate large amounts of data from other cloud providers into r two. It's sort of a one go migration that you can carry out. And the second one is Sippy, which is which allows you to do incremental data migration by populating your r two bucket as objects are requested from your r two bucket. And you will get to see exactly how this works. The way we'll run this is, we will be doing a live migration of, photo library app. And the app is simple. It's basically, an app that allows users to upload pictures, one album that they they can create that other users around the Internet can access. The architecture is really simple as well. You basically have, we're we're using Astro with server side rendering that is using the s three client to talk to the object storage, create pre signed URLs, and render those URLs into a into the progressive web app, which is loaded on a web browser. And the client then connects with the r two store connects with the object storage, reads the read the photos or uploads the photos using the presigned URLs. That's how that's how it works. What we will be doing is replacing the object store, the s three object store with r two in this exercise. And our first candidate for the demo would be super sloper. So once again, super sloper allows you to do a one go migration. You migrate all the files that are stored in your s three source bucket into r two in one go, and you start serving them via r two. That's that's what we'll be looking at. Once again, refresher on the checklist. If you want to work with this work with me on this application, can you go to the GitHub URL. You can also use the GitHub code spaces if you don't want to install more and all of that, which is what I'll also be using. So you can play around with the with the app and change the values as I as I make progress over here. Okay. So as a step one, we will be going to the Cloudflare dashboard where I am already. Hopefully, you are able to see my screen. Oh, yes. You are. Okay. So we are in the Cloudflare dashboard. We are going to navigate to storage and databases, object storage, and overview. This is where I'm able to see a list of r two objects. Now if you are following me, you will need to have the r two subscription enabled on your account if you're doing this on the Cloudflare dashboard already. And what we are going to do is we are going to create a bucket. So let's create a bucket and call as super slurper I feel and let's add a few random characters because this name couldn't be used before. And that's the bucket name that I choose. I'm able to also select a location. Now one of the key features of r two is that you do not have to worry about the location. The top dashboard right now automatically identifies that I'm accessing this from Singapore, so I'm somewhere in Asia Pacific, and this and would automatically create this bucket within my region. But if I want to provide a hint of a region, I'm able to provide that using the drop down below. I'm gonna keep it as automatic right now. If I want to follow jurisdictions, we support US and Europe as jurisdictions where these auto buckets can be hosted as well. Let's proceed with automatic and quickly create this bucket. Okay. So there we there we are. Our bucket is created. So once we have the bucket created, we can go back. So these are the steps that I've conducted. So if you wanna see this recording again, you are able to use this as a reference. Next, I'm gonna create an API token for the bucket to access the bucket that we have created. And this is the token that our application would normally use. So let's go to the dashboard and keep I go back. I go to manage. And over here, I can either create a user token or an account API token. I generally choose an account API token if my application's using it. I select the object read and write permissions. I will specify the bucket that's going to use this, which is going to be demo super slurper. And I'll select the TTL. How long do I want this token to be there? I can select forever. I'll select year or so on if I am recommending that hygiene and create API token. Now I'm not going to create this API token live because I've already done that and the tokens flash on the screen. I don't want you to look at these secrets and start using my r two bucket. So I'm just gonna keep it at that. So that's basically how you're able to create those tokens. I have the tokens saved for myself, and we'll we'll come back to that. Now very quickly, let me take you to the code repository that we are working. So this is the code repository. If some of you have followed the link to the GitHub URL, this is the code repository. And I have this open in my GitHub code spaces. And what you can see over here is a environment sample. So these are the this is the this is the sample environment file that I'd be using. Allow me to show you the code. What we are doing over here is we are using the s three client. Now on the chat, do give me feedback if the text is too small for you to see, and I'll try to zoom it in as much as possible. I believe this should be decent enough for most of you to understand the text that's written. Oh, it's too small. Let's zoom. Hopefully, this is good enough. Is this good enough? Okay. This is this is fine. So let's let's work with it so that I'm also able to actually manage this window. So what you can see is we are using the s three client over here, and most of my environment variables that you see over here get used in my s three client. That's basically what this app does. Let us quickly see this app in action. So NPM and dev. I've already done an NPM install. So I'm good over there. I should be able to load this app. Okay. Excellent. So I have my photo share app running over here. And what you can see below in the footer is the name of the bucket that I'm using. And I'm using the bucket one of my buckets on my AWS account. This is where the app's running. And let me also show you what's going on on this bucket a little bit. So I have what I can do is I can create albums. Let me do demo again. Live workshop. Create an album. This create this created an album for me on this very bucket, and I'm able to upload photos. So I have a few stock pictures downloaded. So let me upload one such stock picture from Unsplash. And, yeah, there there we are. So this this is basically how this app works, and this is all being uploaded to s three bucket on AWS. What I have done is to make sure that the we have some some decent sized data to look at Considering and based in Singapore, I have uploaded a few few pictures from Singapore. So there's a album called Singapore that has a few pictures sitting over here already. And I have one more album, I think, which is on my hometown, Pune, and we are able to see some pictures being served. Now, again, reminder, all of these pictures are being served from the AWS s three bucket. Okay? So we have this this app running. We have the tokens r two bucket created. We have a token created for the bucket. How do we move ahead with it? So step three, we start with data migration. Let's start moving the data that we have in the s three bucket into r two. And to do that, we will go back to the dashboard and go to data migration. And under data migration, I will click on migrate files and and select the provider as Amazon s three. I'll add the bucket name. So allow me to get the bucket name. I wanna make sure that I'm not revealing any of the secrets to any of these buckets so that I'm I'm not exposing myself to some funny business. Okay. I have the bucket name. I'm going to add the access key ID that I've generated on the AWS console, and then I'm gonna populate the access key. I need to be very careful with this one. Yep. So once I've done this, I have the ability to also select buckets of parts. I can define the rules of the for the data to be migrated by SuperSlopo. I can select the prefixes. I can select files that are stored in particular folders if you if you want. These are just prefixes after all. But if you wanna call them folders, we can do that. Select prefixes that super slow bow will focus on and ignore everything else. If you have specific files, specific object keys that you'd like to migrate, you can add those as well even in a form of CSV. Right now, we are just going to do full blown migration. So let's just not add anything over there. Click next. At this point, Superslope is going to validate if my three credentials are indeed correct. The fact that we have moved ahead means that they are correct. And now I'm I'm able to configure the r two bucket that I I can migrate to. So let's go to demo super sloper s D230. That's our bucket, r two bucket, where we want to move things. And we are now going to populate the access key for my r two. As I've told you before, I have already created my keys, so I'm just going to copy them over from another screen over here. Okay. And now I'd like to bring your attention to a very important just one second. Okay. Okay. Now I'd like to bring your attention to a very important configuration, which I'd like you to remember because we will talk about this a bit later. And I have to navigate into this section to really communicate. So I really want you to remember this, override files. Then we'll select, yes, override recommended because well, this is what this basically does is if there is a file already existing in r two matching the object key on s three, this is going to allow me control on whether to skip or override that file. I'm gonna keep this as yes, override, but keep this configuration in mind when we are talking about data migration strategies. So you're gonna click next. And, again, this will validate my r two credentials, make sure that SuperSlope is able to connect with r two, and allows me to review the bucket names. And once again, let's validate this is indeed the bucket name that we are using over here on the web app. So let's come back and see migrate files. So while this is migrating, it takes us to this page where we are able to track the migration. You are able to do up to three concurrent migrations using SuperSlurper. So what we have done right now, just as a refresher, we have configured the s three source, and then we configured the r two destination using the access key ID and access key secret for r two. And we ended up on a page that looks something like this. And as you see, SuperSlopo has found out 140 files sitting on the s three bucket, and it has already migrated all of them. So that was under a minute. It SuperSloper managed to migrate all these files, which is excellent. But how do we see this in action now? What we do next is we will configure the application to use the r two credentials. How do we do that? If you notice in my code base, there are two files sitting over here, the s three file and the r two file. So I've already copied in the keys that I've been using to access these buckets. Especially for the r two bucket, I have keys that can access all my r two buckets. I'm gonna use this for the demo. You can use keys that are focused on your buckets. I'm just making sure that during this demo, I'm not juggling through a lot of keys. What I'm gonna do very quickly is in the command here, I'm going to copy over contents of r two dot local into the dot e n v file. That assures me that my r two configuration is set over there. I am also going to copy. I'm gonna go to this bucket. Oh, it's a good idea to see all the files present over here. Right? So we have the bucket. We already see the files have moved in there. I'm gonna copy the bucket name, and I'm gonna populate that into the environment variables as well. Just give me one moment while I do that. K. Okay. So I feel I feel confident with the switch of environment variables that I've done over here. And by the way, this is a Git repo. So if I change any other file that is not be that is being tracked apart from the environment variables, you'd know. So that's another assurance that I'm not making any code changes apart from the environment variables. And let's NPM run. We should have the uploading here shortly. And if you see, we are now using the r two bucket that we created for for running migration via SuperSlopa. And you're also able to see the s three endpoint over here. And this is important because when you're using AWS r two, you don't really need to worry about the s three endpoint. When you're using any other s three compatible storage provider, as you're when you're using Cloudflare, you'll want to copy the endpoint URL that's visible on your Cloudflare r two dashboard, which is basically your account ID r2.cloudflarestorage.com as your s three endpoint. So if your code does not use s three endpoint, that's one change that you might have to make to your code. My code base over here, of course, already uses the endpoint while constructing the s three client. And now you're able to see that all the images for Pune loaded. Let's validate this with Singapore as well. There you go. And let's do that with there's one more album that I have for Bangkok, and you're able to see all of them. So all of all of the data moved. One single big bang migration with super sloper, easy peasy, no code changes, switch environment variables, and you're paying zero URS costs on all of these album loads at this point. Okay. So let's go back to the slide deck. Let me go back a little. If you notice, I I ran one of these migrations couple weeks ago, which had about 23,000 files. Again, something something that I'd like to point your attention to. Took between two to three minutes to run this whole migration from a speed point of view. Super slow pull can be pretty quick. Of course, there are a few architectural considerations that you must keep in mind. An important one, evaluate e tag dependency. Now r two uses s three compatible e tags, meaning the hashing for the e tags is calculated in a similar way in the same way as s three does. But when you run SuperSloper, SuperSloper will upload a few large files using multipart uploads. And if those parts are different to how your original file on s three was uploaded, the etags are going to be different. So especially if you are storing those etags outside of your s three storage to compare whether a file has changed and so on, you might want to consider this during your migration strategy because this is this is one area where things might break. Secondly, archives are not included. So objects stored in archival storage classes on s three will be skipped by a super sloper. Objects greater than one terabyte in size will be skipped by super sloper. And it is recommended that you you use the three concurrent migrations, up to three concurrent migrations that are allowed if you are uploading large buckets. If you're uploading millions of files, make sure that you are splitting this, especially using prefixes while running super sloper so that so that your migration is much more manageable. Okay. Hopefully, super sloper parts clear. Let's move on to see SIPI in action. Now SIPI is basically, a migration methodology where you will be pointing your application directly to your fresh r two bucket. And what CP will do is every time it sends it sees that an object on r two does not exist, an object that is being requested does not exist on r two, it will make a request to s three and pull that object into r What happens over here is, if you're using this for an application in production, each user request that that's coming in for your objects is basically running the migration for you. So the egress cost during migration that you're facing is basically a function of delivering that object that same object back to the user. So, essentially, you could do a migration at at no cost, but that's that's a simplified way to look at it. Let's take a look at CP in action so that you understand this better. Once again, we go to the dashboard, and we create a fresh bucket. And by now, some of you might have guessed what I'm gonna name this. So I'm gonna name this demo sippy, and let's add a few random characters over here so that we are clashing. And, yep, there's a bucket created. I love how quickly these buckets get created. Once we have a bucket ready, let's see. We're gonna configure on demand migration on this bucket. So we are basically going to go to settings. We're going to go to on demand migration, and we are going to go and enable on demand migration. Now, as I populate the keys over here, they're going to be visible. So I'm going to actually, do this outside of the screen and show you the screen once I've configured CP on this bucket. Give me a moment to configure CP. At the same time, I will make sure you are able to see what these configurations are. To configure on demand migration, what we will be asked is what is the provider. You can select a provider amongst this. It can be Amazon Web Services or Google Cloud Storage or any SD compatible storage. Of course, when you select SD compatible storage, it will ask you for the endpoint URL because you're not using the default Amazon. Right now, we are. So we're gonna select this, and I'm just gonna fill in these details on on my secondary screen. Popping in my s three key. Click enable. Okay. So you're able going to be able to see the action happening here already. Okay. And refresh. Okay. On demand migration has been set. So you're able to see the bucket name from my my s three bucket name over here on my dashboard right now. So this means that this bucket is now configured for SIPI. Take note, this is an empty bucket right now, and we'll see some magic happening shortly. So now we go and configure the application to use r two. And you all know how that's done. Right now, it's just changing of environment variables, but I want you to keep an eye on the bucket name so that you know what my environment variables are finally going to change to. So once again, I'm gonna step away over here and update my environment variables to use the bucket name that we just created. Copy this. And bucket name changed to SIPI. K. And we go back to the code base, and we rerun this application. Okay. And we reload this home page. The request came in. Perfect. And you're now able to see that it is indeed the bucket name that we have just created, and we will now see some of our existing albums. Let's take a look at Oh, first, I want to, again, emphasize empty bucket over here. And let's go with loading the album, Singapore. There you are. Served from this bucket where we have CP configured. And if I refresh this now, you are able to see all the files, all the all the photos for the Singapore album coming in from from s three. Here on, any new request that comes to r two, every time I load this bucket again this album again, all of these images are now being served by r two. No requests for these images going to s three anymore. Once again, let me show you albums slash Pune. Nothing here. Let's load the Pune album here. There's the Pony album loaded, and then let's make a change here first so that it loads. Yeah. And then you see all images from Pony loaded again. I could just watch this in action. So I might just download Bangkok as well, and let's watch all the files for all the images of Bangkok loading as well. Okay. So that's basically how CP works. You're you're you're letting the usage patterns on your application migrate files from s three. And this is especially this especially works if you want to migrate the auto files first. I see a question over there. Will you auto delete s three file after migration? The answer is no. Either of these migration tools do not delete anything on your s three bucket. It's just good hygiene again, allowing you to validate once the migration's done, and then that's going to be that's working for you and you're in control of deleting files on s three. You can now navigate to because you we have enabled CP over here, we can go to metrics. And there is a unique metric shown for every bucket that has CP enabled, which is served by CP, which tells you how many operation were served by r two and how many operations were served by CP served via s three, ultimately. The higher the number, the more confident you can get that these objects are starting to get served on r two, and you're not sending any requests to s three. But these these metrics can basically give you a good idea of where you are in your migration via SAP. So if you're losing a number like 73, where you're seeing, like, r two is serving 75 of the operations and five of the operations coming in from source bucket, you start getting confident that there are fewer files left that are actually being used that need to get migrated still. Again, some important architectural considerations. If you're doing this with Sippy, validate your need patterns. Sippy pulls objects or metadata from s three during direct object trees via get object or head object calls. This means that the list object v two or list objects call is not supported by CP. So if your application does a list object call to under to discover objects to request to, then that's not going to work with CP because your r two bucket is going to return zero objects during that discovery call. You might have a question that, hey, Unkar, the photo library that we are using, does that not do the discovery call? The answer is no. The library app that I've built uses the metadata file to list down the objects, and I'm able to rely on that. However, if you're using list object call, doesn't mean that you can't use CP. It is just that you'll have to go through a code change where you're making the list object call and make sure that call still goes to your s three bucket and not the r two bucket. And that that simple fix can make sure that your migration still runs. Again, just like super sloper, you'll have to make sure that you evaluate e tag dependency because CP as well, if the files are larger than 199 MB, they are going to get uploaded through. They're going to get uploaded in parts, and that's something that, can change your eTax. For large objects, it's important to keep in mind that they are going to get uploaded in parts. So there's gonna be multiple get requests to your s three bucket. So if you have done some very strict accounting around how many requests are going to come into your s three bucket, you might get thrown off a little because CPE and SuperSlipper both are very autonomous in terms of figuring out how they're going to upload those objects and how the parts are going to work. You also need to be careful, especially if there are very, very frequent if if there's a lot of reads happening on your r two bucket while c CP is enabled for the same object that's not present right now, and especially if that object is large, does not get fully copied quickly, there could be multiple requests that go to your s three to get this object. Basically, duplicate requests going to s three to get that object into r two. Once that object is present in r two completely, those requests will stop. But it's important to know that there might be some multiple requests going in just so that there are no surprises in terms of costs. Okay? Now looking at the migration strategy. I've shown two tools, SuperSloper and SIPI. It is not a either or situation. In fact, we recommend that depending on your usage pattern, you can use a combination of these tools while running your migration. Active reads, if you are if you if your source if your object store basically has active reads and writes, then we recommend that you enable CP first so that your copy on on demand starts and you start uploading new files to r two. For example, the app right now is configured with Sippy, and any new album that I create is directly getting created on r two. It's not going to get created on s three at all. So you enable Sippy in such scenarios and then run SuperSloper to trigger a one off migration to copy remaining objects, disabling the overwrite files. If you remember, there was a setting that I brought your brought to your attention while we were configuring SuperSloper, which was a flag to prevent overwriting of files. It's important that you do not overwrite those files and and run SuperSlopo so that anything that has already been migrated by CP does not get migrated, and you're getting that that delta in via SuperSlooper. If your source isn't actively read or return to, you can run SuperSlooper to run a one off migration. Now it's important to know that SuperSlooper will skip objects that are uploaded after its list calls. It'll make a list call and then may start making targeted migration request to all of those objects. So if if it's not be if your source is not being actively returned to, there could be fewer objects that are missed by super sloper. And in the end, you can enable CP to just make sure that any any of those files that may have been missed will get picked up by our by CP when they're requested on your alto bucket. So that's that's a very simplified strategy that you can use. Now what I'd like to do is if your file sizes are more than one terabytes, it would be it's more super sloper and instead of super sloper and CP, you might want to have more control on your migration, especially if a single object size is more than one terabyte. You can use something like rclone. And what I what what I'll do is very quickly show you how rClone works. If some of you are DevOps engineers already using rClone or even data engineers moving data around using rClone, it's a very popular tool. For those not aware, it's a very popular tool that can allow you to connect with a wide range of data services. So what we are going to do is I have r two already configured. Again, I don't want to jump around with keys. So I've already configured r two. So I can show you r two config. And you can see that I have my r two and s three connect remotes already set up. But I'll show you what a setting up a new remote looks like. So I go for new and say let's say r two secondary. When I say I'm just giving it a name. I can select which type of storage this is, and then I can select number four, which is the s three Amazon s three compliant storage. So which is number four. And I can select a provider over here. Cloudflare is number seven, I think. Yep. Then it is number seven. So go to number seven. And then I can enter the credentials. So I can either say that no. I'm gonna enter I'm gonna enter the credentials, and then it'll ask me for access key ID and so on. But, yeah, I'm gonna go through this. I have already set this up. I can see the secondary partially configured over here. Let me quickly delete that number two, and let me show you our clone. You're able to see again the type for both of those remotes is s three. Again, showing how that s three compatibility can be super useful. And we are going to do three on s three, and let me get the bucket name. Oops. Sorry. I have not quit config. I keep doing I keep repeating this mistake sometimes when I forget to quit config. Yeah. So you're able to see the tree over here. And what I can do is I can also do the same thing with my r two remote and query a bucket the CP bucket that we just configured. So let's take a look at the CP bucket where we are able to see the tree structure. And if you're if you're running migration now so let us quickly create bucket because it doesn't take much time to create a bucket. Does it to demo, upload, again, a few random characters. And what I'm able to do now is take this. Let me get the s three bucket name first, and then we get our our tool bucket name. And let's make sure that we're able to see the progress here. And there you go. You're able to see the transfers going on between our two. Now our two gives you a lot of sophisticated controls in terms of parallelization for uploading files, the chunk sizes, and so on. I'm not doing a lot of concurrent uploads intentionally because that tends to glitch my webcast. So I'm gonna just keep it at that, and you'll be able to see some of those files coming in here already. So that's basically that's basically how compatible this is with popular tools like our clone as well if you're already using this. Now question time. I've been, again, speaking for a lot of time, and we've seen a lot of stuff. So here's an interesting question. What is the recommended migration strategy? If your source is actively actively, and I repeat, actively being read and written to with file sizes less than one terabyte. What would you do? Would you run super sloper then enable CP on the bucket or enable CP first then run super sloper while disabling or writing off existing files? I'm going to give you all thirty seconds to process this a little bit. I want most of you to get this answer. Right? Hopefully, the poll's open for you to already answer on your screen as well. If you're not able to see the poll, from that feedback in the chat. Okay. There's the poll's up. I'll give you twenty more seconds to think about it and answer this. Taking a look at the chat in the meantime. Oh, this is what's the full form of r two? I think I think there are there are many many versions of it. I think rapid and reliable is the most popular is what I've what I've heard. K. It's excellent. So Lalitindu is already familiar with our Akron has been using that with OneDrive, Google Drive, and so on. So it's good to know that one of your friend friendly CLI tools is capable of connecting with Akron. Okay. Time to look at the answers. Okay. Interesting one. So what was the question? What is recommend what is the recommended migration strategy if your source is actively being read and written to with file sizes less than one terabyte? The option was run super sloper and then enable CP on the bucket. Now the first option is wrong. What we recommend is the second one. Primarily, because when you run super sloper, one poor source that's actually being actively being written to, SuperSlopo is gonna miss out on a lot because it's gonna make a list call and then miss out on a lot of objects over there. What we recommend is you enable CP first so that what when as as these read requests come into your r two, it already starts populating your r two bucket. And you run super slow for when disabling the overwrite overwriting for existing files. Majority of you got this right, so congratulations for that. I think about 75% of you got this got this right. So I'm really happy. I think I'm I'm sending across the message clearly. Makes me feel confident. Okay. So we will now head to one of the final slides that I have is the r two, object store, has been has all also been been extended for data platform use cases. Sometime last year, we introduced the data data cataloging features with Apache Iceberg data catalog, but directly into our two buckets so that you can connect your Spark or Snowflake or PyIceberg engines query engines directly to the data catalog. So what you can basically do is and let me show you how easy that control is as well. So you can go to the settings, and you can go to our two data catalog, and you can enable this over here. And when you enable this, you get the endpoints. Just give it a moment. Okay. So what this does is hopefully, the keys aren't shown over here. What this does is creates the catalog URI for you that you can use in your existing tooling already and start using the Apache Iceberg tables. What we've also introduced is pipelines, which allow you to ingest events, either via HTTP triggers, run transform them with SQL queries, and finally deliver them as either iceberg tables or just Parquet or JSON files on your r two bucket stores. Even if it's not a data catalog, you can still use pipelines to just store those files. And you can also run SQL queries against your Apache Iceberg tables using distributed analytics using r two SQL. We we have been doing this because we have been seeing a lot of our customers looking at r two, especially because of its zero egress cost as a data as a data storage for data platforms where you are you don't have to worry about querying your own data, how many ever times you want to, while running all kinds of sophisticated analytics. So this just helps us make sure that we're cater catering to these sort of use cases in a much, much better way. What I would like to considering we do have a few minutes still, I would also like to show you something called r two local uploads that we that we launched that we opened recently. This is also local uploads can be enabled again on your settings. And the way it works is one of our engineers built a very interesting demo, and I've been looking at excuses to really bring this up during during my talks. So what local uploads allows you to do is considering your r two bucket is created at a focused on a single region, upload experiences for your users can can seem a little off, especially if they're uploading from a region far away. What local uploads allows you to do is, as you can see over here, I've selected a bucket in Western North America sitting over here. Sitting over here in Singapore. What local uploads allows you to do is what this this demo page will emulate is start uploading files using local uploads and without local uploads. And you wanna upload about five megabyte files multiple times. This is the the number of requests you're multiple so that we benchmark against p 50 metrics. And you'll be able to see that with local uploads, the five megabyte file basically gets uploaded to the nearest edge and then gets uploaded to the primary storage, which is all handled by Cloudflare. So anytime an upload happens, it happens in one of your local one of your local regions, and the file then gets populated to the primary storages. Just making sure that the users of the hook when uploading. If this file is requested for from the same region, it gets served from the same storage. So makes especially for use cases where you are expecting that users will be uploading files and accessing them within the same region, local uploads can allow you to give that upload experience an excellent turnaround time. So, yep, give this a give this a try. There are a few more links, especially the migration strategy. If you'd still like to absorb that, figure out what's going to be beneficial for your use case, do visit this link. There's the r two pricing calculator. There is also r two explorer app that allows you to look at your r two storage just like a file directory. So do give it a give it a look as well. That's basically all of it. Have cleared. We can now open for questions. So if there's any questions, let's would love to look at them on the chat. Let me scroll through. Looks like a lot of these questions are already answered by Timo. Timo has been busy over here with answers. There is one question that that has been asked. Karl, can you show what happens when an object fails fails to copy? And I think this was asked while I was either demoing SuperSloper or CP. That will show up on your dashboard. So let me go back to the dashboard. And so especially if you're using SuperSloper, you will be able to see that that failure amongst the files over here. If something has been skipped or if there is an if there is an error, you'll get the details of that. You can pick up the object keys and retry them if needed. I haven't run into errors amongst the many migrations you can see that I've been prepping for this demo as well. Not running to errors, but you'll you'll get that control. And the fact that SuperSlopper allows you to mention keys just gives you that control. If something specific is missing, you can try that again. Will you delete s three files after migration? I think I've answered that already. We won't be deleting anything. K. So I think someone's requested to look at the log. So let's take a look at the log as well. Oh, that file's opened somewhere else. I had to look in the window. Give me a moment. Just give me one moment while I while I I look in the file. K. Yeah. Got it. Okay. This is what the log looks like, which gives you the status of the it basically gives you the status of the migration job. So all of these super sloper is available via the rest API and via the dashboard. You are able to operate on CP via Wrangler as well along with rest API and the dashboard. You are able to fetch these logs using the rest API as well if that's something that's interesting for you. So let me go back. Let's see. Are there any questions? I think that's about it. I see most of the questions asked. If there aren't many questions, then I think we can close off for today. Thank you for being a wonderful and interactive audience. Have a great evening, Ad. Thank you.