Object Storage as a Backup Target • Re: S3-integrated Storage - Block Size

One thing to note is MinIO used NVMe drives in their Veeam Ready testing: https://www.veeam.com/sys1047

Have you contacted MinIO to see if they have any recommendations on what performance metrics you can expect with an hdd setup?

Thanks

Steve

We understand the performance metrics we can expect with our Minio deployment. Hence the desire for larger blocks, specifically on backup copy jobs or SOBR offload jobs. We have over 4 PB of Minio deployed (in a single cluster across multiple pools) and can ingest at multiple Gbps.

I feel we are caught between a rock and a hard place. NVMe is cost prohibitive (although how amazing would that be!). Veeam uses small objects by default. A happy medium would be for Veeam to allow us to change the Block Size on a Backup Copy Job (or a SOBR Offload) and allow us to set an object size that our Object Storage (in this case Minio) is happier with, fully understanding that there is tradeoffs with backup sizes as a result. We don't always want to change block sizes on the initial backup job, because it may have been in place for years, or the customer's storage can't handle new full backups for the size change to take effect. Whatever the case is, its not always easy to put in place. Nor do we want to use a backup job for direct to S3 because it adds additional load to the production infrastructure. Also, despite my own personal desires, we don't only ingest backups from one backup product (it gets incredibly interesting when you dig into how different vendors deal with backups - from one vendor making one object per block, to another vendor making one object per drive that gets backed up).

It will make a huge difference for those of us stuck between a rock and a hard place.

Also, our issue is slightly different than @emachabert's - we don't have an issue with retention metadata, since that's not handled in any database for Minio.

Statistics: Posted by tyler.jurgens — Sep 03, 2024 4:17 pm

---