[OmniOS-discuss] Backing up HUGE zfs volumes

[Michael Talbott]

I'm trying to find ways of efficiently archiving up some huge (120TB and growing) zfs volumes with millions maybe billions of files of all sizes. I use zfs send/recv for replication to another box for tier 1/2 recovery. But, I'm trying to find a good open source solution that runs on Omni for archival purposes that doesn't have to crawl the filesystem or rely on any proprietary formats.

I was thinking I could use zfs diff to get a list of changed data, parse that into a usable format, create a tar and par of the data, and an accompanying plain text index file. From there, upload that set of data to a cloud provider. While I could probably script it all out myself to accomplish this, I'm hoping someone knows of an existing solution that can produce somewhat similar results.

Ideas anyone?

Thanks,

Michael