ZFS, Unraid Array, or Hybrid? Choosing the Right Storage Solution for Your Needs

Guest blog by Ed Rawlings, aka Spaceinvader One 👾

Unraid has earned a reputation for its exceptional storage management capabilities and extensive support for popular media server applications like Plex, Emby, Jellyfin, and more. However, with the recent addition of ZFS in Unraid 6.12, it's worth exploring various storage scenarios and their implications. Let's examine three potential storage setups and their impact on performance, redundancy, and scalability to determine the optimal location for our media files.

Traditional ZFS

Pros:

• Traditional ZFS ensures robust data protection and self-healing of files (bitrot protection)
• Advanced features such as data compression and snapshots while utilizing zpools.
• High throughput due to using all the disks in the pool at once—ideal for simultaneous access by multiple users.

Cons:

• ZFS pool expansion can be complicated, requiring much more planning as your collection grows.
• The increased RAM and CPU resources that ZFS demands could impact the performance of other server applications, making it less suitable for resource-limited servers.
• ZFS pools typically necessitate all disks spinning while in use, leading to increased power consumption compared to a regular Unraid array.

No ZFS - Using the Unraid Parity-Protected Array

Pros:

• Unraid's native XFS or BTRFS file systems deliver good read speeds for most media server users.
• The array is readily expandable (an essential consideration for you media data hoarders out there as your collections grow! )
• The regular array offers excellent power efficiency as idle disks can be powered or spun down. So, if a movie is playing from disk 1, all the other disks, if not being used, can be spun down.

Cons:

• This configuration lacks ZFS's inherent bitrot protection and robust redundancy capabilities, such as RaidZ3.
• For heavy-use media servers, throughput might be compromised under heavy loads from multiple simultaneous users, particularly if those users are all streaming movies stored on the same disk in the array.

Hybrid Approach

ZFS-formatted disks within the Unraid array

Pros:

• This strategy combines Unraid's array flexibility, allowing for easy capacity expansion, and ZFS's advanced features, such as data compression and snapshots.
• Idle disks can be powered down to conserve energy when unused.

Cons:

• ZFS-formatted disks in Unraid's array do not offer inherent self-healing bitrot protection.
• The throughput is the same as a standard Unraid array and may not operate as efficiently as a pure ZFS pool under the strain of multiple concurrent users.
• ZFS's heightened resource demands could overburden less potent servers.

Compression

In both the Traditional ZFS (pure Zpool) and Hybrid Approach scenarios above, I have highlighted compression and snapshots as advantages for the ZFS dataset where you store media files. First, let's discuss compression. Although I am ambivalent about compression on already compressed files like h264 and the even more compressed h265 - as the space saved will be less significant - ZFS's default compression (lz4) has minimal overhead, making it worthwhile even on already compressed data.

Side note - I won't bore you going into too much detail, but using the "none" compression setting would waste an average of half the record size for large files to span multiple records. This is because the last block of such files may be partially filled, resulting in unused space. Additionally, using the "none" compression in the zpool/dataset setting turns off the detection of all-zero blocks in the middle of files, which means that even blocks of zeros will be stored in their entirety, further wasting space.

While compression may not yield significant space savings for already compressed data, it can still provide some benefits. Additionally, there might be poorly encoded media files or uncompressed ISO dumps of DVDs that can be compressed to save space.

Snapshots

So now, let's talk about snapshots in a media dataset. Why would we want this, as media typically follows a write-once-read-many-times pattern? Consider a situation where a media re-encoding operation by a container like tdarr produces unsatisfactory results. With a snapshot, you can create a clone. This clone is a separate filesystem independent of the original snapshot and dataset. It allows you to access and extract the needed file before deleting the clone, leaving your original dataset unaffected.

Selecting a storage scenario for your media server should be based on your requirements and priorities. There is no universal solution that fits everyone. It is crucial to meticulously plan and consider not only your current needs but also future requirements. Whether you opt for ZFS or not for your media storage, it is excellent that Unraid now offers us the flexibility to make this choice.

Harnessing the Power of ZFS on Unraid

Suppose you have decided you want to use ZFS on your Unraid server. In that case, two videos have been created for a step-by-step guide through upgrading your Unraid cache pool to either a larger drive or just reformatting the one you have to a ZFS file system - all without losing a single byte of data!

Upgrading Your Cache Pool

Over time, your data needs will grow, and so should your cache pools. You may need a whole new pool besides what you have now. Or you don't have any expansion options (no more NVME slots or SATA ports), so this leaves upgrading the pool you already have. Upgrading your cache pool to a larger drive might seem daunting, but with our straightforward guide, it becomes a breeze. Our video tutorial offers a visual aid, taking you through every step of the process.

Reformatting to ZFS

You may be content with your current storage capacity but want to upgrade your file system to ZFS. ZFS can handle a vast amount of data while protecting against data corruption, allowing for snapshots, compression, and replication. Making the switch is worth considering, and this guide focuses on helping you reformat your existing Unraid disks to ZFS without losing any data or breaking Parity.