[Discuss] On Btrfs raid and odd-count disks

[Richard Pieri]

In retrospect, if you're looking at file systems as a means to prevent
write holes with RAID 5/6 then you're going about it wrong. Write holes
happen with every RAID level. They happen with RAID 5 and 6. They happen
with RAID 1 and RAID 10. Do not believe anyone who says that write holes
are unique to RAID 5/6 and their derivatives. They are mistaken. Any two
or more storage devices in a RAID set that are not atomically locked
together can suffer write holes. They can even happen with ZFS.

This is not a RAID issue. RAID is about making the hardware tolerant to
faults. RAID does not care about the integrity of your data.

Write holes happen when power to the storage devices is lost during
write operations. UPS and redundant power are the primary ways of
preventing write holes. If the server doesn't lose power, or it has time
to perform a graceful shutdown when mains fail, then no holes appear in
the data it holds.

Battery-backed cache is the second line of defense against write holes.
The battery prevents cache loss if redundant and backup power fail.
Non-volatile cache (SSD) is becoming a popular alternative to
battery-backed cache, although flash has its own set of power-related
problems.

The last line of defense against corruption is a good backup history.

ZFS and Btrfs will detect and if possible correct single-bit errors.
They may be able to prevent write holes if they can reliably control
every piece of I/O cache in the data stream. This includes the write
acceleration cache found on most modern disks' on-board controllers. Not
all of these reliably honor cache flush instructions from the host and
because of this they cannot be relied upon to maintain data integrity
under power fault conditions.

-- 
Rich P.