Comment by magicalhippo

Unless I misunderstood you, you're describing more how classical RAID would work. The RAID-Z expansion works like you note you would logically expect. You added a drive with four blocks of free space, and you end up with four blocks more of free space afterwards.

You can see this in the presentation[1] slides[2].

The reason this is sub-optimal post-expansion is because, in your example, the old maximal stripe width is lower than the post-expansion maximal stripe width.

Your example is a bit unfortunate in terms of allocated blocks vs layout, but if we tweak it slightly, then

    Da1 Db1 Dc1 Pa1 Pb1
    Da2 Db2 Dc2 Pa2 Pb2
    Da3 Db3 Pa3 Pb3 ___

would after RAID-Z expansion would become

    Da1 Db1 Dc1 Pa1 Pb1 Da2
    Db2 Dc2 Pa2 Pb2 Da3 Db3 
    Pa3 Pb3 ___ ___ ___ ___

Ie you added a disk with 3 new blocks, and so total free space after is 1+3 = 4 blocks.

However if the same data was written in the post-expanded vdev configuration, it would have become

    Da1 Db1 Dc1 Dd1 Pa1 Pb1
    Da2 Db2 Dc2 Dd2 Pa2 Pb2
    ___ ___ ___ ___ ___ ___

Ie, you'd have 6 free blocks not just 4 blocks.

Of course this doesn't count for writes which end up taking less than the maximal stripe width.

[1]: https://www.youtube.com/watch?v=tqyNHyq0LYM

[2]: https://openzfs.org/w/images/5/5e/RAIDZ_Expansion_2023.pdf