[OmniOS-discuss] Resilvering and errors in RAIDZ1 "DEGRADED"

[steve at linuxsuite.org]

Howdy!

            I have a raidz1 pool that had a disk failure, I OFFLINE the
failed disk and
then did a REPLACE. Resilvering started.  I checked back a couple of hours
later
to see how the resilvering was coming along. I noticed that another device
in the pool is in a "REMOVED" state. I assume it failed, as I didn't
remove it.

          So a couple of questions.

              i) Is the recovery of this pool destined for failure because
it is a raidz1 and
                  there are 2 "missing" disks and the resilver hasn't
completed?

             ii) Why isn't this pool in a FAILED or UNAVAILABLE state?

            iii) I can still read the data from the disk, as far as I can
tell???? Is it
                 possible to recover the data, or will I find some data
not there??

           iv) Are the errors in the files  the result of checksum error
but the data
                 is not recoverable...??

           I got a copy :-)

         thanx - steve

root at omnios-test:/dfs1_data52/test# zpool status -v dfs1_data52
  pool: dfs1_data52
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
        continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Tue Aug  9 11:38:20 2016
    4.75T scanned out of 17.0T at 337M/s, 10h33m to go
    387M resilvered, 27.98% done
config:

        NAME                         STATE     READ WRITE CKSUM
        dfs1_data52                  DEGRADED     0     0     8
          raidz1-0                   DEGRADED     0     0    16
            c2t50000C0F01C493DAd0    ONLINE       0     0     0
            c2t50014EE55558B72Ed0    ONLINE       0     0     0
            replacing-2              DEGRADED     0     0     0
              17843744530960610892   OFFLINE      0     0     0  was
/dev/dsk/c7t50000C0F04557AD2d0s0
              c2t50000C0F011E16CEd0  ONLINE       0     0     0 
(resilvering)
            c2t50014EE55558A90Ed0    REMOVED      0     0     0
            c2t5000CCA03B67EE39d0    ONLINE       0     0     0

errors: Permanent errors have been detected in the following files:

        /dfs1_data52/91/59/1501715991.bkt
        /dfs1_data52/92/59/1501715992.bkt