我有一台家用服务器，在一个 NVMe SSD (ext4) 上运行 Debian 10 (Proxmox) 和 2 个 ZFS 池。第一个池子是一个8x8TB的Raid Z2阵列，叫做 vault 另一个是RAID 0 2x1TB阵列，称为 workspace.
我最近想把 workspace. 我停止了对文件系统的所有文件操作，卸载了文件系统，然后继续执行 zfs destroy 池上。我把其中一个硬盘从 workspace 和我重启了电脑。
当我重新启动时，我以为只看到了 vault ZFS池，并且它是健康的。然而，当我检查时，我看到它现在处于DEGRADED状态。
root@homenas:~# zpool status
pool: vault
state: DEGRADED
status: One or more devices could not be used because the label is missing or
invalid. Sufficient replicas exist for the pool to continue
functioning in a degraded state.
action: Replace the device using 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-4J
scan: scrub in progress since Tue Apr 21 19:03:01 2020
2.35T scanned at 6.50G/s, 336G issued at 930M/s, 6.69T total
0B repaired, 4.91% done, 0 days 01:59:31 to go
config:
NAME STATE READ WRITE CKSUM
vault DEGRADED 0 0 0
raidz2-0 DEGRADED 0 0 0
sda ONLINE 0 0 0
sdb ONLINE 0 0 0
sdc ONLINE 0 0 0
sdd ONLINE 0 0 0
sde ONLINE 0 0 0
sdf ONLINE 0 0 0
15380939428578218220 FAULTED 0 0 0 was /dev/sdi1
8563980037536059323 UNAVAIL 0 0 0 was /dev/sdj1
errors: No known data errors
我相信这些硬盘可能是在 /dev/sdX. 不过我不知道为什么一个出了故障，一个不能用了。该 vault ZFS 池仍然处于活动状态，我已经运行了一个备份，将最近的所有数据池化到另一个存储介质上。
然而，是否可以恢复我的 vault 池，并使其恢复健康？如果这是由于一些驱动器在删除后的切换。workspace 池，那么恢复阵列的最佳选择是什么？
这是我从fdisk得到的信息。
root@homenas:~# fdisk -l
Disk /dev/nvme0n1: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Disk model: WDBRPG5000ANC-WRSN
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/nvme0n1p1 34 2047 2014 1007K BIOS boot
/dev/nvme0n1p2 2048 1050623 1048576 512M EFI System
/dev/nvme0n1p3 1050624 976773134 975722511 465.3G Linux LVM
Disk /dev/sda: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: WDC WD80EMAZ-00W
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sda1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sda9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/sdb: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000DM004-2CX1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sdb1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sdb9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/sdc: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000DM004-2CX1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sdc1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sdc9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/sdd: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000DM004-2CX1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sdd1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sdd9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/sde: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000DM004-2CX1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sde1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sde9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/sdf: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000DM004-2CX1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sdf1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sdf9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/sdg: 931.5 GiB, 1000204886016 bytes, 1953525168 sectors
Disk model: WDC WDBNCE0010P
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sdg1 2048 1953507327 1953505280 931.5G Solaris /usr & Apple ZFS
/dev/sdg9 1953507328 1953523711 16384 8M Solaris reserved 1
Disk /dev/sdh: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: WDC WD80EMAZ-00W
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sdh1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sdh9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/sdi: 7.3 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: ST8000DM004-2CX1
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: XXXX123
Device Start End Sectors Size Type
/dev/sdi1 2048 15628036095 15628034048 7.3T Solaris /usr & Apple ZFS
/dev/sdi9 15628036096 15628052479 16384 8M Solaris reserved 1
Disk /dev/mapper/pve-swap: 8 GiB, 8589934592 bytes, 16777216 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/pve-root: 96 GiB, 103079215104 bytes, 201326592 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk /dev/mapper/pve-vm--100--disk--0: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes
Disklabel type: dos
Disk identifier: 0x97e9a0ea
Device Boot Start End Sectors Size Id Type
/dev/mapper/pve-vm--100--disk--0-part1 * 2048 67106815 67104768 32G 83 Linux
我很感激你能提供的任何信息。
我能够解决这个问题，并使RAID阵列恢复到健康状态，而不会丢失任何数据。我通过使用磁盘标识符来解决这个问题，而不是使用磁盘路径来解决这个问题。udev.
如果还有人在向系统添加或删除一个不相关的磁盘后遇到ZFS池故障，并确定他们也是使用磁盘路径而不是最初的静态标识符创建的池，我将提供以下步骤。
备份您的数据；在继续之前，确保您有一份完整的池数据副本。
停止对文件系统的所有写入（我在这种情况下使用的是Docker）。
docker stop $(docker ps -aq)
umount /vault
zpool export vault
zpool import -d /dev/disk/by-id vault
root@homenas:~# zpool status
pool: vault
state: ONLINE
status: One or more devices has experienced an unrecoverable error. An
attempt was made to correct the error. Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
using 'zpool clear' or replace the device with 'zpool replace'.
see: http://zfsonlinux.org/msg/ZFS-8000-9P
scan: resilvered 73.5M in 0 days 00:00:07 with 0 errors on Wed Apr 22 10:38:47 2020
config:
NAME STATE READ WRITE CKSUM
vault ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
wwn-0xabcdefghijklmnoq ONLINE 0 0 0
wwn-0xabcdefghijklmnoq ONLINE 0 0 0
wwn-0xabcdefghijklmnoq ONLINE 0 0 0
wwn-0xabcdefghijklmnoq ONLINE 0 0 0
wwn-0xabcdefghijklmnoq ONLINE 0 0 0
wwn-0xabcdefghijklmnoq ONLINE 0 0 0
wwn-0xabcdefghijklmnoq ONLINE 0 0 25
wwn-0xabcdefghijklmnoq ONLINE 0 0 16
errors: No known data errors
我们可以看到一些数据在重新导入时被重新银化，但很快就被解决了，只需几秒钟。
clear 来重置ZFS的标志。zpool clear vault
scrub 以确保ZFS池处于良好状态。zpool scrub vault
status 以确保游泳池是健康的。root@homenas:~# zpool status
pool: vault
state: ONLINE
scan: scrub repaired 0B in 0 days 01:59:47 with 0 errors on Wed Apr 22 12:46:58 2020
config:
NAME STATE READ WRITE CKSUM
vault ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
wwn-0x5000cca252de17d4 ONLINE 0 0 0
wwn-0x5000c500c4e46bf9 ONLINE 0 0 0
wwn-0x5000c500c4e65198 ONLINE 0 0 0
wwn-0x5000c500c4e616a4 ONLINE 0 0 0
wwn-0x5000c500c4ac129e ONLINE 0 0 0
wwn-0x5000c500c4e3f74a ONLINE 0 0 0
wwn-0x5000cca257eb9299 ONLINE 0 0 0
wwn-0x5000c500c4e50efc ONLINE 0 0 0
errors: No known data errors