DISK交換
昨年の9月にほとんど同じ日記を書いてた。やっててよかったZFS。
10/15に録画サーバが応答しなくなってるのに気づいて、コンソール見てみると断末魔が。
(ada1:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 49 00 40 00 00 00 00 00 00 (ada1:ahcich1:0:0:0): CAM status: ATA Status Error (ada1:ahcich1:0:0:0): ATA status: 61 (DRDY DF ERR), error: 04 (ABRT ) (ada1:ahcich1:0:0:0): RES: 61 04 d1 cc 02 40 21 00 00 00 00 (ada1:ahcich1:0:0:0): Retrying command (ada1:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 49 00 40 00 00 00 00 00 00 (ada1:ahcich1:0:0:0): CAM status: ATA Status Error (ada1:ahcich1:0:0:0): ATA status: 61 (DRDY DF ERR), error: 04 (ABRT ) (ada1:ahcich1:0:0:0): RES: 61 04 d1 cc 02 40 21 00 00 00 00 (ada1:ahcich1:0:0:0): Error 5, Retries exhausted swap_pager: I/O error - pagein failed; blkno 264463,size 4096, error 5 vm_fault: pager read error, pid 98520 (http)
どうにもならないのでリセットして再起動、立ち上がった後のzpoolはこんな感じ。
pool: ztank state: DEGRADED status: One or more devices could not be opened. Sufficient replicas exist for the pool to continue functioning in a degraded state. action: Attach the missing device and online it using 'zpool online'. see: http://illumos.org/msg/ZFS-8000-2Q scan: resilvered 2.64T in 22h39m with 0 errors on Tue Sep 16 11:01:07 2014 config: NAME STATE READ WRITE CKSUM ztank DEGRADED 0 0 0 mirror-0 DEGRADED 0 0 0 7724617931313753760 UNAVAIL 0 0 0 was /dev/ada1p3 ada0p3 ONLINE 0 0 0
これが木曜の夜で、そのままAmazonで同じWD30EZRXを注文、金曜に帰ってきたら届いているという、21世紀ってすごいね。。
DISKを取り替えてシングルユーザで起動したら、何かそのままresilverが走り出した?
pool: ztank state: DEGRADED status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Sat Oct 17 11:48:15 2015 2.35G scanned out of 2.67T at 1/s. (scan is slow, no estimated time) 7.81M resilvered. 0.09% done config: NAME STATE READ WRITE CKSUM ztank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 7724617931313753760 UNAVAIL 0 0 0 was /dev/ada1p3 ada0p3 ONLINE 0 0 0 errors: No known data errors
取り替えるより先にpoolから抜いとくべきだったのかなあ。とりあえず外す。
# zpool detach ztank ada1p3 # zpool status pool: ztank state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Sat Oct 17 12:20:15 2015 29.6G scanned out of 2.67T at 21/s, (scan is slow, no estimated time) 7.81M resilvered, 1.08% done config: NAME STATE READ WRITE CKSUM ztank ONLINE 0 0 0 ada0p3 ONLINE 0 0 0 errors: No known data errors
シングルユーザモードでの、いつもの作業前のお約束。
# mount -u / # zfs mount -o ro ztank/usr # zfs mount ztank/var # zfs mount ztank/var/empty # zfs mount ztank/tmp # /etc/rc.d/adjkerntz start
HDDが同じなので、既存のada0と同じようにgpart。
# gpart show ada0 => 34 5860533101 ada0 GPT (2.7T) 34 6 - free - (3.0k) 40 128 1 freebsd-boot (64k) 168 2097152 2 freebsd-swap (1.0G) 2097320 5858435808 3 freebsd-zfs (2.7T) 5860533128 7 - free - (3.5k) # gpart create -s GPT ada1 ada1 created # gpart add -b 40 -s 128 -t freebsd-boot ada1 ada1p1 added # gpart add -b 168 -s 2097152 -t freebsd-swap ada1 ada1p2 added # gpart add -b 2097320 -t freebsd-zfs ada1 ada1p3 added # gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1 bootcode witten to ada1
このままシングルユーザでattachしてresilver待ちすると、今日の録画予約が間に合わん。。ということで、
リブート通常起動してから、続き。
# zpool status pool: ztank state: ONLINE scan: resilvered 7.81M in 9h17m with 0 errors on Sat Oct 17 21:06:00 2015 config: NAME STATE READ WRITE CKSUM ztank ONLINE 0 0 0 ada0p3 ONLINE 0 0 0 errors: No known data errors
9時間ずれてるな。。
# zpool attach ztank ada0p3 ada1p3 Make sure to wait until resilver is done before rebooting. If you boot from pool 'ztank', you may need to update boot code on newly attached disk 'ada1p3'. Assuming you use GPT partitioning and 'da0' is your new boot disk you may use the following command: gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0
しばらくたった後。
# zpool status pool: ztank state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Sat Oct 17 12:20:15 2015 33.4G scanned out of 2.67T at 10.6M/s, 72h28m to go 33.4G resilvered, 1.22% done config: NAME STATE READ WRITE CKSUM ztank ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ada0p3 ONLINE 0 0 0 ada1p3 ONLINE 0 0 0 (resilvering) errors: No known data errors
残り72時間って。。