DISK交換

昨年の9月にほとんど同じ日記を書いてた。やっててよかったZFS

10/15に録画サーバが応答しなくなってるのに気づいて、コンソール見てみると断末魔が。

(ada1:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 49 00 40 00 00 00 00 00 00
(ada1:ahcich1:0:0:0): CAM status: ATA Status Error
(ada1:ahcich1:0:0:0): ATA status: 61 (DRDY DF ERR), error: 04 (ABRT )
(ada1:ahcich1:0:0:0): RES: 61 04 d1 cc 02 40 21 00 00 00 00
(ada1:ahcich1:0:0:0): Retrying command
(ada1:ahcich1:0:0:0): READ_FPDMA_QUEUED. ACB: 60 08 18 49 00 40 00 00 00 00 00 00
(ada1:ahcich1:0:0:0): CAM status: ATA Status Error
(ada1:ahcich1:0:0:0): ATA status: 61 (DRDY DF ERR), error: 04 (ABRT )
(ada1:ahcich1:0:0:0): RES: 61 04 d1 cc 02 40 21 00 00 00 00
(ada1:ahcich1:0:0:0): Error 5, Retries exhausted
swap_pager: I/O error - pagein failed; blkno 264463,size 4096, error 5
vm_fault: pager read error, pid 98520 (http)

どうにもならないのでリセットして再起動、立ち上がった後のzpoolはこんな感じ。

  pool: ztank
 state: DEGRADED
status: One or more devices could not be opened.  Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
   see: http://illumos.org/msg/ZFS-8000-2Q
  scan: resilvered 2.64T in 22h39m with 0 errors on Tue Sep 16 11:01:07 2014
config:

NAME                     STATE     READ WRITE CKSUM
ztank                    DEGRADED     0     0     0
  mirror-0               DEGRADED     0     0     0
    7724617931313753760  UNAVAIL      0     0     0  was /dev/ada1p3
    ada0p3               ONLINE       0     0     0

これが木曜の夜で、そのままAmazonで同じWD30EZRXを注文、金曜に帰ってきたら届いているという、21世紀ってすごいね。。

DISKを取り替えてシングルユーザで起動したら、何かそのままresilverが走り出した?

  pool: ztank
 state: DEGRADED
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sat Oct 17 11:48:15 2015
        2.35G scanned out of 2.67T at 1/s. (scan is slow, no estimated time)
        7.81M resilvered. 0.09% done
config:

	NAME                                   STATE     READ WRITE CKSUM
	ztank                                    ONLINE       0     0     0
	  mirror-0                             ONLINE       0     0     0
            7724617931313753760  UNAVAIL      0     0     0  was /dev/ada1p3
	    ada0p3                            ONLINE       0     0     0

errors: No known data errors

取り替えるより先にpoolから抜いとくべきだったのかなあ。とりあえず外す。

# zpool detach ztank ada1p3
# zpool status
  pool: ztank
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sat Oct 17 12:20:15 2015
        29.6G scanned out of 2.67T at 21/s, (scan is slow, no estimated time)
        7.81M resilvered, 1.08% done
config:

	NAME        STATE     READ WRITE CKSUM
	ztank       ONLINE       0     0     0
	  ada0p3  ONLINE       0     0     0

errors: No known data errors

シングルユーザモードでの、いつもの作業前のお約束。

# mount -u /
# zfs mount -o ro ztank/usr
# zfs mount ztank/var
# zfs mount ztank/var/empty
# zfs mount ztank/tmp
# /etc/rc.d/adjkerntz start

HDDが同じなので、既存のada0と同じようにgpart。

# gpart show ada0
=>        34  5860533101  ada0  GPT  (2.7T)
          34           6        - free -  (3.0k)
          40         128     1  freebsd-boot  (64k)
         168     2097152     2  freebsd-swap  (1.0G)
     2097320  5858435808     3  freebsd-zfs  (2.7T)
  5860533128           7        - free -  (3.5k)
# gpart create -s GPT ada1
ada1 created
# gpart add -b 40 -s 128 -t freebsd-boot ada1
ada1p1 added
# gpart add -b 168 -s 2097152 -t freebsd-swap ada1
ada1p2 added
# gpart add -b 2097320 -t freebsd-zfs ada1
ada1p3 added
# gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 ada1
bootcode witten to ada1

このままシングルユーザでattachしてresilver待ちすると、今日の録画予約が間に合わん。。ということで、
リブート通常起動してから、続き。

# zpool status
  pool: ztank
 state: ONLINE
  scan: resilvered 7.81M in 9h17m with 0 errors on Sat Oct 17 21:06:00 2015
config:

	NAME        STATE     READ WRITE CKSUM
	ztank       ONLINE       0     0     0
	  ada0p3  ONLINE       0     0     0

errors: No known data errors

9時間ずれてるな。。

# zpool attach ztank ada0p3 ada1p3
Make sure to wait until resilver is done before rebooting.

If you boot from pool 'ztank', you may need to update
boot code on newly attached disk 'ada1p3'.

Assuming you use GPT partitioning and 'da0' is your new boot disk
you may use the following command:

        gpart bootcode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0

しばらくたった後。

# zpool status
  pool: ztank
 state: ONLINE
status: One or more devices is currently being resilvered.  The pool will
	continue to function, possibly in a degraded state.
action: Wait for the resilver to complete.
  scan: resilver in progress since Sat Oct 17 12:20:15 2015
        33.4G scanned out of 2.67T at 10.6M/s, 72h28m to go
        33.4G resilvered, 1.22% done
config:

	NAME        STATE     READ WRITE CKSUM
	ztank       ONLINE       0     0     0
	  mirror-0  ONLINE       0     0     0
	    ada0p3  ONLINE       0     0     0
	    ada1p3  ONLINE       0     0     0  (resilvering)

errors: No known data errors

残り72時間って。。