As the forecasts for the weather are pretty bad, decided to go for a small weekend project.
I do have redundant, or copies of a md array on both systems, the centos guest os on my esxi and centos standalone on my „SAN“. Bother are equipped with 4 x 2 TB disks, but the centos on esxi having the WD Red series, while the SAN has a mixture of old toshibas and WD disks. So the plan is to change – one by one – the disks in the SAN by the disk from the esxi guest. Also having no data disk or storage inside the Dell T20 enclosure, which is pretty difficult to change disk(s) , if one is failing. Having to shutdown ESXi is not an option, as I would loose also my internet connection (pfsense running as vm there).
The centos SAN proved to be stable and fast in the past few months.
First copy from the WD Red to the SAN, but keeping date and permission of the source files.
[root@centnas iscsi_share]# cp -Rp /storage/ada.local .
Next, I had to identify in my SAN, which disk is which. To be able later changing the disks one by one.
[root@centsan ~]# ls -l /dev/disk/by-id/ lrwxrwxrwx. 1 root root 9 16. Aug 12:59 ata-TOSHIBA_MK2002TSKB_4231KAPQF -> ../../sdc lrwxrwxrwx. 1 root root 10 16. Aug 12:59 ata-TOSHIBA_MK2002TSKB_4231KAPQF-part1 -> ../../sdc1 lrwxrwxrwx. 1 root root 9 16. Aug 12:59 ata-VB0250EAVER_9VMRPQ07 -> ../../sda lrwxrwxrwx. 1 root root 10 16. Aug 12:59 ata-VB0250EAVER_9VMRPQ07-part1 -> ../../sda1 lrwxrwxrwx. 1 root root 10 16. Aug 12:59 ata-VB0250EAVER_9VMRPQ07-part2 -> ../../sda2 lrwxrwxrwx. 1 root root 9 16. Aug 12:59 ata-WDC_WD2003FYYS-02W0B0_WD-WMAY01103310 -> ../../sdd lrwxrwxrwx. 1 root root 10 16. Aug 12:59 ata-WDC_WD2003FYYS-02W0B0_WD-WMAY01103310-part1 -> ../../sdd1
So I was able to identify the disk, when shutdown the SAN, put a label on the disk caddies, so it’s easy to identify which disk is in which slot.
After Sat evening gone, went on on Sunday. Opened my DELL T20 took out all WD Drives.
[root@centsan ~]# mdadm --fail /dev/md0 /dev/sdb1
Shutdown the SAN, replaced ONE disk power it on again, then repartitioned the replaced disk.
[root@centsan ~]# parted /dev/sdb (parted) rm 1 (parted) mklabel gpt Warnung: Die bestehende Partitionstabelle und alle Daten auf /dev/sdb werden gelöscht. Wollen Sie fortfahren? Ja/Yes/Nein/No? ja (parted) mkpart primary 0% 100% (parted) align-check alignment type(min/opt) [optimal]/minimal? Partitionsnummer? 1 1 aligned [root@centsan ~]# mdadm --add /dev/md0 /dev/sdb1
So I’m replacing one disk by one, and rebuild the RAID5 array. I guess, I could also just, swap all disks from T20 into the SAN.