The server is running CentOS and it has been successfully running a two disk software RAID. I wish to add a third disk to this for redundancy, so when one disk fails, the system still has two disks operating until a replacement for the failed disk arrives.
When installing CentOS from a DVD-ROM, that's where I have set-up the software two disk RAID in the past. But how do you add a third disk to the existing two disk software RAID in a running system?
2 disks and redundancy suggest RAID1 is already used. To increase redundancy I do not think RAID5 is an option as it offers more space but - with 3 disks it is the same as RAID1 with 2 and allows losing one disk.
So presuming RAID1:
mdadm --add /dev/mdX /dev/sdY
mdadm --grow --raid-devices=3 /dev/mdX
Using lvm2 (either mirror or dm-raid) it would be:
vgextend VG /dev/sdY
lvconvert -m2 VG/LV
You want to reshape your RAID-1 to a three-disk RAID-5 setup. md, the software RAID you're probably using, fortunately supports reshaping.
ＷＡＲＮＩＮＧ： Make sure you have backed up your data just in case something goes catastrophically wrong (not likely, but always a possibility).
You didn't provide any details about your setup, so I'll do a full example from scratch anyone can try.
root@node51 [/tmp]# truncate -s 1G 1.img
root@node51 [/tmp]# truncate -s 1G 2.img
root@node51 [/tmp]# losetup /dev/loop1 1.img
root@node51 [/tmp]# losetup /dev/loop2 2.img
root@node51 [/tmp]# mdadm --create --verbose /dev/md0 --level=1 --raid-devices=2 /dev/loop{1,2}
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
--metadata=0.90
mdadm: size set to 1048000K
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
How the array looks as RAID-1:
root@node51 [/tmp]# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Mar 9 15:30:18 2016
Raid Level : raid1
Array Size : 1048000 (1023.61 MiB 1073.15 MB)
Used Dev Size : 1048000 (1023.61 MiB 1073.15 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Wed Mar 9 15:30:24 2016
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Name : node51:0 (local to host node51)
UUID : 9f03708b:dfe27926:16485757:02cbc146
Events : 17
Number Major Minor RaidDevice State
0 7 1 0 active sync /dev/loop1
1 7 2 1 active sync /dev/loop2
I'll demonstrate a simple filesystem setup and resize here, but you can find more detailed resizing tips elsewhere (see "Additional Resources" at the bottom of this answer).
root@node51 [/tmp]# mkfs.ext4 /dev/md0
mke2fs 1.42.9 (4-Feb-2014)
Discarding device blocks: done
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
65536 inodes, 262000 blocks
13100 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376
Allocating group tables: done
Writing inode tables: done
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done
root@node51 [/tmp]# mount /dev/md0 /mnt
root@node51 [/tmp]# df -hT /mnt
Filesystem Type Size Used Avail Use% Mounted on
/dev/md0 ext4 992M 1.3M 923M 1% /mnt
root@node51 [/tmp]# truncate -s 1G 3.img
root@node51 [/tmp]# losetup /dev/loop3 3.img
root@node51 [/tmp]# mdadm --add /dev/md0 /dev/loop3
mdadm: added /dev/loop3
Once you have added the third disk, you should see it as a spare:
root@node51 [/tmp]# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Mar 9 15:30:18 2016
Raid Level : raid1
Array Size : 1048000 (1023.61 MiB 1073.15 MB)
Used Dev Size : 1048000 (1023.61 MiB 1073.15 MB)
Raid Devices : 2
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Wed Mar 9 15:31:10 2016
State : clean
Active Devices : 2
Working Devices : 3
Failed Devices : 0
Spare Devices : 1
Name : node51:0 (local to host node51)
UUID : 9f03708b:dfe27926:16485757:02cbc146
Events : 18
Number Major Minor RaidDevice State
0 7 1 0 active sync /dev/loop1
1 7 2 1 active sync /dev/loop2
2 7 3 - spare /dev/loop3
root@node51 [/tmp]# mdadm --grow /dev/md0 --level=5 --raid-devices=3
mdadm: level of /dev/md0 changed to raid5
mdadm: Need to backup 128K of critical section..
This may take a long time. You can check cat /proc/mdstat to see the progress:
root@node51 [/tmp]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md0 : active raid5 loop3[2] loop2[1] loop1[0]
1048000 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]
[=====>...............] reshape = 25.0% (262208/1048000) finish=0.1min speed=87360K/sec
unused devices: <none>
root@node51 [/tmp]# mdadm --detail /dev/md0
/dev/md0:
Version : 1.2
Creation Time : Wed Mar 9 15:30:18 2016
Raid Level : raid5
Array Size : 2096000 (2047.22 MiB 2146.30 MB)
Used Dev Size : 1048000 (1023.61 MiB 1073.15 MB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Wed Mar 9 15:31:49 2016
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 64K
Name : node51:0 (local to host node51)
UUID : 9f03708b:dfe27926:16485757:02cbc146
Events : 41
Number Major Minor RaidDevice State
0 7 1 0 active sync /dev/loop1
1 7 2 1 active sync /dev/loop2
2 7 3 2 active sync /dev/loop3
Here, you can see my filesystem of about 1GiB become 2GiB by the online expanding feature of ext4:
root@node51 [/tmp]# df -hT /mnt
Filesystem Type Size Used Avail Use% Mounted on
/dev/md0 ext4 992M 1.3M 923M 1% /mnt
root@node51 [/tmp]# resize2fs /dev/md0
resize2fs 1.42.9 (4-Feb-2014)
Filesystem at /dev/md0 is mounted on /mnt; on-line resizing required
old_desc_blocks = 1, new_desc_blocks = 1
The filesystem on /dev/md0 is now 524000 blocks long.
root@node51 [/tmp]# df -hT /mnt
Filesystem Type Size Used Avail Use% Mounted on
/dev/md0 ext4 2.0G 1.5M 1.9G 1% /mnt
For other filesystem resizing scenarios, see "Additional Resources" at the end of this answer.
Here's how to tear down and delete the example above:
root@node51 [/tmp]# umount /mnt
root@node51 [/tmp]# mdadm --stop /dev/md0
mdadm: stopped /dev/md0
root@node51 [/tmp]# losetup -d /dev/loop1
root@node51 [/tmp]# losetup -d /dev/loop2
root@node51 [/tmp]# losetup -d /dev/loop3
root@node51 [/tmp]# rm -fv /tmp/{1,2,3}.img
removed ‘/tmp/1.img’
removed ‘/tmp/2.img’
removed ‘/tmp/3.img’
I wrote a few other answers covering filesystem resizing. You can use the same concept to grow the filesystem in your RAID.
