mdadm modes
mdadm has seven modes:
Assemble
Used the most, but mostly runs in the background. Every time the system is booted, assemble will need to run.
Assemble scans the drives, looks for superblocks, and rebuilds the arrays.
You need an initramfs when booting off a raid array – because mdadm is a user-space program.
If the root partition is located on an array then we have a catch-22 situation: we can’t boot until we have the root partition in read-write live mode, and we can’t have root until we’ve booted and are then able to run mdadm.
Create
This is used a lot in practice: it is used to create arrays, and writes the superblocks for arrays that have them.
It also initialises to make sure the disks of a mirror are identical, or that the parities in an arry are correct.
Grow
Grow takes care of all operations relating to changing the size of an array, such as changing raid level, changing the number of active devices, adding disks, etc.
Manage
Manage is the default mode for mdadm. It is used primarily to add and remove devices. Manage has a number of additional options such as –add which are also found in grow mode
Build
This is an older mode, used to (re)create an array. It should only be used with care.
Misc
This covers a number of other actions.
mdadm Command Overview
root@asus:~#
root@asus:~# mdadm –manage –help
Usage: mdadm arraydevice options component devices…
This usage is for managing the component devices within an array.
The –manage option is not needed and is assumed if the first argument
is a device name or a management option.
The first device listed will be taken to be an md array device, any
subsequent devices are (potential) components of that array.
Options that are valid with management mode are:
–add -a : hotadd subsequent devices to the array
–re-add : subsequent devices are re-added if there were
: recent members of the array
–remove -r : remove subsequent devices, which must not be active
–fail -f : mark subsequent devices a faulty
–set-faulty : same as –fail
–replace : mark device(s) to be replaced by spares. Once
: replacement completes, device will be marked faulty
–with : Indicate which spare a previous ‘–replace’ should
: prefer to use
–run -R : start a partially built array
–stop -S : deactivate array, releasing all resources
–readonly -o : mark array as readonly
–readwrite -w : mark array as readwrite
root@asus:~#
Some Basic mdadm Commands
To display detailed information about a RAID device, use mdadm -D with the RAID device to query:
mdadm -D /dev/md0
How to automount RAIDs on reboot:
Note that the UUIDs used in mdadm.conf are related to the MD RAID device driver array, whereas the UUIDs used in fstab are related to filesystems on the RAID array – a subtle difference.
You have to use filesystem UUID numbers for the fstab which are different to the mdadm.conf ones.
You can list them with
blkid /dev/disk/by-id/md-uuid-${UUID}
Alternatively you can use
blkid ${RAID_DEVICE}.
[root@centos1vm ~]# mdadm -D /dev/md127
/dev/md127:
Version : 1.2
Creation Time : Sat Apr 9 00:48:11 2022
Raid Level : raid1
Array Size : 1527808 (1492.00 MiB 1564.48 MB)
Used Dev Size : 1527808 (1492.00 MiB 1564.48 MB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Sat Apr 9 11:53:15 2022
State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Consistency Policy : resync
Name : centos1vm:RAID50 (local to host centos1vm)
UUID : 66470135:4f0bd96a:357fe54e:2baa3a7b
Events : 17
Number Major Minor RaidDevice State
0 9 1 0 active sync /dev/md1
1 9 2 1 active sync /dev/md2
[root@centos1vm ~]#
the above is actually a RAID50: consisting of two RAID5s (md1 and md2 respectively)
see below:
[root@centos1vm ~]# mdadm -D /dev/md1
/dev/md1:
Version : 1.2
Creation Time : Sat Apr 9 00:41:10 2022
Raid Level : raid5
Array Size : 1529856 (1494.00 MiB 1566.57 MB)
Used Dev Size : 509952 (498.00 MiB 522.19 MB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Sat Apr 9 11:53:15 2022
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : resync
Name : centos1vm:RAID5a (local to host centos1vm)
UUID : 57784eb2:7d894249:e368bbd6:4fef82a1
Events : 20
Number Major Minor RaidDevice State
0 8 25 0 active sync /dev/sdb9
1 8 26 1 active sync /dev/sdb10
2 8 27 2 active sync /dev/sdb11
4 8 28 3 active sync /dev/sdb12
[root@centos1vm ~]# mdadm -D /dev/md2
/dev/md2:
Version : 1.2
Creation Time : Sat Apr 9 00:41:25 2022
Raid Level : raid5
Array Size : 1529856 (1494.00 MiB 1566.57 MB)
Used Dev Size : 509952 (498.00 MiB 522.19 MB)
Raid Devices : 4
Total Devices : 4
Persistence : Superblock is persistent
Update Time : Sat Apr 9 11:53:15 2022
State : clean
Active Devices : 4
Working Devices : 4
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Consistency Policy : resync
Name : centos1vm:RAID5b (local to host centos1vm)
UUID : cb67bc06:d32061ea:eda8873f:c15b9fbf
Events : 20
Number Major Minor RaidDevice State
0 8 29 0 active sync /dev/sdb13
1 8 30 1 active sync /dev/sdb14
2 8 31 2 active sync /dev/sdb15
4 259 0 3 active sync /dev/sdb16
[root@centos1vm ~]#
This displays RAID level, the array size, the health of the individual drives, the UUID of the array, and the component devices and their current roles.
You can get shortened details for an array, eg to add to the /dev/mdadm/mdadm.conf file, by using –brief or -b flags with the -D:
mdadm -Db /dev/md0
[root@centos1vm ~]# mdadm -Db /dev/md127
ARRAY /dev/md127 metadata=1.2 name=centos1vm:RAID50 UUID=66470135:4f0bd96a:357fe54e:2baa3a7b
[root@centos1vm ~]# mdadm -Db /dev/md1
ARRAY /dev/md1 metadata=1.2 name=centos1vm:RAID5a UUID=57784eb2:7d894249:e368bbd6:4fef82a1
[root@centos1vm ~]# mdadm -Db /dev/md2
ARRAY /dev/md2 metadata=1.2 name=centos1vm:RAID5b UUID=cb67bc06:d32061ea:eda8873f:c15b9fbf
[root@centos1vm ~]#
To get a quick human-readable overview of a RAID device, use the -Q option:
mdadm -Q /dev/md127
[root@centos1vm ~]# mdadm -Q /dev/md1
/dev/md1: 1494.00MiB raid5 4 devices, 0 spares. Use mdadm –detail for more detail.
/dev/md1: device 0 in 2 device active raid1 /dev/md/RAID50. Use mdadm –examine for more detail.
[root@centos1vm ~]# mdadm -Q /dev/md2
/dev/md2: 1494.00MiB raid5 4 devices, 0 spares. Use mdadm –detail for more detail.
/dev/md2: device 1 in 2 device active raid1 /dev/md/RAID50. Use mdadm –examine for more detail.
[root@centos1vm ~]# mdadm -Q /dev/md127
/dev/md127: 1492.00MiB raid1 2 devices, 0 spares. Use mdadm –detail for more detail.
[root@centos1vm ~]#
Display all Disks
you can display a list of all disks on the system with lsblk:
[root@centos1vm ~]#
[root@centos1vm ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 10G 0 disk
├─sda1 8:1 0 1G 0 part /boot
└─sda2 8:2 0 9G 0 part
├─cs_centos–base-root 253:0 0 8G 0 lvm /
└─cs_centos–base-swap 253:1 0 1G 0 lvm [SWAP]
sdb 8:16 0 10G 0 disk
├─sdb1 8:17 0 500M 0 part
│ └─md127 9:127 0 1.5G 0 raid10 /RAID10
├─sdb2 8:18 0 500M 0 part
│ └─md127 9:127 0 1.5G 0 raid10 /RAID10
├─sdb3 8:19 0 500M 0 part
│ └─md127 9:127 0 1.5G 0 raid10 /RAID10
├─sdb4 8:20 0 1K 0 part
├─sdb5 8:21 0 500M 0 part
│ └─md127 9:127 0 1.5G 0 raid10 /RAID10
├─sdb6 8:22 0 500M 0 part
├─sdb7 8:23 0 500M 0 part
├─sdb8 8:24 0 500M 0 part
├─sdb9 8:25 0 500M 0 part
│ └─md1 9:1 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb10 8:26 0 500M 0 part
│ └─md1 9:1 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb11 8:27 0 500M 0 part
│ └─md1 9:1 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb12 8:28 0 500M 0 part
│ └─md1 9:1 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb13 8:29 0 500M 0 part
│ └─md2 9:2 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb14 8:30 0 500M 0 part
│ └─md2 9:2 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb15 8:31 0 500M 0 part
│ └─md2 9:2 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb16 259:0 0 500M 0 part
│ └─md2 9:2 0 1.5G 0 raid5
│ └─md125 9:125 0 1.5G 0 raid1 /RAID50
├─sdb17 259:1 0 500M 0 part
│ └─md127 9:127 0 1.5G 0 raid10 /RAID10
├─sdb18 259:2 0 500M 0 part
│ └─md127 9:127 0 1.5G 0 raid10 /RAID10
└─sdb19 259:3 0 500M 0 part
sr0 11:0 1 9.2G 0 rom
[root@centos1vm ~]#
Adding Capacity to a RAID
Adding a new device to a mirror raid won’t change the size of the raid. This is because the new device will store an extra copy of the data and there won’t be any extra net space created.
But adding a device to a parity array will normally increase the space available.
For example, if we have a 3 by 4TB using raid-5, then that gives us an 8TB sized raid.
Adding a fourth 4TB will give us a 12TB raid. But note that we have to run the resize process in order to utilize it.
the syntax is
mdadm --add /dev/<md device> /dev/<disk you are adding> mdadm --grow --raid-devices=<expanded total no of devices> /dev/<md device>
example:
[root@centos1vm ~]# mdadm –add /dev/md/RAID10 /dev/sdb17
mdadm: added /dev/sdb17
[root@centos1vm ~]#
[root@centos1vm ~]# mdadm –grow –raid-devices=5 /dev/md/RAID10
[root@centos1vm ~]#
To allow recovery after unexpected power failure, an additional option –backup-file= can be specified. Make sure this file is on a different disk!
mdadm –grow –raid-devices=4 –backup-file=/root/grow_md0.bak /dev/md0
next do an unmount:
umount /dev/md/RAID10
[root@centos1vm ~]# umount /dev/md/RAID10
[root@centos1vm ~]# df
Filesystem 1K-blocks Used Available Use% Mounted on
devtmpfs 653280 0 653280 0% /dev
tmpfs 672308 0 672308 0% /dev/shm
tmpfs 672308 8884 663424 2% /run
tmpfs 672308 0 672308 0% /sys/fs/cgroup
/dev/mapper/cs_centos–base-root 8374272 3198204 5176068 39% /
/dev/sda1 1038336 356816 681520 35% /boot
/dev/md125 1470992 881312 496908 64% /RAID50
tmpfs 134460 0 134460 0% /run/user/0
[root@centos1vm ~]#
followed by a file integrity check:
e2fsck -f /dev/md127
then resize filesystem:
resize2fs /dev/md127
[root@centos1vm ~]# resize2fs /dev/md127
resize2fs 1.45.6 (20-Mar-2020)
Please run ‘e2fsck -f /dev/md127’ first.
[root@centos1vm ~]# e2fsck -f /dev/md127
e2fsck 1.45.6 (20-Mar-2020)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Inode 2 ref count is 5, should be 4. Fix<y>? yes
Pass 5: Checking group summary information
Inode bitmap differences: -(1121–2189)
Fix<y>? yes
/dev/md127: ***** FILE SYSTEM WAS MODIFIED *****
/dev/md127: 1120/63744 files (1.1% non-contiguous), 118303/254976 blocks
[root@centos1vm ~]#
[root@centos1vm ~]#
After mdadm completes growing the array it does not automatically modify /etc/mdadm.conf.
This will mean mdadm will not be able to locate the grown device /dev/md0.
So you next need to edit /etc/mdadm.conf and correct the num-devices information set for your raid.
[root@centos1vm etc]#
[root@centos1vm etc]# mdadm –detail –scan
ARRAY /dev/md2 metadata=1.2 name=centos1vm:RAID5b UUID=afc89f2b:03016129:218a6a68:2dd2e6e5
ARRAY /dev/md1 metadata=1.2 name=centos1vm:RAID5a UUID=57784eb2:7d894249:e368bbd6:4fef82a1
ARRAY /dev/md/RAID10 metadata=1.2 name=centos1vm:RAID10 UUID=60ae2e29:c1621782:65c1b59c:67c9d0b4
INACTIVE-ARRAY /dev/md126 metadata=1.2 name=centos1vm:RAID10 UUID=3dd59b4a:6f3cdf67:f89659d6:5e0f1c0d
ARRAY /dev/md/RAID50 metadata=1.2 name=centos1vm:RAID50 UUID=66470135:4f0bd96a:357fe54e:2baa3a7b
[root@centos1vm etc]# blkid /dev/md/RAID10
/dev/md/RAID10: UUID=”7e92383a-e2fb-48a1-8602-722d5c394158″ BLOCK_SIZE=”4096″ TYPE=”ext4″
[root@centos1vm etc]#
taking the blkid UUID output as shown above, put the new UUID in the /etc/mdadm.conf:
7e92383a-e2fb-48a1-8602-722d5c394158
so it now looks like this:
[root@centos1vm etc]# mdadm –detail –scan
ARRAY /dev/md2 metadata=1.2 name=centos1vm:RAID5b UUID=afc89f2b:03016129:218a6a68:2dd2e6e5
ARRAY /dev/md1 metadata=1.2 name=centos1vm:RAID5a UUID=57784eb2:7d894249:e368bbd6:4fef82a1
ARRAY /dev/md/RAID10 metadata=1.2 name=centos1vm:RAID10 UUID=60ae2e29:c1621782:65c1b59c:67c9d0b4
INACTIVE-ARRAY /dev/md126 metadata=1.2 name=centos1vm:RAID10 UUID=3dd59b4a:6f3cdf67:f89659d6:5e0f1c0d
ARRAY /dev/md/RAID50 metadata=1.2 name=centos1vm:RAID50 UUID=66470135:4f0bd96a:357fe54e:2baa3a7b
[root@centos1vm etc]#
so it now looks like this:
[root@centos1vm etc]# mdadm –detail /dev/md/RAID10
/dev/md/RAID10:
Version : 1.2
Creation Time : Sat Apr 9 12:10:16 2022
Raid Level : raid10
Array Size : 1274880 (1245.00 MiB 1305.48 MB)
Used Dev Size : 509952 (498.00 MiB 522.19 MB)
Raid Devices : 5
Total Devices : 5
Persistence : Superblock is persistent
Update Time : Mon Apr 11 17:51:54 2022
State : clean
Active Devices : 5
Working Devices : 5
Failed Devices : 0
Spare Devices : 0
Layout : near=2
Chunk Size : 512K
Consistency Policy : resync
Name : centos1vm:RAID10 (local to host centos1vm)
UUID : 60ae2e29:c1621782:65c1b59c:67c9d0b4
Events : 59
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 18 1 active sync /dev/sdb2
2 8 19 2 active sync /dev/sdb3
3 8 21 3 active sync /dev/sdb5
4 259 1 4 active sync /dev/sdb17
[root@centos1vm etc]#
you can see the new disk sdb17 listed.
finally do,
root@centos1vm etc]# mdadm –assemble –scan
and then remount:
and then remount:
mount /dev/md127 /RAID10
df
/dev/md127 987480 440788 479316 48% /RAID10
[root@centos1vm etc]#
Adding More Disks
a recap…
ADD – then grow – then resize!
mdadm –add /dev/md/RAID10 /dev/sdb18
mdadm –grow –raid-devices=6 /dev/md/RAID10
umount /dev/md/RAID10
e2fsck -f /dev/md127
resize2fs /dev/md127
[root@centos1vm ~]# mdadm –grow –raid-devices=6 /dev/md/RAID10
mdadm: Need 1 spare to avoid degraded array, and only have 0.
Use –force to over-ride this check.
[root@centos1vm ~]# mdadm –add /dev/md/RAID10 /dev/sdb18
mdadm: added /dev/sdb18
[root@centos1vm ~]# mdadm –grow –raid-devices=6 /dev/md/RAID10
[root@centos1vm ~]# umount /dev/md/RAID10
[root@centos1vm ~]# e2fsck -f /dev/md127
e2fsck 1.45.6 (20-Mar-2020)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/md127: 1120/63744 files (1.1% non-contiguous), 118303/254976 blocks
[root@centos1vm ~]# resize2fs /dev/md127
resize2fs 1.45.6 (20-Mar-2020)
Resizing the filesystem on /dev/md127 to 382464 (4k) blocks.
The filesystem on /dev/md127 is now 382464 (4k) blocks long.
[root@centos1vm ~]#
then edit the mdadm.conf for the new UUID for the expanded raid:
old UUIID:
[root@centos1vm ~]# mdadm –detail –scan
ARRAY /dev/md1 metadata=1.2 name=centos1vm:RAID5a UUID=57784eb2:7d894249:e368bbd6:4fef82a1
ARRAY /dev/md2 metadata=1.2 name=centos1vm:RAID5b UUID=afc89f2b:03016129:218a6a68:2dd2e6e5
ARRAY /dev/md/RAID10 metadata=1.2 name=centos1vm:RAID10 UUID=60ae2e29:c1621782:65c1b59c:67c9d0b4
INACTIVE-ARRAY /dev/md126 metadata=1.2 name=centos1vm:RAID10 UUID=3dd59b4a:6f3cdf67:f89659d6:5e0f1c0d
ARRAY /dev/md/RAID50 metadata=1.2 name=centos1vm:RAID50 UUID=66470135:4f0bd96a:357fe54e:2baa3a7b
[root@centos1vm ~]#
new UUID:
[root@centos1vm ~]# blkid /dev/md/RAID10
/dev/md/RAID10: UUID=”7e92383a-e2fb-48a1-8602-722d5c394158″ BLOCK_SIZE=”4096″ TYPE=”ext4″
[root@centos1vm ~]#
change this to the new UUID:
7e92383a-e2fb-48a1-8602-722d5c394158
[root@centos1vm ~]# nano /etc/mdadm.conf
[root@centos1vm ~]#
[root@centos1vm ~]#
[root@centos1vm ~]# cat /etc/mdadm.conf
ARRAY /dev/md1 metadata=1.2 name=centos1vm:RAID5a UUID=57784eb2:7d894249:e368bbd6:4fef82a1
ARRAY /dev/md2 metadata=1.2 name=centos1vm:RAID5b UUID=afc89f2b:03016129:218a6a68:2dd2e6e5
ARRAY /dev/md/RAID10 metadata=1.2 name=centos1vm:RAID10 UUID=7e92383a-e2fb-48a1-8602-722d5c394158
INACTIVE-ARRAY /dev/md126 metadata=1.2 name=centos1vm:RAID10 UUID=3dd59b4a:6f3cdf67:f89659d6:5e0f1c0d
ARRAY /dev/md/RAID50 metadata=1.2 name=centos1vm:RAID50 UUID=66470135:4f0bd96a:357fe54e:2baa3a7b
[root@centos1vm ~]#
check with:
[root@centos1vm ~]# mdadm –detail /dev/md/RAID10
mdadm: Unknown keyword INACTIVE-ARRAY
/dev/md/RAID10:
Version : 1.2
Creation Time : Sat Apr 9 12:10:16 2022
Raid Level : raid10
Array Size : 1529856 (1494.00 MiB 1566.57 MB)
Used Dev Size : 509952 (498.00 MiB 522.19 MB)
Raid Devices : 6
Total Devices : 6
Persistence : Superblock is persistent
Update Time : Mon Apr 11 18:21:46 2022
State : clean
Active Devices : 6
Working Devices : 6
Failed Devices : 0
Spare Devices : 0
Layout : near=2
Chunk Size : 512K
Consistency Policy : resync
Name : centos1vm:RAID10 (local to host centos1vm)
UUID : 60ae2e29:c1621782:65c1b59c:67c9d0b4
Events : 99
Number Major Minor RaidDevice State
0 8 17 0 active sync set-A /dev/sdb1
1 8 18 1 active sync set-B /dev/sdb2
2 8 19 2 active sync set-A /dev/sdb3
3 8 21 3 active sync set-B /dev/sdb5
4 259 1 4 active sync set-A /dev/sdb17
5 259 2 5 active sync set-B /dev/sdb18
[root@centos1vm ~]#
then remount, and you can see that as we now have 6 disks, we now have 1.5G available instead of 1GB:
[root@centos1vm ~]# mount /dev/md/RAID10 /RAID10
[root@centos1vm ~]#
[root@centos1vm ~]#
[root@centos1vm ~]# df -h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 638M 0 638M 0% /dev
tmpfs 657M 0 657M 0% /dev/shm
tmpfs 657M 8.7M 648M 2% /run
tmpfs 657M 0 657M 0% /sys/fs/cgroup
/dev/mapper/cs_centos–base-root 8.0G 3.1G 5.0G 39% /
/dev/sda1 1014M 349M 666M 35% /boot
/dev/md125 1.5G 861M 486M 64% /RAID50
tmpfs 132M 0 132M 0% /run/user/0
/dev/md127 1.5G 431M 933M 32% /RAID10
[root@centos1vm ~]#
To start a RAID
To start all arrays defined in the config file /etc/mdadm.conf or /proc/mdstat, type:
mdadm –assemble –scan
To start a specific array, you can pass it in as an argument to mdadm –assemble:
mdadm –assemble /dev/md0
This should work if the raid is defined in the configuration file.
If the correct definition for the raid is missing from the config file, the raid can still be started by specifying the component disk devices:
mdadm –assemble /dev/md0 /dev/sda /dev/sdb /dev/sdc /dev/sdd
Once the raid is assembled, it can be mounted as usual:
eg
mount /dev/md0 /mnt/md0
The raid should now be accessible at the mount point.
Adding a Spare Device to a Raid
Spare disks or devices can be added to any raid that offer redundancy (such as RAID 1, 5, 6, or 10).
The spare disk will not be actively used by the raid unless an active device fails. When this happens, the raid will resync the data to the spare drive to repair the array to full health.
Spares cannot be added to non-redundant raids (RAID 0) because the array will not survive the failure of a drive.
To add a spare, use the mdadm –add command:
eg
mdadm /dev/md0 –add /dev/sde
Provided the raid is not in a degraded state, the new device will be added as a spare.
If the device is currently degraded, the resync operation will immediately begin and will utilize the spare disk just added to replace the faulty drive.
After you add a spare, update the configu file to include the newly added disk:
in /etc/mdadm/mdadm.conf and append your current configuration:
mdadm –detail –brief /dev/md0 | sudo tee -a /etc/mdadm/mdadm.conf
Increasing the Number of Active Disks in a RAID
You can enlarge a RAID by increasing the number of active disks within the array.
The exact procedure will depends slightly on the RAID level of the array.
For RAID 1 or 10
First add the new disk as a spare:
mdadm /dev/md0 –add /dev/sde
Then find out the current number of RAID devices in the array:
mdadm –detail /dev/md0
Next reconfigure the array to include an additional active device. This spare will be used to meet the extra drive requirement:
mdadm –grow –raid-devices=3 /dev/md0
The array will now reconfigure with an additional active disk.
You can view the current syncing progress with:
cat /proc/mdstat
You can continue using the raid while the process completes.
For RAID 5 or 6
Add the new device as a spare, as shown in the previous example for Raid 1/10:
mdadm /dev/md0 –add /dev/sde
Ascertain the current number of RAID devices in the array:
mdadm –detail /dev/md0
Next, reconfigure the array to have an additional active disk. This spare will be used to satisfy the extra drive requirement.
When growing a RAID 5 or RAID 6 array, make sure you include an additional option called –backup-file.
This should point to a location outside the raid where a backup file containing critical information will be stored during the grow process, after which it is automatically deleted. In the event of something going wrong, the data contained in the backup file is used to rebuild the raid to its original pre-expansion state.
mdadm –grow –raid-devices=4 –backup-file=/root/md0_grow.bak /dev/md0
The raid will now reconfigure with the additional active disk.
To view the progress of syncing the data in real-time type:
cat /proc/mdstat
You can continue using the raid during this process.
Finally, once the grow process has completed, you MUST then run a resize filesystem command.
This is necessary in order to expand the filesystem on the raid to utilize the additional space:
eg
resize2fs /dev/md0
The expanded raid is now complete and ready for full use.
To remove a raid disk
For a device to be removed, first mark it as “failed” within the raid.
You can check if there is a failed device by using mdadm –detail:
mdadm –detail /dev/md0
If you need to remove a drive that does not have a problem at present, first manually mark it as failed with the –fail option:
mdadm /dev/md0 –fail /dev/sdc
Once the device has the status “failed”, it can then be removed from the raid using mdadm –remove:
mdadm /dev/md0 –remove /dev/sdc
You can then replace it with a new drive, using the same mdadm –add command that is used to add a spare:
mdadm /dev/md0 –add /dev/sdd
To delete an entire RAID
Quick overview:
To Delete a RAID:
Edit /etc/fstab and delete the line for the raid mount point.
Edit /etc/mdadm/mdadm.conf and delete the lines you added earlier via mdadm | tee.
Unmount the volume: umount <mountpoint>
Wipe the ext4 filesystem: sudo wipefs –all –force /dev/<md device>
Stop the RAID volume: sudo mdadm –stop /dev/<md device>
Zero the superblock on all the drives:
mdadm –zero-superblock /dev/<disk1> <disk2> ….etc
To destroy a raid, including all data contained within, first stop the array.
First, unmount the filesystem:
umount /mnt/md0
Next, stop the array:
mdadm –stop /dev/md0
Then delete the raid itself with the –remove command, specifying the RAID device:
mdadm –remove /dev/md0
Make sure you specify the correct raid device!
Once the array itself is removed, you should use mdadm –zero-superblock on each of the component devices.
This erases the md superblock, which is a header used by mdadm to assemble and manage the disks in a raid.
If this is still present, it can cause problems when trying to reuse the disk for other purposes.
To check if a superblock for a raid is still present on the device, use:
lsblk –fs
and see the column FS-TYPE
if this shows
linux_raid_member
then you need to erase it:
eg for each disk:
mdadm –zero-superblock /dev/sda /dev/sdb /dev/sdc
Other important cleaning up measures when deleting a raid:
Don’t forget also to remove any definition for the deleted raid in the fstab file, else this can cause a reboot of the machine to hang!
Also remove any definition of the deleted raid from the mdadm.conf file
Finally, update the initramfs by typing:
update-initramfs -u
This will remove the deleted raid device from the boot environment.