Logical Volume Management with RAID (mdadm)


Logical Volume Management


/ has just run out of space, what do we do now? With a normal partition setup, increasing the size of a partition is difficult as it can’t just be resized: typically you would have to backup your files, delete the partitions, create a new partition layout, create file systems, and copy all your files back on.

That’s a lot of work to just add some room to a partition that’s running out of space. Graphical tools such as gparted make this easier, but the system would still have to be taken down for maintenance and may not be easily available for the server in question. Here is where LVM comes in:

[root@localhost ~]# lvextend -L+1G /dev/new_volume/lv_root
  Size of logical volume new_volume/lv_root changed from 2.00 GiB (512 extents) to 3.00 GiB (768 extents).
  Logical volume lv_root successfully resized

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/new_volume-lv_root
                      2.0G  3.0M  1.9G   1% /mnt

[root@localhost ~]# resize2fs /dev/mapper/new_volume-lv_root
resize2fs 1.41.12 (17-May-2010)
Filesystem at /dev/mapper/new_volume-lv_root is mounted on /mnt; on-line resizing required
old desc_blocks = 1, new_desc_blocks = 1
Performing an on-line resize of /dev/mapper/new_volume-lv_root to 786432 (4k) blocks.
The filesystem on /dev/mapper/new_volume-lv_root is now 786432 blocks long.

[root@localhost ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/new_volume-lv_root
                      2.9G  3.0M  2.8G   1% /mnt

There are no guarantees with partition resizing. Always keep full up-to-date backups of your data.

With minimal time and work the root partition has now been increased by 1GB, and the system didn’t need to be taken off-line to do it. It was all done while the system was still running.

LVM can be used on a single hard drive, but it also takes multiple hard drives and pools the available space into a single logical volume group. The space is then allocated to the logical volumes that will be used for the Linux file system. If space is running low in an LVM, additional hard drives can be added to the volume group to provide space to extend the logical volumes. Logical volumes can be extended and shrunk as required.

lvg

LVM provides the ability to dynamically allocate space, but it does not provide redundancy (although newer versions may support this). If a disk died in a volume group the entire volume would be destroyed.

LVM on Linux software RAID array


The purpose of a Redundant Array of Independent Disks (at level 1 or higher) is to provide redundancy and keep the system up and running when a hard disk fails; if this happens, the missing data is provided on-the-fly from the redundant information on the other hard disks. This allows the system to continue uninterrupted until the drive can be replaced and the missing data is rebuilt. Using LVM with a RAID allows LVM to continue working even if one of the physical disks dies.

Using a RAID array with LVM does have its disadvantages. RAID will will use the size of the smallest disk when building the array due to the need to stripe information across the drives. In the example above the smallest disk is 100GB, and so only 100GB would be used from each of the other two drives despite their larger capacity.

In a RAID 5 array, the space of one drive will be used for redundancy which makes it unavailable for storage. This issue can be reduced by using more hard drives, but the more hard drives there are the greater the risk of failure becomes.

7

Total available space in a RAID 5 array = “size of smallest device” x (number of disks – 1)

100GB x (3 – 1) = 200GB. With LVM the entire 550GB was available for use in the volume group, but with a RAID 5 array only 200GB out of the 550GB would be available for use due to the mismatched device sizes. To make better use of the disk space the drive sizes should be matched more closely.

lvg-mdadm

Example Setup


I have used a virtual machine with 5 partitions to create an mdadm RAID 5 array. Normally the RAID setup would be done with multiple disks but the concept works for this example. The mdadm RAID array is then used to create a volume group from which the logical volumes can be created.

[root@localhost ~]# mdadm --create /dev/md0 --raid-devices=5 --level=5 /dev/xvdb5 \\
> /dev/xvdb6 /dev/xvdb7 /dev/xvdb8 /dev/xvdb9
[...]
md0: WARNING: xvdb9 appears to be on the same physical disk as xvdb8.
[...]
md0: WARNING: xvdb5 appears to be on the same physical disk as xvdb8.
True protection against single-disk failure might be compromised.
md/raid:md0: device xvdb8 operational as raid disk 3
md/raid:md0: device xvdb7 operational as raid disk 2
md/raid:md0: device xvdb6 operational as raid disk 1
md/raid:md0: device xvdb5 operational as raid disk 0
md/raid:md0: allocated 0kB
md/raid:md0: raid level 5 active with 4 out of 5 devices, algorithm 2
md0: detected capacity change from 0 to 6476005376
[...]
mdadm: array /dev/md0 started.
 md0: unknown partition table
[root@localhost ~]# cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid5 xvdb9[5] xvdb8[3] xvdb7[2] xvdb6[1] xvdb5[0]
      6324224 blocks super 1.2 level 5, 512k chunk, algorithm 2 [5/5] [UUUUU]

unused devices:

The RAID device is now created and ready to be used. Now to create the LVM volume on the RAID device.

[root@localhost ~]# pvcreate /dev/md0
  Physical volume "/dev/md0" successfully created
[root@localhost ~]# vgcreate new_volume /dev/md0
  Volume group "new_volume" successfully created
[root@localhost ~]# lvcreate -L 500M -n lv_swap new_volume
  Logical volume "lv_swap" created.
[root@localhost ~]# lvcreate -L 2G -n lv_root new_volume
  Logical volume "lv_root" created.
[root@localhost ~]# lvcreate -l 100%FREE -n lv_home new_volume
  Logical volume "lv_home" created.
[root@localhost ~]# mkfs.ext4 /dev/mapper/new_volume-lv_root
[...]
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done

[root@localhost]~# mkfs.ext4 /dev/mapper/new_volume-lv_home
[...]
Writing inode tables: done
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done