Getting CentOS on RAID after using text mode installation...

Well, this was a hack. I got a firewall to install with only 512MB RAM which isn't enough for graphical install. And, using text install means that there are no customizations available, in particular, no customized disk partitioning. But since I don't install anything important without RAID I had to somehow do it. The options are:

1. Temporarily add more RAM only during installation process.
2. Use kickstart file.
3. Juggle with partitions.

Option 1 wasn't acceptable since I didn't have any extra RAM modules available, and option 2 seamed too complicated as I don't have USB flash drive to store kickstart file on it and network install is a bit too much on home network. So, I decided to go with option 3. Note that there was one more thing that was in favor for option 3, namely one disk has capacity of 80GB while the other 250GB, which means there is extra space I can use as a temporary storage. Still, even if I didn't have it, option 3 is still viable with a bit more juggling.

So, the general idea is as follow:

1. Install CentOS on a temporary partition.
2. Create RAID array and move CentOS there.
3. Fix boot.

Those three steps are not in strict sequence and so clearly separated, as we'll see, but are logically grouped. Also, just to clarify, a helper partition is a disk space after 80th GB on a second (larger) disk!

Install CentOS on a helper partition

This is a first step and it's easy. Boot from DVD and start installation process. Except from one small detail. Namely, how to persuade installer to use helper partition when it does almost everything automatically? Well, that's actually easy to solve. After Anaconda starts, but before doing anything, switch to second virtual terminal (Alt+F2) and use fdisk to create partitions for RAID. In my case those were:

1. /dev/sda1 and /dev/sdb1 of size 256MB for /boot partition
2. /dev/sda2 and /dev/sdb2 of size 2G for swap
3. /dev/sda3 and /dev/sdb3 that take the rest of space to fill 80GB and that will be root (/) partition.

This will take first 80GB on both disks, and leave empty space on a second disk. Now, go back to installer and continue installation. When the installer asks you where to install Linux, select "Emtpy space". Note that later I was thinking that it was better if I created another partition so that free space is smaller. This speeds up the process of installation as creating file systems is faster! Anyway, for a suggestion on minimal CentOS installation you can look what I wrote in this post.

Create RAID arrays and move CentOS there

After the installer finishes, don't boot into a new system. Boot again from DVD and select rescue mode. Also, allow installer to search for existing installations and select to mount them in read-write mode! Finally, select shell from a menu that appears.

Now, create RAID arrays. If you used partitions as I did, then the following sequence of commands will do the work.

mdadm -C /dev/md0 --metadata=0.90 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1

This creates first RAID1 array for boot partition. Note that I'm using metadata version 0.90. This is because grub doesn't understand later formats! That "little" fact costed me a lot of time!

mdadm -C /dev/md1 --level=1 --raid-devices=2 /dev/sda2 /dev/sdb2
mdadm -C /dev/md2 --level=1 --raid-devices=2 /dev/sda3 /dev/sdb3

Those two commands create RAID1 arrays for swap and root. Maybe at this point you'll need to activate arrays (i.e. it isn't done automatically by above mdadm commands):

mdadm --activate --scan

Now, create file systems on those arrays:

mkfs.ext4 /dev/md0
mkfs.ext4 /dev/md2
mkswap /dev/md1

RAID arrays are now created. The next step is to move installed CentOS from temporary partition. First, you have to mount destination filesystems:

mkdir /mnt/s

mount /dev/md2 /mnt/s

mkdir /mnt/s/boot

mount /dev/md0 /mnt/s/boot

After they are mounted, copy all the files:

cd /mnt/sysimage

rsync -av bin boot etc home lib lib64 opt root sbin tmp usr var /mnt/s/

Note that I skipped some in-memory file systems like proc, dev, and similar ones. You should only create those directories without content as their content is recreated during each boot and held in memory:

cd /mnt/s
mkdir dev media mnt proc selinux srv sys

That's it for filesystems. We need now to adjust /etc/fstab and /etc/grub.conf files as they reference temporary partition/filesystem used by installer. So, change /etc/fstab to contain the following lines/filesystems:

/dev/md2    /       ext4    defaults   1  1
/dev/md0    /boot   ext4    defaults   1  2
/dev/md1    swap    swap    defaults   0  0

There will be also lines starting with tmpfs, devpts, sysfs and proc. Leave those as is and remove all the other lines. As for the /etc/grub.conf file, you need to modify any occurrences of (hdN,M). Those will be in two places. One in splashimage line, that one isn't so important. And the other in line starting with root keyword. That one IS important! Also, remove from the line that starts with keyword kernel any word that contains substring LV (logical volume!). After a change, this line should look something like this:

kernel /vmlinuz-2.6.32-220.el6.x86_64 ro root=/dev/md2 LANG=en_US.UTF-8 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us

Note that this is a single line! The important part is root=, others might differ if you selected some other language or keyboard layout during installation process. Also, I removed keywords rhgb and quiet. Those prevent me to see kernel messages during the boot process. And those messages could be very important, especially in this case when we are doing something that could impact early boot process itself!

Finally, remove temporary partition. Do that using fdisk, just delete that partition. Note that this will take effect on next reboot, so nothing will happen and the computer won't crash. :)

And, that's it for moving CentOS to RAID partition.

Fixing boot loader

This is the last step. Boot from DVD and select rescue mode. Again allow installer to scan disk for Linux partitions. But this time installer should find RAID partitions and CentOS on them. Mount them in read-write mode and enter shell again. Execute the following command:

chroot /mnt/sysimage

This will activate targeted CentOS on RAID.

First, we have to recreate /etc/mdadm.conf. Note that if you don't do that the system won't boot. This troubled me until I figured out that that file contained stale data filled by Anaconda and I didn't refresh its content. So, to refresh it, do the following:

mdadm --examine --scan > /etc/mdadm.conf

Open /etc/mdadm.conf in the editor and add the following line at the beginning:

DEVICES /dev/sd*[0-9] /dev/hd*[0-9]

Also be careful, I typed etc instead of dev and it costed me two additional reboots. :)

Now, recreate initramfs image. Do that with the following command:

mkinitrd -f -v /boot/initramfs-2.6.32-220.el6.x86_64.img 2.6.32-220.el6.x86_64

Be careful, -f forces mkinitrd to overwrite existing initramfs file, so maybe it would be good to make a copy of that file, just in case.

One final step and that's it, reinstall the boot loader! Do this with the following commands:

grub-install /dev/sda

grub-install /dev/sdb

If the two commands return error (some problem with stage1 or stage2 files) then do it "manually" like this:

# grub
grub> root (hd0,0)
grub> setup (hd0)
grub> root (hd1,0)
grub> setup (hd1)
grub> exit

Note that you type text in bold, while everything else is response from the system. And that's it! Reboot the system, this time from the hard disk, and you should have minimal CentOS installation on RAID partitions!