Setting up RAID with Debian on a Netra X1

If you haven't installed Debian on your Netra X1 yet, take a look here.

I have a number of Netra X1 systems with identical hard drives. In all of my other systems, I try to make the RAID1, at a minumum. I've done a great deal of digging, and I have yet to find anyone else publishing a procedure for setting up RAID1 volumes. In fact, a number of people have said it's impossible. I found a way to implement RAID1 on boot/root volumes. This same procedure could be used to convert any single drive system into a RAID1 pair.

(2006-05-09) UPDATE: There may be a problem with the instructions below. You'll likely want to start the /boot partition at cylinder 1 instead of 0. Silo corrupts the ext2/3 partition when writing the bootblock if you are using RAID on the first cylinder. Apparently the ext2/3 filesystem leaves space for the disklabel and boot sector when it's created on a plain disk, but doesn't when it's created on a RAID volume.

To repair an already built system using my partition layout:

1) apt-get install rsync parted
2) mkdir /boot2 ; cd /boot2; rsync -vaP /boot/. .
3) /etc/init.d/sysklogd stop
4) /etc/init.d/klogd stop
5) umount /boot
6) mdadm --stop /dev/md0
7) (fdisk /dev/hda and /dev/hdc) (delete partition 1, re-create it starting at 1, set type to "fd")
8) partprobe
9) mdadm --create -l1 -n2 -x0 /dev/md0 /dev/hd[ac]1
10) mkfs.ext3 /dev/md0
11) mount /dev/md0 /boot
12) cd /boot ; rsync -vaP /boot2/. .
13) cd /dev; mv hda3 hda3.old; ln -s hda hda3; mv hdc3 hdc3.old; ln -s hdc hdc3
14) silo
15) mkinitrd -o /boot/initrd.img-`uname -r` `uname -r`
16) (wait for RAID to rebuild)
    cd /etc/mdadm ; echo DEVICE partitions > mdadm.conf ; mdadm -Ds >> mdadm.conf
17) reboot
18) at OBP: setenv boot-device disk0:3 disk2:3 net
19) boot

Except for the obvious manual bits, I was cutting-n-pasting the commands above, and have repaired my Blade 100, an Ultra 2 and seven Netra X1's using this procedure. The only odd bit is that "boot disk" doesn't work any longer, hence the need for step 18.

What you need is:

A Netra X1 with two identical hard drives
Debian 3.1r0 (Sarge) installed and working on the first hard drive
A serial cable that works with the Netra X1

The procedure:

Power on your system and boot it:

 lom>poweron
 lom>
 LOM event: +6d+0h0m1s host power on

 Sun Netra X1 (UltraSPARC-IIe 400MHz), No Keyboard
 OpenBoot 4.0, 1024 MB memory installed, Serial #50674428.
 Ethernet address 0:3:ba:5:3a:fc, Host ID: 83053afc.


 ok boot
 Boot device: disk:a  File and args:
 SILO Version 1.4.9
 boot: Linux

Verify that you are working with two identical hard drives:
```
 netra0:~# cat /proc/ide/hd?/model
 ST320413A
 ST320413A
```

Copy the partition table over to the second drive:

 dd if=/dev/hda of=/dev/hdc bs=1024k count=10

Edit the partition table of the second drive so that all of the partitions, except #3, have type 'fd'.

 netra0:~# fdisk -l /dev/hda /dev/hdc

 Disk /dev/hda (Sun disk label): 16 heads, 63 sectors, 38792 cylinders
 Units = cylinders of 1008 * 512 bytes

    Device Flag    Start       End    Blocks   Id  System
 /dev/hda1             0       248    124992   83  Linux native
 /dev/hda2           248      2232    999936   83  Linux native
 /dev/hda3             0     38792  19551168    5  Whole disk
 /dev/hda4          2232     10169   4000248   83  Linux native
 /dev/hda5         10169     12153    999936   83  Linux native
 /dev/hda6         12153     18105   2999808   82  Linux swap
 /dev/hda7         18105     28026   5000184   83  Linux native
 /dev/hda8         28026     38792   5426064   83  Linux native

 Disk /dev/hdc (Sun disk label): 16 heads, 63 sectors, 38792 cylinders
 Units = cylinders of 1008 * 512 bytes

    Device Flag    Start       End    Blocks   Id  System
 /dev/hdc1             0       248    124992   fd  Linux raid autodetect
 /dev/hdc2           248      2232    999936   fd  Linux raid autodetect
 /dev/hdc3             0     38792  19551168    5  Whole disk
 /dev/hdc4          2232     10169   4000248   fd  Linux raid autodetect
 /dev/hdc5         10169     12153    999936   fd  Linux raid autodetect
 /dev/hdc6         12153     18105   2999808   fd  Linux raid autodetect
 /dev/hdc7         18105     28026   5000184   fd  Linux raid autodetect
 /dev/hdc8         28026     38792   5426064   fd  Linux raid autodetect

Install mdadm to manage the arrays.
```
 apt-get install mdadm
```
It'll ask you a series of questions that are highly dependent on your needs. One key one is: "Yes, automatically start RAID arrays"
Load the RAID1 module:
```
 modprobe raid1
```

Create the RAID1 volumes. Note that we're setting one mirror as "missing" here. We'll add the second half of the mirror later because we're using it right now.

 mdadm --create -n 2 -l 1 /dev/md0 /dev/hdc1 missing
 mdadm --create -n 2 -l 1 /dev/md1 /dev/hdc2 missing
 mdadm --create -n 2 -l 1 /dev/md2 /dev/hdc4 missing
 mdadm --create -n 2 -l 1 /dev/md3 /dev/hdc5 missing
 mdadm --create -n 2 -l 1 /dev/md4 /dev/hdc6 missing
 mdadm --create -n 2 -l 1 /dev/md5 /dev/hdc7 missing
 mdadm --create -n 2 -l 1 /dev/md6 /dev/hdc8 missing

Make the filesystems:

 mke2fs -j /dev/md0
 mke2fs -j /dev/md1
 mke2fs -j /dev/md2
 mke2fs -j /dev/md3
 mkswap /dev/md4
 mke2fs -j /dev/md5
 mke2fs -j /dev/md6

Install the dump package:
```
 apt-get install dump
```

Mount the new volumes, dump & restore from the running copies:

 mount /dev/md1 /mnt
 cd /mnt
 dump 0f - / | restore rf -
 rm restoresymtable

 mount /dev/md0 /mnt/boot
 cd /mnt/boot
 dump 0f - /boot | restore rf -
 rm restoresymtable

 mount /dev/md2 /mnt/usr
 cd /mnt/usr
 dump 0f - /usr | restore rf -
 rm restoresymtable

 mount /dev/md3 /mnt/tmp
 cd /mnt/tmp
 dump 0f - /tmp | restore rf -
 rm restoresymtable

 mount /dev/md5 /mnt/var
 cd /mnt/var
 dump 0f - /var | restore rf -
 rm restoresymtable

 mount /dev/md6 /mnt/export
 cd /mnt/export
 dump 0f - /export | restore rf -
 rm restoresymtable

Set up the chroot environment:

 mount -t proc none /mnt/proc

 chroot /mnt /bin/bash

Edit /boot/silo.conf, and change the following line:
```
 root=/dev/md1
```

Edit /etc/fstab, and point them to the MD devices:

 # /etc/fstab: static file system information.
 #
 #                
 proc            /proc           proc    defaults        0       0
 /dev/md1        /               ext3    defaults,errors=remount-ro 0       1
 /dev/md0        /boot           ext3    defaults        0       2
 /dev/md6        /export         ext3    defaults        0       2
 /dev/md3        /tmp            ext3    defaults        0       2
 /dev/md2        /usr            ext3    defaults        0       2
 /dev/md5        /var            ext3    defaults        0       2
 /dev/md4        none            swap    sw              0       0
 /dev/hdc        /media/cdrom0   iso9660 ro,user,noauto  0       0
 /dev/fd0        /media/floppy0  auto    rw,user,noauto  0       0

Save the MD information to /etc/mdadm/mdadm.conf:
```
 echo DEVICE partitions >> /etc/mdadm/mdadm.conf
```
```
 mdadm -D -s >> /etc/mdadm/mdadm.conf
```
2006-03-01: added the first line here, because it appears that Sarge ships with either no mdadm.conf, or a blank one. Without it, your raid volumes won't start correctly.
Rebuild the initrd (to add the RAID modules, and boot/root RAID startup information):
```
 mkinitrd -o /boot/initrd.img-`uname -r` `uname -r`
```
Leave the chroot environment:
```
 exit
```
Unmount /boot. klogd uses the System.map file, and we need to kill it to unmount /boot.
```
 pkill klogd
 # wait a few seconds
 umount /boot
```
Add /dev/hda1 to /dev/md0 -- the /boot mirror
```
 mdadm --add /dev/md0 /dev/hda1
 watch cat /proc/mdstat
```
Wait until the mirror is complete. CTRL-C to exit watch.
Mount the mirrored /boot:
```
 umount /mnt/boot
 mount /dev/md0 /boot
```
Stamp the boot loader onto both disks, and reboot:
```
 silo && reboot
```

Assuming it booted up correctly, verify that we're running on the mirrored copies:

 df -h

If so, add the other partitions into their respective mirrors:

 mdadm --add /dev/md0 /dev/hda1
 mdadm --add /dev/md1 /dev/hda2
 mdadm --add /dev/md2 /dev/hda4
 mdadm --add /dev/md3 /dev/hda5
 mdadm --add /dev/md4 /dev/hda6
 mdadm --add /dev/md5 /dev/hda7
 mdadm --add /dev/md6 /dev/hda8
 watch cat /proc/mdstat

And wait until the the mirrors are done building.

Edit /etc/mdadm/mdadm.conf and remove any references to the RAID volumes. Refresh the mdadm.conf information:
```
 mdadm -D -s >> /etc/mdadm/mdadm.conf
```
Rebuild the initrd one more time. The previous time only included one half of each mirror for root and swap.
```
 mkinitrd -o /boot/initrd.img-`uname -r` `uname -r`
```
Reboot one more time for good measure. Congratulations! You now have a Netra X1 running RAID1.

If you're experienced enough, you may be able to find a few ways to take shortcuts. The steps above aren't optimized. There are a few redundant steps, but I thought going through them would more clearly illustrate what I was doing. This entire operation could also be scripted, and if I had more systems to build, I probably would have done so.