1/20/09

The adventure of (re)installing linux on Tyan s5150 with rocketraid 2220 and Hitachi Deskstar T7K250

More than 3 years ago, I built a backup server for others to use. It's set up with a Tyan S5150 motherboard with Pentium 630 (3.0Ghz), Highpoint RocketRaid 2220 (rr2220), 7 SATA drives in a 3U rackmount Antec case. I don't remember how much it costed me, but it's sure more than $2000 (hard disks alone was $1000).

Recently the server was freed up. Even though the hard disks are real cheap now (the cheapest 1T drive is $94 now at newegg, Seagate 7200.11 1.5T drive is selling for $129) I hate to see those hard-drives idle, so I brought it home and intended to build a file server.
5 of the 7 SATA drives are 250GB Hitachi Deskstar T7k250. These drives are from the same batch, all produced in July 2005. The problem with these drives is that they can not be used by any other computers. Either bios doesn't read these disks or the it read as 0MB and can not use them. I also have 3 USB SATA hardware enclosures, none of those disks work with any of the enclosures. These are the only SATA drives I have that have this problem (I have a lot of drives). What's strage is that the same 250G T7K250 drives that produced a month later (August 2005) have no such problem.
So I have to use rr2200 (which is a pci-x card) for these drives, which means I have to use Tyan S5150 board because it has a PCI-X slot. As I try to install the board, I found I have to use a EPS12V power-supply. But all I have is ATX. So i pulled EPS12V PS from the Antec case (I can't use the case, it's like a helicopter when it's turned on). Basically the whole server got re-used except the metal shell.

The server used to have Fedora Core 4 on it. I remembered it took a lot of time for me to install and configure it. But unfortunately I didn't document what I did. Huge mistake!! Now the problem only begins.

First of all, With the raid card rr2220 inserted, and a raid array configured. The server can't even boot to the point it trying to load OS (any OS on any media, hard disk or CD or USB).
It only boot to an OS after I deleted the raid array, leaving the 5 disks unconfigured. Still, I can not boot Centos, Ubuntu installation CD. I can only boot with Fedora 10 CD, and only after adding "edd=off" to boot line.
It take a long time to get to welcome screen, at times it appeared stalled. But tty4 shows it's some disk error, and still trying.
During the configuring the installation, this show up.
You bet it's a BUG! The installation aborted.
This bug is caused by conflicting drivers sata_mv and tg3. You may wonder why a sata driver (sata_mv) conflict with a ethernet card driver (tg3), beats me.

Only after I pulled out the rr2220 can I successfully install fedora 10. So I did the installation. Boot the new OS, blacklisted sata_mv, compiled and installed so-called "open-source" driver from Highpoint. Did a modprobe hptmv6, it shows the driver loaded succesfuly and it saw 5 drives.
But I can not use them. Because somehow they can not be used a individual disks, They can only be used as raid arrays. So I went ahead and configured RAID arrays. But lo and behold, I can NOT boot again. I tried a GRUB CD, it doesn't boot either. I do have spare floppy drive, but I remember I tried that 3 years ago it didn't work so I didn't bother.

So I did some research (i.e. google) and it seems the problem is that rr2220 was trying to be first and only boot device no matter what you told motherboard bios. The problem can only be corrected by change in rr2220 bios. The change requires disabling INT13 and/or EBDA reallocation.
But the Linux bios update utility from Highpoint have no such options, only with their windows version can you make this change. What could I do?

Running out of options, I installed windows xp (after un-insert rr2220 of course), download flash utility and new bios, re-insert rr2220, reboot, run the utility, check "disable INT13" and "disable Allocating EBDA " and "disable stop or error", then flashed the bios.
Yes, I wasted an hour just to install Windows and run Highpoint bios flashing utility.

Now it's Fedora 10 installation all over again. Pulled out raid card, install fedora 10, boot to fedora 10, blacklist "sata_mv", download and built and install "open-source" rr2220 driver, install driver. Shutdown, re-insert Raid card, boot, create raid arrays, continue boot, fdisk, mkfs, mount, install and configure Samba.


Finally my file server is set up. It has 1.2 Tera Byte of storage. Not much, but I got to use the 5 disks that otherwise can't be used. Besides, it has room to expand. RocketRaid 2220 is an 8 port SATA2 card. Tyan motherboard itself has 4 SATA ports. So in theory, at today's 1.5T per drive, I can have 1.5*12 = 18 Tera Byte of storage (at 5M/song, it's 3 million MP3 songs, or at 6G/dvd it's 3000 DVD movies).
When the price on 1.5T drive drops to sub-$50 range, I will get some of those (assuming per GB SSD price is still much higher then).