This morning at around 5:25 A.M., Dave sent me a message that he couldn't reach my website. It was up, since I could reach it on the LAN, but when I logged into a machine a Stanford, I was not able to ping www.anderoid.com. It turned out that our IP address, which as been constant for the last couple years or so, changed yesterday around 4:30 P.M.
They agreed to provide us with a static address a couple years back. I hope it won't change again any time soon.
We initially bought three Western Digital 120 GB PATA IDE drives for the Linux machine, and one of them failed after a few months, so we bought a couple Western Digital 120 GB SATA drives of the same size to add redundancy. One of them died several months ago, and another of them (or the replacement) died last month. They were all under warranty fortunately, so we replaced them for only the cost of shipping. Instead of giving us a new 120 GB drive, they provided a 160 GB.
Needless to say, I've been disappointed with the reliability of these Western Digital drives.
My brother bought me a new hard drive for my laptop for Christmas. The old one was whining too much, and particularly because I sleep with the laptop a foot or two from my head, the noise has been a nuisance for many months. It was a bit low on space too. I bought a regular (parallel) IDE to mini (laptop) IDE adapter on eBay for 5 cents plus $5 shipping and handling. I powered down the linuxbox and hooked up the drive to the adapter and the adapter to the IDE cable that was used by the CD ROM. I then booted the linuxbox and copied the contents of the old 12GB drive to a file using 'dd if=/dev/hdb of=/r/backup/seanslaptop'. After it finished, I powered down again and hooked up the new one, a 60GB Hitachi with fluid bearings. The old contents were copied to the new with 'dd if=/r/backup/seanslaptop of=/dev/hdb' and in fifteen minutes or so it was finished. I then installed the new drive in the laptop and it works great. It is very quiet; I can just hear the head seeks, but they don't bother me. To enjoy the added space, I used my dad's copy of Partition Magic version 8.0 to resize the NTFS partition from 12GB to 25GB. (I decided to leave room for another OS, just in case.)
Our uninterruptable power supply interrupted our power on June 21. The Belkin 1200VA UPS stopped providing juice to its protected sockets immediately after a power surge, which was due to a distant transformer blowing up (a frequent occurance these last few weeks, for some reason). I called Belkin and they sent out a replacement UPS via UPS, and I'm sending the defective one back to them. Their customer support was good, but I'm disappointed in the product for not better handling our erratic electricity.
Last week, I upgraded the to latest kernel (2.6.7-rc1) on our Linux machine (which is the file server, web server, etc). Then after a few hours it claimed two of the hard drives had suffered failures, so the software RAID drive was messed up. I rebooted into the old kernel and was luckily able to reconstruct the RAID drive. (The drives were no longer in a failed state and I didn't need to copy from backups.) I decided to try the new kernel again, so I booted it and mounted the RAID drive read-only. I smelled a burnt plastic odor while hooking up network cables to use both ethernet ports on the motherboard, and upon further investigation, noticed that the power supply fan was not turning and it was very hot to touch. This is bad, because it is the only case fan. An autopsy revealed that a couple diodes had burned out in the power supply electronics. We went to Fry's Electronics and picked up a quiet and efficient Seasonic Super Tornado power supply replacement for around $70. Things appear to be running OK, but I fear that the drive failures were actually due to the new kernel, and they will fail again. The old kernel has some security holes and doesn't utilize both NICs, so I really want the new one to work.
I installed the instant messaging client Gaim a few days ago. I had been using Yahoo's IM for several years, but I thought I would give Gaim a try because it can connect to numerous IM networks, including Yahoo's, ICQ's, MSN's, Jabber's, Gadu-Gadu's, Napster's, and Zephyr's. It runs on Windows and Linux. It features a spell-check add-on, aspell, which I can use.
I've noticed a few bugs on the Windows version, but it is fairly usable. For example, the window title bar stops flashing when new messages are received after a while. This can result in me not noticing that I have just been sent a message because I'm using another application and turn off the message received noise. Another failing is in not implementing Yahoo's buzz fully. Instead of making a ding-dong noise, it simply writes "Buzz!!!" to the IM chat window. I normally turn off all noise except for the ding dong, so that people can still reach me when I'm in the room but not staring at the screen. I hope they fix this. There appears to be some facility for writing plug-ins, so perhaps I'll do that if I'm desperate.
I finally got around to adding a spell checker. I installed ieSpell, which is a plug-in for MS Internet Explorer. An option is provided on the context menu and in the Tools menu to spell check the contents of a text box. It works reasonably well and is free for personal use. Another solution will be needed for Mozilla.
This morning I noticed that the clock on the linux computer was off by five minutes. I had not adjusted it in months, and I suppose it simply drifted that much. To prevent the problem from recurring, I installed a Network Time Protocol Daemon, which periodically queries remote computers for the time. They are known as "stratum 2" machines, and they are synchronized with "stratum 1" machines, which are synced to GPS satellites, atomic clocks, etc. The NTP daemon is able to measure the systematic drift in the clock and compensate for it; some temperature-dependent drift may still occur. The period of polling other computers is initially about 64 seconds and gradually increases to about 1024 seconds (or whatever you set it to). I understand the accuracy associated with using NTP is around 50ms.
My blog has been down. The reason is partly that the software RAID on which my directory sat was down, which is due to a temporary power outage knocking out two drives at once, and I’ve been busy with fixing things. (RAID level 5 can tolerate at most one drive going bad.) Both of the drives became out of sync with the third, though one of them was really ok physically. The other one apparently developed too many bad blocks on a track and the hardware bad block remapping couldn't deal with it, so it appeared to be in a failed state. The md software RAID package unfortunately can't deal with bad blocks on disks, unlike modern filesystems, such as ext2. Fortunately I was able to remove the data using a tool called mdadm, which can tell a disk that is out of sync to have the same event count as another, thus appearing to be in sync to md. For most (99.99999%+) of the data, this works fine, and I was able to tar everything up and copy it to our Windows XP machine. But that was only the beginning.
Firstly, I felt I needed a solution that would mostly eliminate the power instability, which plague us biannually, due to tree limbs falling on power lines during the high winds of Fall and Spring. We sometimes suffer brief brownouts, which leave electronic equipment in an unstable state, or power surges, which fry unprotected electronics, or blackouts that last hours. (The last year has been particularly bad -- we lost a high-end editing VCR, a bread machine, several surge protection strips, and a microwave.) So we bought a new Belkin 1200VA UPS (Uninterruptible Power Supply), which immediately supplies power to the networking equipment, Linux machine, and XP machine for 5-10 minutes after an outage begins, and then tells them to shutdown gracefully. It also has surge protection and helps filter the signal. It works great, and I wish we had one years ago.
Secondly, since bad blocks eventually build up in any working hard drive, a way of remapping the bad blocks was needed for our RAID setup. I believe I found a solution with EVMS (Enterprise Volume Management System), which is an open source project funded by IBM that has a bad-block-remapping "plug-in." It also sports a nice GUI, and provides many other features, such as snapshots, LVM, multipath setups, and clustering. The only drawback is that it hasn't been available very long, so it probably has a few bugs.
To complicate matters, the old system disk we were using was only 10 GB, and after putting on swap space, home directories, and tons of software packages, it was nearly full. I was growing tired of keeping the Redhat packages up to date, and dealing with the dependencies was a (slow and manual) pain. Perhaps if we paid them it would not be a nuissance, but that seemed unnecessary, since I had been reading that other distributions provided free and nearly automated solutions to keeping packages current. To top it off, Redhad decided to stop catering to desktop platforms and focus on their server market. So I wanted to give another distro a try. But first we needed more disk system disk space.
Our Asus A7n8x Deluxe motherboard can support two SATA (Serial Advanced Technology Attachment) drives, so we went to PC-Club in Bellevue and bought a couple 120GB Western Digital SATA drives (WD1200JD) for around $270. Serial-ATA is supposed to eventually replace the widespread Parallel-ATA IDE, and though the cables' diameters and connectors are smaller and more manageable, SATA currently isn't much faster.
After the drives were installed, I spent several days reading about different Linux distributions; I considered Suse, Mandrake, Knoppix, Debian, Linux from Scratch, Gentoo, and others. I decided on Gentoo, since the package management seemed well respected, there were many active developers, it's conducive to a better understanding of how things work "under the hood," and it is source code-based, allowing for custom compilation optimizations for the processor used and hacking of packages' code. If you have a slow processor, you may not appreciate the hours required to compile everything from scratch, but fast modern processors can make quick work of it, and future ones will be even faster.
There are many different linux kernels available for use with gentoo. Not all versions of the Linux kernel support SATA, unfortunately. The 2.6 kernel, which is still in beta testing, the -ac (Alan Cox) versions of the 2.4 kernel, and probably a few others support SATA. EVMS version 2, which has the bad block replacement, requires component called Device Manager, which is included in the 2.6 kernels but must be installed in 2.4 kernel versions. I didn't have much luck persuading the 2.4 kernel versions to work. Either they had SATA working or EVMS 2, but not both. Only the vanilla 2.6 kernel was readily able to handle both after a few patches. Finally, after several attempts, I have the 2.6 (beta 9) kernel and EVMS 2 working.
The five 120GB disks each have 800MB, 11GB, 50GB, and 50GB partitions. About 2.4GB of swap is spread across three of the 800MB partitions, while the other two are used as a boot partition and a backup boot partition. The OS is on one of the 11GB and backed up to another. Two 11GB partitions are combined as a fast 22GB RAID level 0 (striped) volume for scratch space that is not backed up. As for the 50 GB partitions, they are used to create two 200GB volumes, each with RAID level 5. The reason for two rather than a single large volume is to guard against filesystem corruption; files on one volume will be backed up to the other until I’m confident that the Reiser filesystem is stable. For all volumes except one of the big RAIDs and the boot partitions, I am using Reiser 3.6.8, which is fast but I don’t trust it, after reading about how others who have been burned with earlier versions of Reiser. The boot partitions are ext3, because I know that our bootloader, Grub, can deal with it. The other big RAID volume is formatted with JFS (IBM’s Journaling File System, ported from AIX), which is good for large files, much like XFS. It is still green, and not many people have used it yet, so I don’t trust it either.
At this point I'm feeling a combination of relief and paranoia. I'm happy with Gentoo and I'm glad to have things working, but if an IDE controller goes out, leaving the RAID partitions unsynchronized, I don't know if EVMS can deal with it without losing data, and the mdadm tool can't help.
I've spent the last several hours configuring our linux box to use MoveableType for creating weblogs. It uses PostgreSQL to store the entries, user info, etc, and that task took some time as well. MoveableType is basically html, images, and a few cgi Perl scripts, which are run by the Apache server. Overall, installing and configuring MoveableType wasn't that bad.