Disk Failure. Or Not.

I switched on my workstation this morning and was greeted with a ton of errors from one of the disks. Thank goodness for RAID! The obvious, initial conclusion was that the drive had failed. However, further examination showed that one of the three RAID1 devices shared across the drives was still working. While this condition is possible I suppose (I'm not exactly sure how), it cast doubt on the obvious conclusion.

I spent the next two hours looking for other possible hardware problems. The only other suspicious possibility was that the SATA adapter was pulled out about 1-2 mm of the slot on the unsecured end of the card. Again it's not enough to be conclusive. I ran the SMART tests on the disk and they passed. Unfortunately only failing these tests is considered conclusive. The RAID devices rebuilt successfully. This is a lengthy process and accesses the entire drive, so it should have triggered any hard failures.

In the end, the safest thing to do is buy a spare drive. Just in case.

Don't You Just hate It...

Got an email from a customer yesterday describing a hanging application, a database client. They were able to replicate the problem on two computers. When I went to investigate today, guess what? Yes of course, the problem has completely vanished and is not reproducible at all. I know something went wrong yesterday because there are errors in the PostgreSQL logs, but there's nothing that leads to the next clue. Dead end.

Don't you just hate it when computer problems mysteriously fix themselves? You just know they are going to come back later.