It really is amazing how easy it is to put together a system with a terabyte (or more) of storage. For less than $500 you can purchase four Maxtor 300 Gigabyte IDE drives ($118 each from New Egg), putting you well over one terabyte in capacity. If you are looking for something a bit bigger Apple has made that pretty easy with their Xserve RAID unit, up to 7 terabytes for $13,000. I believe hard drive capacity will continue to go up and price per terabyte will drop.
At work I’ve got few systems now with a terabyte or more of storage. While most modern operating systems support filesystems in the terabyte range, their advanced features don’t seem to work well. FreeBSD had no problem running with a terabyte plus filesystem (fs), but snapshots on that fs were pretty much useless because they took forever. Still having to run fsck when problems happen would undoubtedly be equally as painful. Fortunately there has been talk of adding journaling to FreeBSD, hopefully that work will at least take care of the fsck problem. General work on large file systems under FreeBSD is being done as part of project Big Disk.
You might think that a company with more money than many countries (Microsoft) would be able to produce an OS (Windows 2003 Server) that wouldn’t have those sorts of issues, but you’d be wrong. Windows 2003 does fine using a one terabyte plus fs, until want to use things like Shadow Copy and then things start to fall apart rather quickly. From what I’ve been able to determine things work fine for awhile and then shadow copy stops working at some point after the system has been up for awhile. That wouldn’t be so bad if that was the only thing that went wrong, but it isn’t. In Windows 2003 the built in backup software makes use of shadow copy, which makes sense, if shadow copy didn’t roll over and die on large file systems. When ntbackup tries to create a shadow copy and that fails, ntbackup stops and the whole back up fails. Obviously that makes it rather difficult to get a clean backup, at least using ntbackup. But it gets even worse, when ntbackup fails because shadow copy fails it erases all of the old successful snapshots that shadow copy made earlier.
After hunting around for awhile I came across Article 833167 at support.microsoft.com claiming that there is a hotfix that is supposed to fix the problem, but it isn’t publicly available. I have to contact Microsoft support and request the hotfix, after which they’ll determine if they are going to charge me for said support. At some point I’ll have to bite the bullet and ask for the hotfix, but I get the feeling they don’t have a lot of confidence in it yet. How long until SP2 for Windows 2003 comes out? What ever it is I doubt that I can wait that long.
Given how easy and cheap it is to equip a system with a terabyte (or more) of hard drive space it is disappointing that some things don’t work better on that sort of scale.