- Performance The overhead for keeping track of what has changed and what hasn't was not designed for this usage scenario and is, apparently, hugely expensive. I do not have any hard numbers (unlike me) but I'd eyeball it in the 20-30% range. That's quite a bit. Sometimes it felt much worse than that!
- Reliability drbd is designed for use in an HA environment. However, I encountered numerous kernel crashes and other weirdnesses when it was put under heavy I/O. At one point, I had to *reboot* my file server 3 times in one day, normally the only time I reboot it is for a new kernel.
Monday, December 8, 2008
Network Block Device + MD RAID1 = Fun
For the last few years I've been making use of drbd to provide a sort of semi-connected network raid1 as part of my overall backup and disaster recovery system. Recently, I've been experimenting with using nbd (network block device) and Linux MD raid1 (with bitmaps) to provide a similar functionality, and have some interesting findings as a result. Essentially, drbd takes some sort of storage (any seekable file) on this machine and mirrors it over the network to another machine. drbd is primarily found in HA (high-availability) environments. I've been using drbd like this: I took a pair of 80G drives and placed them in two machines: the first machine is my local fileserver, and the second machine is a workstation that is booted occasionally (about once a day). I configured drbd to use each of these drives as a mirror of the other, and set the server as primary. Then I formatted the newly-available block device (/dev/drbd0 in my case) with ext3, mounted it, and used it for rsync+hardlink-style backups (now I'm using rdiff-backup). Whenever the workstation would come up, drbd would take note and only synchronize the blocks of the underlying storage device that had changed. This was very fast, easily saturating my 100mbit network and under non-jumbo-framed (standard 1500 byte) gig-e would sustain north of 15-20 MiB/s. Not bad. However, I had a few problems with drbd:
Posted by Jon at 5:45 PM