For some time now, I've been frequenting a small bicycle/coffee shop, The Uphill Grind in Cross Plains, a small town West of Madison and Middleton.
I've tried the coffee at the "other" coffee shop in Cross Plains, and was not impressed. The stuff available at The Uphill Grind, though, is something worth driving all the way out to Cross Plains for (ok, so I'm only 10 minutes away but some of you aren't...).
The coffee (as reported by every single person I've dragged out there) is really exceptional, and the host (Michael) is friendly, knowledgeable, and makes some seriously strong stuff, if that's the way you like it. I've become addicted to caramel espressos.
I also believe pretty strongly in supporting small businesses, so not only do I frequent his coffee selection, but I've also had some service done on my bicycle. In case you didn't know this, Cross Plains is smack in the middle of what is basically becoming Biking Mecca. The scenery and environs are phenominal, and shops like the Uphill Grind are very active in helping out at events and such.
Consider checking them out - the coffee alone is worth it.
Monday, October 12, 2009
Thursday, September 3, 2009
Lost Print Jobs Issue Mysteriously Resolves Itself
Previously, I blogged about a friend's situation where print jobs appeared to go off into the ether. I inquired about the situation recently and he said that it mysteriously resolved itself.
Weird.
Weird.
Sunday, August 2, 2009
HP Quality is really slipping
Over the years, I've had a bunch of printers. My first printer (which I still have, and it still works) is an Apple ImageWriter II. A dot matrix printer. My second printer was an HP DeskJet 520. It worked (but was noisy) when I got rid of it a year or so back. I've had a slew of Epson inkjets (clogged heads doomed them all), a bunch of HP Inkjets (various issues), and some others.
For the longest time, HP used to be the go-to company for quality inkjets. Canon never used to have very good Linux support. Epson had reasonable (but not great) Linux support, but their printer design (permanent heads) doomed every one of them to mediocre or troublesome printing with a few years. HP, however, had excellent, early Linux support and their for some of their printers, the print heads are built into the cartridges - this makes fixing a print head issue as easy as replacing the cartridge or cartridges. I'd love to give Kodak a try but their refusal to provide any sort of Linux support - at all - not even basic drivers - puts them out of the running. I won't even talk about Lexmark's awful injket printers.
However, the overall quality of HP printers over the last few years has really left me wondering what happened to this once-great "engineeer's" company. A failed LCD panel doomed one printer - rendering it totally worthless for anything but basic printing - and a google search revealed this to be a very common problem on that series. Another printer (DJ6540) I have occasionally loses its mind and needs to be hard reset. This officejet is a higher-end inkjet, certainly, and I expected no problems. By and large that has been the case. However, driver installation on Windows has proven to be a complete and total disaster. I had to reinstall windows to get the drivers to install - sorta - and their firmware updater absolutely refuses to upgrade a printer unless it is configured in windows as a printer. This, despite the fact that the firmware updater clearly tries and appears to support upgrading over TCP/IP. Did anybody test this thing? How did this ever get released to consumers? A google search also reveals I'm not alone. Driver problems, driver installer failures, and so on. I expected more out you, HP, and I'm not only disappointed I guarantee you it's going to influence the advice I give and the buying decisions I make in the future. I will be trying your competitors, but sadly I suspect that in general printing is a race to the bottom.
Failed Diagnosis of Lost Print Jobs
A friend of mine came to me recently and told me that, for some reason, his printer wasn't working any more. I set this friend up with an openSUSE home server for printing, DNS, DHCP, and internet access (via qtsmppd) and by and large it works great. The printing is managed by CUPS, and the printer is an HP OfficeJet Pro L7680. It's a nice printer with Vivera inks. It is fast and generally works great. I have it set up to print over port 9100.
I asked him to bring the machine to me and he did so. When I examined the logs, there was nothing amiss - according to the CUPS logs, jobs were queued, accepted by the printer, and when complete removed -- exactly as expected. I set an identical printer up in exactly the same way, but left it off. I queued a print job and, as expected, CUPS indicated that it could not contact the printer and simply kept retrying the job. When I turned the printer on, it began accepting jobs and the job printed out just fine. No matter what I did, I could not get CUPS to lose the jobs. How do I reconcile this with the earlier behavior of "I printed a bunch of stuff but nothing ever came out of the printer". I may need to see how things work in the exact environment the server is in to find out.
Monday, July 27, 2009
Crazy Storm
Thursday, June 25, 2009
KDE 4.2 and KDE 4.3
It's been a while since I've commented on KDE. Partly, it's because I've been insanely busy with life and work, but I'm pleased to report that somewhere around 4.2.2 KDE started getting pretty usable. I ran 4.2.4 for a while before upgrading to one of the 4.3 betas, and other than one weird issue, it's actually be quite pleasant to use. By and large everything works, there are still some obnoxious issues (like the konsole issues with -e) and so on, but overall it's really coming together and is, dare I say it, pretty attractive.
I'm not much for eye candy that doesn't actually make my day easier. It's not like I run fvwm or anything (although I like fvwm and have a nice config which only took me a WEEK to make), but while some advanced eye candy is actually pretty useful (it draws your eye here or it helps you remember where something went, etc...) most of the eye candy is just that - it's candy. Fun once in a while, but not very good for you. And it rots your teeth.
So KDE 4.3 is faster, a bit leaner, a bunch noisier (have you ever tailed .xsession?), but overall pretty reliable. Few crashes. Bluetooth doesn't work. The network manager integration is just getting to the point where I use it instead of knetworkmanager (which always worked very well thank you). Plasma does crash now and again, but now when it does so it's more of a surprise than a regular occurance, and so far it's not caused me to lose work.
I do have one really weird issue, so weird that I can't even figure out how to file a bug report. When 4.3RC1 becomes available over the next few days, I hope that fixes it.
Am I looking forward to openSUSE 11.2? Oh yeah. I did grab the dvd .iso and play with it in qemu, and it boots fast, and since it'll have KDE 4.3 that's all the more reason to get excited...
hpcups broke my printing
I had a thoroughly impressive rant goin' on here but I just don't have the energy to type it all out. Suffice it to say that hplip-hpcups suffers from some fairly basic forgot-to-catch-the-errorcode problems, and it ate up an hour or more of my life.
As a friend says, I'm entertaining when I have my indignation all worked up, but it's better in person.
Wednesday, March 4, 2009
Adobe, Taxes, and Government
My State Government and especially Adobe have annoyed me. For reasons that escape me, the Wisconsin e-File Form-1, Form-1A, and Form-WIZ, all located at the Wisconsin Department of Revenue e-file page here, all require Adobe PDF Reader version 9. I ask: Why?
For those that don't know, I run Linux and only Linux, and Adobe PDF Reader 9 is not available for Linux. Even if it were, I'm not sure I would use it. It's a horrible, bloated, insecure bit of software. It also comes with "for free" Adobe AIR. I don't want Adobe AIR. I want to view PDFs. Do I have to install Adobe AIR to view PDFs, too? Adobe PDF Reader 9 and Adobe AIR are not Open Source, and so I am loathe to give them any quarter.
Adobe has a decidedly anti-Linux stance, in my opinion - apparently they feel there is no market for a 64-bit version of Flash (for Linux), and the other solutions (like nspluginwrapper), no matter how well-intentioned or implemented, still can't cure the problem of trying to use a 64-bit browser and a 32-bit plugin. I use the 32-bit Firefox just so that flash doesn't crash literally every other time I use it. I still dislike the use of flash in any case, as I have no control whatsoever over how it operates, interacts with my system, and so on. It seems like once a month there is a security problem in Flash!
The PDF format itself is supposed to be reasonably open, but if that's true then why can't I use okular, kpdf, evince, or xpdf to view and use the PDFs as provided by my own state government?
There are several issues here. The first issue is that my state government is requiring the use of proprietary software to use e-File when I see no reason for that requirement. Open Formats and USABLE documents should be a legal requirement for governments! The Open Office document standard should be used, and when something like PDF appears necessary, at least use a version that works on more than the very-latest install of proprietary software.
My guess is that there is a bit of javascript or whatever inside the PDF which tries to identify the viewer and simply bails if it's not Adobe PDF Reader 9. The actual text is:
To view the full contents of this document, you need a later version of the PDF viewer. You can upgrade to the latest version of Adobe Reader from www.adobe.com/products/acrobat/readstep2.html For further support, go to www.adobe.com/support/products/acrreader.html
In light of these recent Adobe PDF view issues, the recent (and seemingly rather serious) Adobe PDF security issues, the recent Adobe Flash issues, and so on I'm wondering if people aren't going to start to flock to the reputedly vastly better architected Silverlight by, shudder, Microsoft, a company with it's own long and glorious history of supporting open standards. It would appear, however, that Silverlight (for now) is an exception, as there is a reasonably free and open implementation with Moonlight. Given what I've seen so far, I'm starting to give hope that Moonlight starts to put the hurt on Flash, or that Adobe truly opens the Flash *format* so that truly open and free implementations can begin to flourish.
The most remarkable thing about free software like Linux, BSD, KDE, and others is that once people use it, they stay with it not necessarily because it works better or looks better or is more reliable - I think it comes down to choice. People *like* having choice and more and more often people are recognizing at some level things like that vendor lock-in (Adobe, Microsoft) grate against people in very fundamental if non-obvious ways. People *don't* like being told what to do, what to run, what they can and can't do.
UPDATE: at least part of the issue has to do with Okular (via poppler's) incomplete support for javascript. The Adobe spec for the javascript support is reported to be some 700 pages long. So at least part of the problem lies there. However, does it *really* require Adobe PDF Viewer 9 or would 8 suffice? Javascript appeared in version 7, if I recall.
Tuesday, March 3, 2009
6 bladed razor. ugh.
Sadly, it has happened.
The 6-bladed razor.
Ugh.
Blog entry from a blog entry.
6-bladed razor
Sunday, March 1, 2009
Getting suspend-to-RAM working on my Lenovo T61p
I spent a bit of time this weekend trying to get suspend-to-ram (S3) working on my work-issued laptop. It works great. However, there are (as usual) some issues.
The first issue is that it I cannot use s2ram when I'm running X, because for whatever reason s2ram with the right options works great outside of X but when the NVidia driver is in use there is a conflict of sorts (I believe it has to do with POSTing the card after return from suspending). I have to use the NVidia driver because vesa is limited in geometry and the nv driver misdetects my display badly. I'll note that until openSUSE 11.1 the nv driver didn't work *at all*, just a hung laptop, so this is progress and I'm glad for it!
In any case, I had to force pm-utils not to use s2ram at all by removing /usr/sbin/s2ram. One package or another pulls in suspend (the package containing /usr/sbin/s2ram) as a dependancy and the pm-utils "functions" script checks to see if it exists and is executable - using it if possible. I don't want it to use s2ram so I simply removed /usr/sbin/s2ram. I filed a bug for a more elegant solution, which allows a config variable to simply specify "use the in-kernel method always".
As an aside, for those that might find this useful, using s2ram without the NVidia driver active works great with: --vbe_mode --vbe_post --acpi_sleep 2
With the NVidia driver active, using the "in-kernel" S3 mode works great:
echo -n "mem" > /sys/power/state
I ran into issues with the above failing if ksysguardd is running (probably related to ksysguardd checking the temp of my CPUs) - suspending simply fails. I've worked around this by using a script in /etc/pm/sleep.d to SIGSTOP ksysgardd and the SIGCONT upon wakeup, and this appears to work but it hasn't received much testing.
One thing that complicated things is the lack of documentation on where to configure things and what to put there. After I had determined that s2ram (with the aforementioned options) works fine when X is not running, I tried to make it work in X. pm-utils sometimes pulls it's options from s2ram, sometimes from /etc/pm/config.d/*, and other times from HAL. HAL was configured suboptimally for my laptop (a Lenovo T61p) so, but ultimately I determined that using s2ram wasn't going to work no matter what options I sent along, provided I was using the NVidia driver. Thus, the HAL config for my laptop (/usr/share/hal/fdi/information/10freedesktop/20-video-quirk-pm-lenovo.fdi) didn't get used at all, as far as I can tell.
If you want the HAL config it looks like this, more-or-less replicating the above s2ram config. THIS DOES NOT WORK with NVidia drivers loaded.
<match key="system.hardware.product" prefix_outof="6460">
<merge key="power_management.quirk.s3_bios" type="bool">false</merge>
<merge key="power_management.quirk.vbe_post" type="bool">true</merge>
<merge key="power_management.quirk.s3_mode" type="bool">true</merge>
<merge key="power_management.quirk.vbemode_restore" type="bool">true</merge>
</match>
The best documentation I found was pm-utils' documentation which detailed (confusingly, however) the origination for each decision and config. Expanding and improving that document, as well as promoting it as the de-facto source of information, might have saved me a bunch of time.
Next up: getting my laptop's video mode switching to work.
Saturday, February 21, 2009
openSUSE 11.1 - 32bit -> 64bit upgrade not so smooth
I recently upgraded a 64 bit machine that was running a 32 bit version of openSUSE 10.3. I upgraded it to 11.1. Yes, I know this is a jump of two versions. The update didn't entirely hose the machine, but tons of stuff was left as i586 instead of x86_64, including a bunch of pam stuff. Thus, ssh did not work. LOTS of stuff didn't work. Like half of YaST2. Once I was able to get in, a 'zypper dup' sorted out *some* of the problems, but not all of them.
As I write this I'm still in the process of un-hosing the machine...
UPDATE: it's basically unhosed. kudateapplet was missing a dependancy on kdelibs4 (or perhaps one of it's dependencies. Oh well. It's working great now!
KDE 4.2 coming along, still not there
As I mentioned earlier, I gave KDE 4.2 a try and it was a disaster. Since then, the team responsible have been producing almost nightly new releases, and these new releases have improved things considerably.
Specifically, konsole, while still not without it's issues, is very usable now. The taskbar is usable, and the desktop area is usable. Assuming you turn desktop effects off, things generally work OK - I get 2 or 3 crashes a day. Some of these crashes are pretty annoying. krunner keeps getting hung in futex, which indicates some sort of threading problem. When that happens, alt-f2 (something I use quite frequently) goes belly up (as alt-f2 runs krunner), and for some reason all of dbus starts acting weird, negatively impacting many apps including gtk+ and most of KDE.
One of the big issues I have with konsole is the auto-saving of it's window sizes. KDE 3.5 used to allow you to save the dimensions of the window (as well as other changes you've made to the session) in an on-demand fashion. However, KDE 4.2's konsole does this more or less every time you make a change. I open hundreds of konsoles a day, easily, as some of them are very short-lived. I really, really want konsole to open to the same size every single time. I resize many of konsole instances to sizes that are totally unsuitable for other purposes. Ugh! It also took me a few days to find out how to disable the "start konsole in the same working directory" when creating new tabs. That's also hugely annoying. I've found a BUNCH of bugs in konsole and won't detail them all here, but konsole is KDE 4's biggest step backwards yet.
The next issue is the seemingly more haphazard way in which the system settings are arranged, compared to KDE 3.5. I sometimes have to look in 2 or 3 different modules to find what I'm looking for, especially when it has to do with themes or visuals or what-not.
Overall, however, KDE 4.2 is coming along and I sincerely hope and believe that these issues will get resolved - I suddenly have hope for the future of KDE again. Good job, folks!
Saturday, January 31, 2009
KDE 4.2
I recently gave KDE 4.2 a try, thanks to the good folks at Novell. The install went fine, and it wasn't nearly as crashy as it used to be - I "only" got one crash of the window manager. By and large it works better, but it's still not nearly as usable as 3.X was for me. There is no way to highlight a program in the main menu and put it on the task bar. Moving things on the task bar as yet eludes me. The "no icons on your desktop" decision is not just dictatorial but it's painful for users and I see no merit in it whatsoever. Konsole *still* doesn't display fonts worth a damn, when configuring fonts the "example" display looks great but the *actual* display still looks terrible. At startup, there was a short-lived message about the size of my screen being undetectable, at which point it looked like it tried to start a program but failed, didn't retry, and didn't tell me anything about what went wrong. Nice. The visual effects are nice, but I could care less if I spent more time, effort, and energy getting my job done. fvwm2 is looking better every day.
KDE 3.X has worked extremely well for me over the years - solid, consistent, predictable, and most of all *functional*. Maybe KDE 4.3 will get start to get good enough to use for me, but I'm getting pretty sick of the dictatorial attitude that KDE has taken on lately - one of the biggest reasons I stopped using GNOME was because of these reasons - crashy software and dictatorial attitude. Being "pretty" doesn't mean a damn thing if I can't use the software effectively.
Wednesday, January 7, 2009
openSUSE 11.0 -> 11.1 Upgrade Issues
I recently upgraded a number of machines to openSUSE 11.1 from openSUSE 11.0. By and large the upgrade went perfectly, however, a number of issues cropped up almost immediately. The first was that I could not log in.
I use an encrypted home directory via pam_mount. The pam options for various programs got good and hosed by the upgrade, and openSUSE's response to the bug I (and others) filed was "you should always check your .rpmsave files after an upgrade". Lame. I might be experienced enough to check /etc/pam.d/* for changes but the overwhelming majority of folks aren't. If I had used the (easy) system tools (YaST) to create an encrypted home directory in 10.3 or 11.0 and then had it stop working in 11.1, and been unable to fix it, I'd have been royally cranked. My various attempts at fixing the problem also resulted in a number of bugs filed.The next problem, and a far more serious one, was repeated filesystem corruption the same volume. Coincidence? I think not. I use ext3, typically with journal=data, and haven't had filesystem corruption of any kind since about 2000, and it was a rarity even then. Further investigation showed me that while my home directory was being mounted correctly, it wasn't being *un* mounted correctly. Furthermore, subsequent logins re-did the loopback and dm-crypt setup so I had 3 or 4 or 5 mappings, only the last of which was any good. This resulted in some lost work for me, an hour or two above and beyond what I had lost as part of the investigation. I filed another bug on that and it's yet to receive any attention after almost two weeks. Look, I know the developers are busy, but releasing a new operating system around the Holidays when lots of folks are going to be one, or taking, or recovering from vacation doesn't strike me as a great idea. Still, anybody with encrypted home directories mounted via pam_mount might be experiencing the same problems and THAT doesn't make for good PR - "openSUSE 11.1 hosed my home directory" is not a headline I would like to read, if I worked for Novell.
So I set about to determine if a newer version of pam_mount could help solve my problem, since rolling back to the older version didn't. I built newer versions of libHX and pam_mount (which has a thoroughly scary changelog, although one must remember the great complexity it must deal with) using the openSUSE Build Service (which really is a truly awesome and wonderful thing!). That didn't work (no amount of futzing could cajole it to mount my home directory), so I went ahead with the latest versions of each and sat down for a longer debugging session. Armed with diff and lots of other stuff, I basically determined that the latest version will work just fine - it correctly avoids duplicate mounts and deals better with unmounting and so on and so forth - but the trick is two-fold:- You have to basically remove almost everything from /etc/security/pam_mount.conf.xml except for the <volume> elements and, of course, the outer element.
- You also have to fill out options which didn't need to be specified with 0.35 (or 0.47) because the values were the same as the defaults, but the defaults have changed. Those options are:
- fskeycipher - use aes-256-cbc here as it's the (former?) default
- fskeyhash - use md5 here as it is the former default
I hope this post serves three purposes:
- To help some other poor schmuck out of the near-disaster that was *my* upgrade experience
- Hopefully to cajole Novell into giving a *bit* more thought into how others might react to losing access to their $HOME
- To rant (this is the biggest one).
Some updates:
The bugs I filed did get some attention! The verdict: from a clean 11.0 upgraded to 11.1 the bugs to not appear. I went so far as to reinstall the stock pam_mount version, try stuff, and back and forth. No such luck. However, since my workstation is now working again, at least I'm not so cranky. Someday if I get some ambition I'll try to find out what is going wrong here, but I got some help from Novell and that's more than I deserved!
Thursday, January 1, 2009
KDE4 > KDE3? Not so much.
A recent upgrade to openSUSE 11.1 afforded me the opportunity to give KDE4 a try again. I'm a long-time fan of KDE3 and was hoping for pleasant surprises. I got half my wish. KDE4 may be a bit prettier in places (but not the fonts), but it's neither as consistent or as usable as KDE3 was. It's certainly not as stable or as fast (even with desktop effects turned off), and there have been a bunch of non-obvious changes. To me, it feels like a couple of steps backwards in experience and utility.
In particular, konsole is a BIG step backwards - if you open up a konsole with a given command (like ssh SOMEHOST), then new tabs will *also* execute that command (something KDE3's konsole did not do - quite logically). If you hit the "close tab" button (which I to root around in preferences to find) konsole hangs and has to be killed. Sweet! There are a fraction of the preferences available, some things I found quite useful are now gone.
Replacing konqueror with dolphin as the default file browser was not a step forward, either. The inability to right-click on a menu item (in an application menu) and add it to the bar was removed - so how do I add firefox, konsole, and the 2 or 3 other apps I open up 400 times a day? It's not clear to me and if it's not clear to me then, IMO, it's a step backwards. How do I configure my desktops so I can easily mouse between them by dragging windows or even just moving my cursors across the edges? I don't see it anywhere!
So far, KDE4 has removed features I use, added ones I find annoying, hurt stability and speed, and impeded my workflow. I'm sure there are good reasons for KDE4, especially architectural ones, but so far my experience does not bear out any advantage, and some significant disadvantages.
You may say to yourself, "you just have to know where to look" but if I wanted that sort of opaqueness I'd run GNOME, or at least older versions of GNOME - maybe they've improved some. KDE3 was a very *usable*, *stable* desktop environment and so far KDE4 is worse in nearly every aspect I care about. It's too bad because I know a great deal of work has gone into it, but I'm really beginning to wonder if that work has been with the right goals and priority management in mind.
Saturday, December 13, 2008
NFSv4 on openSUSE 11.0
For the past decade or so I've been using NFS inside my home network. NFS is one of those things that has a low-enough barrier to entry that it's really easy to get going, but it's opaque and esoteric enough that when things go wrong it can be a profoundly unreal experience. NFS has many weaknesses, however, among them security (there isn't any), performance (it's not as good as it should be, even if it's not BAD), and complexity (lots of ports, portmapper, choice of UDP/TCP, and so on), and something called "cache coherency" which is essentially a tradeoff between performance and correct behavior. NFSv4 goes a long, long way to fixing most of these problems. However, it brings some new behaviors and requirements which cause some headache.
First, the good: NFSv3 operates over what seems like a half-dozen or more ports. Those ports are frequently dynamically assigned, which means that you get to involve portmapper. Portmapper's job is basically to say "nfs is on X, lockd is on Y, and statd is on Z" and so on. It also complicates firewall management, and can cause all manner of headaches. NFSv4 operates over ONE PORT - 2049. Excepting gss (security: authentication and authorization) which has it's own ports, id mapping which may operate over LDAP or NIS or whatever, the core NFSv4 protocol operates entirely over one port and uses TCP. That's awesome!
More good: NFSv4 has better cache coherency, locking, a better transport, and so on.
The bad: The exports file format has changed very slightly, and *how* filesystems are exported has changed a bunch. NFSv4 exports a single "root" filesystem under which all others may or may not show up. This root filesystem is identified in /etc/exports with "fsid=0" (or fsid=root). To save time, let's assume that you are going to place this root on your server at /exports. On your server, "mkdir /exports". Despite the documentation suggesting that symlinks could be used to expose filesystems that are not rooted in /exports, that's not true. On linux, you really only have three choices:
- Move/Copy the contents to the exported directory
- Use bind mounts (mkdir -p /exports/pictures-of-cheese && mount -o bind /pictures-of-cheese /exports/pictures-of-cheese)
- Mount the filesystem directly: mount /dev/sdb1 /the-hoff
Once you've done this, you can fiddle your /etc/exports file. Mine looks like this:
/exports 192.168.100.0/24(fsid=0,insecure,no_subtree_check) /exports/the-hoff 192.168.100.0/24(rw,nohide,insecure,no_subtree_check,root_squash)
OK?
Set:
NFS4_SUPPORT="yes" in /etc/sysconfig/nfs
and restart portmapper and NFS.
Setting up your client to use NFSv4 is done the same way, by editing /etc/sysconfig/nfs. Do so and restart portmapper and nfs.
Mount the filesystem on the client:
mkdir /nfs mount 192.168.100.1:/ /nfs -t nfs4
If that doesn't work, I can't help you. Note that unlike NFSv3 we are mounting the *root* filesystem. If you are used to mounting /the-hoff (hah), just use a symlink to point into /nfs/the-hoff. Make sure it shows up before you continue.
Now the fun begins. id mapping. NFSv3 using AUTH_SYS (what almost everybody was using) worked like this: ids were used to identify users and groups. That's basically it. A user "bob" with id 1000 on the client would map 100% to user "sally" on the server if sally had id 1000. Names were irrelevant.
With NFSv4 that's no longer the case. Names are important. And to make things awesome, you can't use the old behavior, and if a name doesn't match it's automatically set to the "guest" user or group, the value of which is set in /etc/idmapd.conf - let me tell you - on machines which have slightly different /etc/passwd that's a fun one. It's an argument for LDAP or some other centralized directory, but it's still a pain.
So now I've got NFSv4 up and running. Why doesn't anything work? The default debug level of idmapd.conf is 0. I set it to 999 and got just a bit more noise in the logs, helping me to figure all of this out.
Otherwise it seems to be working OK but not great. It's working better than CIFS on 2.6.25.18 - while testing rdiff-backup (to a CIFS mount) last night I got the kernel to oops 4 times - 4 reboots in 15 minutes does not make a stable filesystem. I tried 2.6.27.something and it worked much better, but given the long-standing locking issues with CIFS I'm not about to switch to it. Don't believe me? Google for 'cifs' and 'sqlite'. Remember, everybody and their brother is now using sqlite, firefox and xbmc two examples I can think of right away.
And now the post is done.
Monday, December 8, 2008
NBD performance enhancements
As I blogged about earlier, I've been making use of NBD lately as my block-device-over-network exporter of choice (previously AoE). I got pretty reasonable performance out of NBD, easily in the 10s of MB/s - basically the limit of my iffy network hardware, however I wanted to know if there was more to be found. And there is! nbd-server, for one, uses the following basic paradigm:
- read data from network
- process that data into instructions
- do the instructions say to read data? If so, lseek to the right place on disk, read data into a buffer, copy a reply (header) plus the data into another buffer, and send that whole buffer back to the client.
- do the instructions say to write data? If so, read more data from the network into a buffer, lseek to the right place on disk, and write it. Construct a reply and send that back.
- Other instructions processed here....
Network Block Device + MD RAID1 = Fun
For the last few years I've been making use of drbd to provide a sort of semi-connected network raid1 as part of my overall backup and disaster recovery system. Recently, I've been experimenting with using nbd (network block device) and Linux MD raid1 (with bitmaps) to provide a similar functionality, and have some interesting findings as a result.
Essentially, drbd takes some sort of storage (any seekable file) on this machine and mirrors it over the network to another machine. drbd is primarily found in HA (high-availability) environments. I've been using drbd like this: I took a pair of 80G drives and placed them in two machines: the first machine is my local fileserver, and the second machine is a workstation that is booted occasionally (about once a day). I configured drbd to use each of these drives as a mirror of the other, and set the server as primary. Then I formatted the newly-available block device (/dev/drbd0 in my case) with ext3, mounted it, and used it for rsync+hardlink-style backups (now I'm using rdiff-backup). Whenever the workstation would come up, drbd would take note and only synchronize the blocks of the underlying storage device that had changed. This was very fast, easily saturating my 100mbit network and under non-jumbo-framed (standard 1500 byte) gig-e would sustain north of 15-20 MiB/s. Not bad.
However, I had a few problems with drbd:
- Performance The overhead for keeping track of what has changed and what hasn't was not designed for this usage scenario and is, apparently, hugely expensive. I do not have any hard numbers (unlike me) but I'd eyeball it in the 20-30% range. That's quite a bit. Sometimes it felt much worse than that!
- Reliability drbd is designed for use in an HA environment. However, I encountered numerous kernel crashes and other weirdnesses when it was put under heavy I/O. At one point, I had to *reboot* my file server 3 times in one day, normally the only time I reboot it is for a new kernel.
Friday, November 14, 2008
Nearest Book Meme
Meme from http://pradeepgowda.com/blog/2008/11/12 and http://www.openhex.org/blogs/nicoe/200811130741_Phrase_from_nearest_book_meme
- Grab the nearest book
- Open it to page 56
- Find the fifth sentence
- Post this sentence on your blog along with these instructions
- Don't dig your favourite book, the cool or intellectual one: the closest.
Here is mine:
"Where is it written in the Constitution," he demanded, "in what article or section is it contained, that you may take children from their parents, and parents from their children, and compel them to fight the battles of any war, in which the folly or wickedness of Government may engage it?" - Ron Paul quoting Daniel Webster, The Revolution
Thursday, October 30, 2008
Monday, October 6, 2008
More Notes on Acer 7720 Wireless
Today I helped some friends update their Acer 7720 laptop. It's running openSUSE 10.3 and one of the updates is a new kernel version, probably the riskiest of the periodic updates. Unfortunately, upon reboot, the wireless was shot. I spent an hour with their laptop until I hit upon just the right google which said something about the ipw drivers. Now, I had already removed all of the ipw* stuff as I remembered the laptop works with the iwl not the ipw series. I *thought* I had removed all of it but I had forgotten the most important bit, the firmware. Once that was removed and the machine rebooted, everything worked perfectly. Yay!
Lessons:
- never update the kernel when you have to be able to use the machine shortly after that, especially if you are using wireless or something equally finicky
- be thorough with the axe - if I had removed the all of the right stuff, right away, the "issue" would have taken 5 minutes to resolve.
- Remove EVERYTHING ipw*
Fin.
Thursday, September 4, 2008
Why Does Printing on UNIX Suck So Much?
It's 2008 and printing on Linux (UNIX) still sucks. What follows is a serious grump about printing on UNIX, CUPS in particular, and why photo printing at home is still insane.
CUPS has brought UNIX printing into the 21st Century but it's still cumbersome, opaque, and buggy. Only the very latest not-even-released-yet code has a USB driver that doesn't gack all over the floor most of the time. Have you ever straced the usb printing driver? My CPU was spun all the way up because the usb driver, apparently, doesn't know that when read(2) returns 0 (zero), it means (for streams) "don't bother trying anymore, there is no more data here. Ever." It's been doing that for years.
It seems as though, no matter what program I use: gwenview, gthumb, the gimp, or anything else I can't quite get what I want. Trying printing a 4x6 without tearing out all of your hair. I have yet to get a single 4x6 to print which wasn't resized to some much-smaller-than-4x6 size, or have something else really wrong with it.
How about the fact that every time CUPS is released I have to, typically, completely blow away the old configuration and start over? Upgrading from each previous version of CUPS has been painful to say the least.
I've been doing this for almost 15 years and I've seen Linux (and BSD) grow from "usable by hardcore geeks" to "usable by regular folks", but I don't understand why it needs to be this hard. I don't even have the vocabulary to properly describe just how frustrating it is having to deal with printing on UNIX. Every time it's the same old problems and some new ones, too, just for variety. I can honestly say I've probably spent more time fighting printing than any other single task (other than "using" the computer). The level of opaqueness that CUPS in particular exhibits still baffles me. If I'm having problems and I strace various processes to see what they are doing, I'll see error messages and informational messages, but no level of logging ever delivers them to a log file. They're being /dev/null'd or whatever. Who thought that was a good idea?
I've never gotten firefox to print the header and footer correctly. Some part of some document everywhere is cut off. Same goes for Open Office. Manually fiddling with your margins is something I might have expected to do in 1995, not 2008. CUPS has all this great info on the printer, thanks to the PPD - what it's actual printable area is, 4000 options, you name it. Why, then, don't some of the most basic things ever seem to work? One thing that really works well is the Printout Mode stuff - whoever thought that up was definitely smarter than your average bear.
I've given up entirely on Epson after no less than 3 different inkjets with clogged heads, never to print again, despite repeated attempts at repair. Canon, glorious though their printouts may be (no personal experience), aren't known for being exactly Linux friendly. Perhaps that's changed in the last year. Every time I look at a printer (for myself or much more likely for someone else) I check out the really awesome linuxprinting.org and, if it's not supported, I pass. Period. Kodak, despite all of the noise lately, appear to be completely useless for those of us that don't run Windows. HP, while Linux friendly, has come up with this hplip thing that mostly kinda-sorta works, but every now and again goes crazy and has to be killed off. I'll admit to having the best luck with HP printers, they last and the text printing is really fantastic. I still have an old HP laster with about 1 jillion (metric) copies on it and it still works great. I can turn it off, leave it for a year, and come back and it still works. 10 points for laser! I have no personal experience with Canon, but friends and family do and they love them. They're all running Windows or own a Mac, too. I've begun to think the entire inkjet thing is a huge scam.
Why even bother printing from home? I've all but given up on printing photos from home when I can use Picasa (online or the wine-ified version) and order printouts from any of a dozen places, for pickup or even delivery, for cheap. Picasa is awesome. Google really gets "just make it easy".
As far as printing photos this way, what is the level of frustration? That depends on the provider - I like using the LifePics provider as they are fairly local and seem to work really well. I tried using Walgreens (they are even closer and have a 1 hour wait) but, so far, they've been down 2 out of 3 tries. Photoworks didn't do a thing for me (I thought the interface wasn't very good), but Kodak Easyshare and Shutterfly are my favorites from a UI perspective. Fast, easy, unobtrusive. Snapfish is another common one. I'll never use WalMart.
Surely some people reading this will go, "Well, you idiot, all you have to do is this and that and edit this and tweak that and don't forget to wave a dead chicken over it (thrice)." and that's fine. Maybe I'm getting old but I'm pretty tired of having to do stuff like that just to get things to work. Modern unices are pretty much plug and play these days - yeah, setting up dual monitors and such can be "fun" but most of the time it just works great. Plug in your new USB keyboard? Hey! A keyboard! And it lights up! Plug in just about any camera, USB storage, and so on and so forth and Linux recognizes it and off you go. Yay! Plug in your printer and be prepared to lose the next hour (or more) of your life. You'll never get it back.
You might also be saying, "Well, stop grumping and do something about it." You bet. I'll get right on that in my copious spare time. I have contributed (and will continue to contribute) to a great many free/open/libre software projects over the years. That doesn't mean I won't try to help in some way - I'll file the appropriate informative, useful bug reports and even supply patches when I'm able.
[1] megapickles, because it's funnier.
Tuesday, July 29, 2008
Saturday, July 19, 2008
File Recovery / Undelete on ext3
I had the unfortunate situation where I had deleted some files (not many, only 2 or 3 files) and really did not want to re-do the week's worth of work. Google to the rescue. I found ext3grep, printed out the instructions (huge, need condensing for common-case), and built the software. It required a number of fixes to compile, which I'll forward upstream when I get a chance.
Did it work? More or less, yes. I got my files back. The time-range searching doesn't seem to work, as I could narrow down the window of deletion to about 15 minutes but the list of recovered files was HUGE, including files I had deleted weeks ago. However, the software did what it was supposed to do, and easily enough. It takes some time, but it did the job.
I gave them 10 Euro (about 400 American Pesos).
Thursday, July 10, 2008
RAID5,6 and 10 Benchmarks on 2.6.25.5
| Copyright: | Copyright © 2008 Jon Nelson |
|---|---|
| Date: | Jul 2008 |
This is an expansion of a previous post ( http://pycurious.blogspot.com/2007/12/some-raid10-performance-numbers.html ).
Since that time, I have redeployed using RAID10,f2. The redeployment went very well, but I'm not getting the performance I quite desired. More on that in another post. In the meantime, I slightly enhanced one of my benchmark scripts and decided to give it a go again.
1 Hardware and Setup
- the kernel is 2.6.25,5, openSUSE 11.0 "default" kernel for x86-64
- the CPU is an AMD x86-64, x2 3600+ in power-saving mode (1000 MHz)
- the motherboard is an EPoX MF570SLI which uses the nVidia MCP55 SATA controller (PCIe).
- in contrast to an earlier test, this time thera are 4 drives - 4 different makes of SATA II, 7200 rpm drives.
- each drive is capable of not much more than 80 MB/s (at best - the outermost tracks) and, on average, more like 70 MB/s for the portions of the disk involved in these tests
- the raids are comprised of 4x 4GB partitions, all from the first 8G of the disk.
- the system was largely idle but is a live system
- the system has 1 GB of RAM
- in contrast to the earlier test, the 'cfq' scheduler was used. I forgot to change it.
- the stripe_sizes and caches, queues sizes, and flusher parameters were left at their defaults
2 Important Notes
the caches were dropped before each invocation of 'dd':
echo 3 > /proc/sys/vm/drop_cachesthe 'write' portion of the test used conv=fdatasync
I did not test filesystem performance. This is just about the edge capabilities of linux RAID in various configurations.
I did not use iflag=direct (which sets O_DIRECT)
I ran each test 5 times, taking the mean average.
3 Questions
Initially, I just wanted to run a bunch of tests and eyeball the results. It's easy to do that, and draw conclusions from the data. However, it is maybe more useful to ask yourself, "What questions can be answered?" Here are a few questions I came up with, and the answers I came up with:
What did you really test?
Basically I tested streaming read and write performance to a series of raid levels and formats, using different chunk sizes for each.
I did not want to use any filesystem which only gets in the way for this kind of test - I wasn't testing the filesystem, I was testing to see how different raid formats, layouts, and chunk sizes make a difference.
A future installment may include filesystem testing as well, which I find just as if not more important, however it's so much more variable that I'm not really sure much sense can be found in the noise.
Why didn't you include my-favorite-raid-level?
I only wanted to include raid levels for which there is some redundancy. I could have included raid 1+0 but my test script is not sufficiently smart for that. Perhaps I'll include that in a future installment.
Can I have the source to the test program?
Sure. I'll try to make it available if somebody asks, but it's really nothing special. Futhermore, it's my intent to refine it a bit to support filesystem testing (via bonnie++ or iozone, preferred) and so on.
When using raid5, does the format matter?
If you squint your eyes a bit, the write performance, regardless of format, were all pretty close. The read performance was more variable, but still did not vary all that much. Chunk size seemed to matter more. Left-symmetric did the best overall, however.
How is the performance graphed versus predicted?
Left to the reader to comment!
Did you do the readahead settings for the test?
No. I left them at their defaults.
What are you using to generate this?
I am using reStructuredText, combined with Pygments.
Which tool do you use to make graphs?
Google Charts (by way of pygooglechart), a bunch of shell and Python. I used flot previously.
How do the individual drives perform?
The drives are:
<6>ata3.00: ATA-7: Hitachi HDT725032VLA360, V54OA52A, max UDMA/133 <6>ata4.00: ATA-8: SAMSUNG HD321KJ, CP100-10, max UDMA7 <6>ata5.00: ATA-7: ST3320620AS, 3.AAK, max UDMA/133 <6>ata6.00: ATA-8: WDC WD3200AAKS-75VYA0, 12.01B02, max UDMA/133And their performance:
/dev/sdb: Timing buffered disk reads: 218 MB in 3.01 seconds = 72.44 MB/sec /dev/sdc: Timing buffered disk reads: 234 MB in 3.00 seconds = 77.92 MB/sec /dev/sdd: Timing buffered disk reads: 228 MB in 3.02 seconds = 75.60 MB/sec /dev/sde: Timing buffered disk reads: 234 MB in 3.02 seconds = 77.57 MB/sec
What difference does the scheduler make?
As can clearly be seen on the RAID5 graphs, the IO scheduler can make a big difference. Using cfq or nooop, reads start out almost a full point faster than the others, and writes are 1/2 point faster.
On the other hand, for RAID6, the scheduler doesn't seem to make much difference at all. At least for streaming reads/writes, which is all I'm testing here.
For RAID10,n2 and RAID10,o2 the story is the same as for RAID6, but there is some impact (up to 1.0 points!) for RAID10,f2.
What revisions have you made to this document?
I re-ran the tests to include 2048K chunk sizes, and removed 128K as it wasn't very interesting and it cluttered up the graphs.
I also re-ran the entire set of tests for the other three schedulers, noop, anticipatory, and deadline.
I re-did the graphs using the Google Charts API (by way of pygooglechart) instead of using flot. There was nothing wrong with flot, in fact I found the software really nice to use, but some people found the google charts "prettier" and it's somewhat easier for me to use.
4 Unanswered Questions
While I don't have the data in this article, I did originally perform these tests on 2.6.22.18. The results were rather noisier, and in most cases a bit worse.
Why aren't raid10,f2 reads getting closer to 4.0x ?
What's with the strange drop in performance at 512K chunk sizes for RAID10,f2 for the deadline and noop schedulers, only to rise again at 1024K (and then drop at 2048K)?
Why are raid10,o2 reads so AWFUL?
Neil Brown was kind enough to suggest re-running with a larger chunk size, which I did.
The read performance did, indeed, perform better. Up to the 3.0 mark, in fact.
Why do raid6 reads behave the way they do? I would have expected a more linear graph - the raid6 write graph is very smooth.
From 64 to 256k chunk size, there is little change (in either direction, for reads or writes) but at 512K the reads really improve and continue to do so as the chunk size increases.
What should the theoretical performance of the various raid levels and formats look like?
For raid10,f2 I would suspect that 4.0 would be perfect (for reads), and for sustained writes something like 1.5.
I get 1.5 like this:
the avg. speed of writing a given chunk of data should look like this:
avg of writing to outer track + writing to inner track -> (70 + 35) / 2.0, (assuming inner track is 1/2 the speed of outer tracks) and theoretically we could write to 2 devices at a time, so... (( 70 + 35 ) / 2.0) * 2.0 / 70.0 = 1.5x.
In reality, we do a bit better than that, probably due to the fact that I'm not using the whole disk and therefore the speed of the inner tracks of the region I'm actually using is greater than would otherwise be true.
5 Tables, Charts n Graphs
The following results are expressed in terms of a single with (a baseline) with 1.0 being the speed of a single drive (about 70MB/s).
| scheduler | level | layout | chunk | writing | reading |
|---|---|---|---|---|---|
| cfq | raid10 | f2 | 64 | 1.48 | 3.01 |
| cfq | raid10 | f2 | 128 | 1.49 | 3.88 |
| cfq | raid10 | f2 | 256 | 1.50 | 3.68 |
| cfq | raid10 | f2 | 512 | 1.52 | 3.65 |
| cfq | raid10 | f2 | 1024 | 1.47 | 3.76 |
| cfq | raid10 | f2 | 2048 | 1.52 | 3.73 |
| cfq | raid10 | n2 | 64 | 1.78 | 1.89 |
| cfq | raid10 | n2 | 128 | 1.85 | 1.87 |
| cfq | raid10 | n2 | 256 | 1.82 | 2.00 |
| cfq | raid10 | n2 | 512 | 1.84 | 2.15 |
| cfq | raid10 | n2 | 1024 | 1.83 | 2.42 |
| cfq | raid10 | n2 | 2048 | 1.83 | 2.70 |
| cfq | raid10 | o2 | 64 | 1.83 | 1.96 |
| cfq | raid10 | o2 | 128 | 1.80 | 1.96 |
| cfq | raid10 | o2 | 256 | 1.84 | 1.98 |
| cfq | raid10 | o2 | 512 | 1.80 | 1.98 |
| cfq | raid10 | o2 | 1024 | 1.83 | 2.49 |
| cfq | raid10 | o2 | 2048 | 1.80 | 3.13 |
| cfq | raid5 | left-asymmetric | 64 | 1.72 | 2.51 |
| cfq | raid5 | left-asymmetric | 128 | 1.67 | 2.79 |
| cfq | raid5 | left-asymmetric | 256 | 1.52 | 2.92 |
| cfq | raid5 | left-asymmetric | 512 | 1.31 | 2.76 |
| cfq | raid5 | left-asymmetric | 1024 | 1.06 | 3.44 |
| cfq | raid5 | left-asymmetric | 2048 | 0.56 | 3.25 |
| cfq | raid5 | left-symmetric | 64 | 1.74 | 2.71 |
| cfq | raid5 | left-symmetric | 128 | 1.73 | 2.76 |
| cfq | raid5 | left-symmetric | 256 | 1.55 | 2.97 |
| cfq | raid5 | left-symmetric | 512 | 1.34 | 2.88 |
| cfq | raid5 | left-symmetric | 1024 | 1.08 | 3.44 |
| cfq | raid5 | left-symmetric | 2048 | 0.58 | 3.50 |
| cfq | raid5 | right-asymmetric | 64 | 1.75 | 2.70 |
| cfq | raid5 | right-asymmetric | 128 | 1.61 | 2.88 |
| cfq | raid5 | right-asymmetric | 256 | 1.58 | 2.88 |
| cfq | raid5 | right-asymmetric | 512 | 1.28 | 2.88 |
| cfq | raid5 | right-asymmetric | 1024 | 1.04 | 3.25 |
| cfq | raid5 | right-asymmetric | 2048 | 0.54 | 3.31 |
| cfq | raid5 | right-symmetric | 64 | 1.75 | 2.79 |
| cfq | raid5 | right-symmetric | 128 | 1.69 | 2.81 |
| cfq | raid5 | right-symmetric | 256 | 1.56 | 2.88 |
| cfq | raid5 | right-symmetric | 512 | 1.30 | 2.75 |
| cfq | raid5 | right-symmetric | 1024 | 1.01 | 3.02 |
| cfq | raid5 | right-symmetric | 2048 | 0.49 | 3.24 |
| cfq | raid6 | 64 | 1.30 | 1.76 | |
| cfq | raid6 | 128 | 1.24 | 1.96 | |
| cfq | raid6 | 256 | 1.17 | 1.91 | |
| cfq | raid6 | 512 | 1.04 | 2.70 | |
| cfq | raid6 | 1024 | 0.87 | 2.92 | |
| cfq | raid6 | 2048 | 0.60 | 3.31 | |
| deadline | raid10 | f2 | 64 | 1.78 | 2.63 |
| deadline | raid10 | f2 | 256 | 1.82 | 3.80 |
| deadline | raid10 | f2 | 512 | 1.72 | 3.32 |
| deadline | raid10 | f2 | 1024 | 1.75 | 3.61 |
| deadline | raid10 | f2 | 2048 | 1.47 | 3.40 |
| deadline | raid10 | n2 | 64 | 1.96 | 1.21 |
| deadline | raid10 | n2 | 256 | 1.88 | 1.85 |
| deadline | raid10 | n2 | 512 | 1.84 | 2.10 |
| deadline | raid10 | n2 | 1024 | 1.89 | 2.41 |
| deadline | raid10 | n2 | 2048 | 1.84 | 2.59 |
| deadline | raid10 | o2 | 64 | 1.80 | 1.94 |
| deadline | raid10 | o2 | 256 | 1.82 | 1.96 |
| deadline | raid10 | o2 | 512 | 1.73 | 1.94 |
| deadline | raid10 | o2 | 1024 | 1.87 | 2.63 |
| deadline | raid10 | o2 | 2048 | 1.82 | 3.13 |
| deadline | raid5 | left-asymmetric | 64 | 1.67 | 2.55 |
| deadline | raid5 | left-asymmetric | 256 | 1.43 | 2.84 |
| deadline | raid5 | left-asymmetric | 512 | 1.22 | 2.76 |
| deadline | raid5 | left-asymmetric | 1024 | 1.04 | 3.27 |
| deadline | raid5 | left-asymmetric | 2048 | 0.52 | 3.31 |
| deadline | raid5 | left-symmetric | 64 | 1.61 | 2.32 |
| deadline | raid5 | left-symmetric | 256 | 1.42 | 2.89 |
| deadline | raid5 | left-symmetric | 512 | 1.26 | 2.89 |
| deadline | raid5 | left-symmetric | 1024 | 1.08 | 3.14 |
| deadline | raid5 | left-symmetric | 2048 | 0.55 | 3.31 |
| deadline | raid5 | right-asymmetric | 64 | 1.68 | 2.15 |
| deadline | raid5 | right-asymmetric | 256 | 1.50 | 2.88 |
| deadline | raid5 | right-asymmetric | 512 | 1.23 | 2.83 |
| deadline | raid5 | right-asymmetric | 1024 | 0.97 | 3.44 |
| deadline | raid5 | right-asymmetric | 2048 | 0.47 | 3.24 |
| deadline | raid5 | right-symmetric | 64 | 1.64 | 2.11 |
| deadline | raid5 | right-symmetric | 256 | 1.50 | 2.84 |
| deadline | raid5 | right-symmetric | 512 | 1.22 | 2.83 |
| deadline | raid5 | right-symmetric | 1024 | 1.00 | 3.02 |
| deadline | raid5 | right-symmetric | 2048 | 0.43 | 3.19 |
| deadline | raid6 | 64 | 1.22 | 1.73 | |
| deadline | raid6 | 256 | 1.20 | 1.75 | |
| deadline | raid6 | 512 | 1.04 | 2.45 | |
| deadline | raid6 | 1024 | 0.89 | 3.19 | |
| deadline | raid6 | 2048 | 0.57 | 3.32 | |
| anticipatory | raid10 | f2 | 64 | 1.62 | 2.59 |
| anticipatory | raid10 | f2 | 128 | 1.59 | 3.50 |
| anticipatory | raid10 | f2 | 256 | 1.61 | 3.46 |
| anticipatory | raid10 | f2 | 512 | 1.65 | 3.73 |
| anticipatory | raid10 | f2 | 1024 | 1.61 | 3.58 |
| anticipatory | raid10 | f2 | 2048 | 1.47 | 3.80 |
| anticipatory | raid10 | n2 | 64 | 1.87 | 1.21 |
| anticipatory | raid10 | n2 | 128 | 1.83 | 1.45 |
| anticipatory | raid10 | n2 | 256 | 1.83 | 1.90 |
| anticipatory | raid10 | n2 | 512 | 1.83 | 2.20 |
| anticipatory | raid10 | n2 | 1024 | 1.82 | 2.45 |
| anticipatory | raid10 | n2 | 2048 | 1.82 | 2.70 |
| anticipatory | raid10 | o2 | 64 | 1.82 | 1.91 |
| anticipatory | raid10 | o2 | 128 | 1.85 | 1.94 |
| anticipatory | raid10 | o2 | 256 | 1.86 | 2.05 |
| anticipatory | raid10 | o2 | 512 | 1.80 | 1.96 |
| anticipatory | raid10 | o2 | 1024 | 1.83 | 2.63 |
| anticipatory | raid10 | o2 | 2048 | 1.78 | 3.19 |
| anticipatory | raid5 | left-asymmetric | 64 | 1.62 | 2.42 |
| anticipatory | raid5 | left-asymmetric | 128 | 1.59 | 2.63 |
| anticipatory | raid5 | left-asymmetric | 256 | 1.48 | 2.79 |
| anticipatory | raid5 | left-asymmetric | 512 | 1.32 | 2.88 |
| anticipatory | raid5 | left-asymmetric | 1024 | 1.10 | 3.37 |
| anticipatory | raid5 | left-asymmetric | 2048 | 0.54 | 3.25 |
| anticipatory | raid5 | left-symmetric | 64 | 1.67 | 2.49 |
| anticipatory | raid5 | left-symmetric | 128 | 1.62 | 2.76 |
| anticipatory | raid5 | left-symmetric | 256 | 1.52 | 2.83 |
| anticipatory | raid5 | left-symmetric | 512 | 1.32 | 2.76 |
| anticipatory | raid5 | left-symmetric | 1024 | 1.10 | 3.32 |
| anticipatory | raid5 | left-symmetric | 2048 | 0.58 | 3.25 |
| anticipatory | raid5 | right-asymmetric | 64 | 1.67 | 2.17 |
| anticipatory | raid5 | right-asymmetric | 128 | 1.55 | 2.63 |
| anticipatory | raid5 | right-asymmetric | 256 | 1.48 | 2.76 |
| anticipatory | raid5 | right-asymmetric | 512 | 1.30 | 2.92 |
| anticipatory | raid5 | right-asymmetric | 1024 | 1.09 | 3.37 |
| anticipatory | raid5 | right-asymmetric | 2048 | 0.52 | 3.37 |
| anticipatory | raid5 | right-symmetric | 64 | 1.72 | 2.19 |
| anticipatory | raid5 | right-symmetric | 128 | 1.67 | 2.63 |
| anticipatory | raid5 | right-symmetric | 256 | 1.47 | 2.88 |
| anticipatory | raid5 | right-symmetric | 512 | 1.32 | 2.88 |
| anticipatory | raid5 | right-symmetric | 1024 | 1.07 | 3.02 |
| anticipatory | raid5 | right-symmetric | 2048 | 0.47 | 3.20 |
| anticipatory | raid6 | 64 | 1.26 | 1.75 | |
| anticipatory | raid6 | 128 | 1.22 | 1.67 | |
| anticipatory | raid6 | 256 | 1.19 | 1.77 | |
| anticipatory | raid6 | 512 | 1.03 | 2.59 | |
| anticipatory | raid6 | 1024 | 0.91 | 3.08 | |
| anticipatory | raid6 | 2048 | 0.58 | 3.24 | |
| noop | raid10 | f2 | 64 | 1.40 | 2.71 |
| noop | raid10 | f2 | 256 | 1.42 | 3.80 |
| noop | raid10 | f2 | 512 | 1.42 | 3.38 |
| noop | raid10 | f2 | 1024 | 1.42 | 3.65 |
| noop | raid10 | f2 | 2048 | 1.46 | 3.38 |
| noop | raid10 | n2 | 64 | 1.84 | 1.21 |
| noop | raid10 | n2 | 256 | 1.83 | 1.90 |
| noop | raid10 | n2 | 512 | 1.85 | 2.18 |
| noop | raid10 | n2 | 1024 | 1.85 | 2.45 |
| noop | raid10 | n2 | 2048 | 1.83 | 2.55 |
| noop | raid10 | o2 | 64 | 1.82 | 1.90 |
| noop | raid10 | o2 | 256 | 1.85 | 1.92 |
| noop | raid10 | o2 | 512 | 1.80 | 1.97 |
| noop | raid10 | o2 | 1024 | 1.62 | 2.63 |
| noop | raid10 | o2 | 2048 | 1.78 | 3.13 |
| noop | raid5 | left-asymmetric | 64 | 1.75 | 2.63 |
| noop | raid5 | left-asymmetric | 256 | 1.62 | 2.92 |
| noop | raid5 | left-asymmetric | 512 | 1.37 | 2.92 |
| noop | raid5 | left-asymmetric | 1024 | 1.09 | 3.32 |
| noop | raid5 | left-asymmetric | 2048 | 0.54 | 3.50 |
| noop | raid5 | left-symmetric | 64 | 1.78 | 2.20 |
| noop | raid5 | left-symmetric | 256 | 1.62 | 2.88 |
| noop | raid5 | left-symmetric | 512 | 1.37 | 2.88 |
| noop | raid5 | left-symmetric | 1024 | 1.12 | 3.25 |
| noop | raid5 | left-symmetric | 2048 | 0.58 | 3.37 |
| noop | raid5 | right-asymmetric | 64 | 1.78 | 2.23 |
| noop | raid5 | right-asymmetric | 256 | 1.61 | 2.97 |
| noop | raid5 | right-asymmetric | 512 | 1.38 | 2.89 |
| noop | raid5 | right-asymmetric | 1024 | 1.04 | 3.30 |
| noop | raid5 | right-asymmetric | 2048 | 0.52 | 3.25 |
| noop | raid5 | right-symmetric | 64 | 1.78 | 2.29 |
| noop | raid5 | right-symmetric | 256 | 1.65 | 2.84 |
| noop | raid5 | right-symmetric | 512 | 1.38 | 2.92 |
| noop | raid5 | right-symmetric | 1024 | 1.09 | 3.03 |
| noop | raid5 | right-symmetric | 2048 | 0.47 | 3.19 |
| noop | raid6 | 64 | 1.29 | 1.72 | |
| noop | raid6 | 256 | 1.21 | 1.84 | |
| noop | raid6 | 512 | 1.05 | 2.56 | |
| noop | raid6 | 1024 | 0.88 | 3.08 | |
| noop | raid6 | 2048 | 0.61 | 3.31 |
Tuesday, June 24, 2008
10 Second Filesystem Performance Table
- Source: 8.3 GiB, 154362 files on a Linux Software RAID 10, f2 layout.
- Destination: Western Digital 300G SATA - WDC WD3200AAKS-75VYA0, one partition.
| Filesystem | Copy | Delete |
| xfs (defaults) | 1133 | 311 |
| ext3 (defaults) | 767 | 63 |
| ext2 (defaults) | 752 | 57 |
| jfs | 769 | 121 |
| spadfs | 766 | 7 |
Sunday, June 22, 2008
DAR - Disk ARchiver
I first looked into dar a few years ago when I needed something to back up a friend's Windows machine at the filesystem level, and make it available to him (also on Windows) later. I more or less arbitrarily chose DAR. Since then, I haven't really used it. While rdiff-backup is working well for me, I'm always looking at other options. One option I was looking at is archive or file-based backups which, instead of storing entire trees as trees of their own (copies), stores the trees in a single or small set of files. Think "tarball" or "zip file" and you've got the idea. After some recent bad experiences with GNU tar, GNU cpio, and even pax, however, I was looking somewhat outside the usual circle of suspects. I remember my experience with dar so I thought I'd give dar a more in-depth look.
First, a bit about dar. dar stands for Disk ARchiver. To quote from the home page at http://dar.linux.free.fr/:
dar is a shell command that backs up directory trees and files. It has been tested under Linux, Windows, Solaris, FreeBSD, NetBSD, MacOS X and several other systems, it is released under the GNU General Public License (GPL).
dar looks very professionally done and the project appears healthy. dar has a great record of "getting it right the first time" - a sign of quality code and management of process. dar's feature list is rather impressive (more on this later). On the surface, it looks like dar will make your bread and slice it too.
My enthusiasm diminished almost immediately, however. I must preface this bit by saying that not once did I encounter a situation where I doubted the robustness of the code, but rather felt that some suboptimal user interface design decisions had been made. Some were even borderline show stoppers for me.
The first issue I ran into is that dar doesn't write to the file I told it to - it creates a new filename based on what I told it to use. dar calls this the basename. For example, let's say I asked it to create an archive and write it to file "foo". It won't create a file "foo" and write to it, but instead, it will create "foo.1.dar". I asked for "foo" I should get "foo". I understand why this was done but I don't have to like it - I'd rather complicate the file naming with --serialnames and "foo.%serial" or even a secondary helper program. dar clearly has the built-in support for splitting archives, but forcing the basename concept on the user doesn't win it any points from me.
Moving on, the second issue I ran into is the reporting of files that it archives: they are always shown as absolute paths, even when I am not specifying an absolute path. I'm used to tar and cpio and zip and just about everything else out there doing what I tell it to - if I say to process "foo" (implicitly "./foo") it processes "foo", not "/wherever/you/are/now/foo". I always use relative pathnames. Indeed, tar goes out of its way to always use relative pathnames, unless told to really really use absolute pathnames. cpio is most commonly invoked using the find(2) utility and performs no relative-to-absolute path munging. dar insists on reporting the full path of the files it archives. I'll show you by way of example:
dar --create files-to-archive --fs-root=./files-to-archive/ -v
(Again, this would create files-to-archive.1.dar, not files-to-archive. Grr.)
Let's create and populate files-to-archive:
[username:~] rm -rf files-to-archive; mkdir files-to-archive && ( for i in $(seq 1 10); do touch files-to-archive/file-$i ; done ) [username:~] find files-to-archive/ files-to-archive/ files-to-archive/file-7 files-to-archive/file-5 files-to-archive/file-4 files-to-archive/file-3 files-to-archive/file-6 files-to-archive/file-2 files-to-archive/file-9 files-to-archive/file-1 files-to-archive/file-10 files-to-archive/file-8 [username:~]
Groovy. Now, let's archive them:
[username:~] dar --create files-to-archive --fs-root=./files-to-archive/ -v Adding file to archive: /home/username/files-to-archive/file-7 Adding file to archive: /home/username/files-to-archive/file-5 Adding file to archive: /home/username/files-to-archive/file-4 Adding file to archive: /home/username/files-to-archive/file-3 Adding file to archive: /home/username/files-to-archive/file-6 Adding file to archive: /home/username/files-to-archive/file-2 Adding file to archive: /home/username/files-to-archive/file-9 Adding file to archive: /home/username/files-to-archive/file-1 Adding file to archive: /home/username/files-to-archive/file-10 Adding file to archive: /home/username/files-to-archive/file-8 Writing archive contents... -------------------------------------------- 10 inode(s) saved with 0 hard link(s) recorded 0 inode(s) changed at the moment of the backup 0 inode(s) not saved (no inode/file change) 0 inode(s) failed to save (filesystem error) 0 inode(s) ignored (excluded by filters) 0 inode(s) recorded as deleted from reference backup -------------------------------------------- Total number of inode considered: 10 -------------------------------------------- EA saved for 0 inode(s) -------------------------------------------- [username:~]
But! But! But I didn't say process /home/username/files-to-archive/ I said to process ./files-to-archive. That's annoying. The output makes me question whether I invoked dar correctly and how the files are actually stored in the archive. Are they stored with absolute paths or relative? If they are stored with absolute paths (mirroring what is displayed), then that's not what I asked for. If they are stored with relative paths (which is what I asked for), then the display is confusing!
Let's see what's in the archive:
[username:~] dar --list files-to-archive [data ][ EA ][compr] | permission | user | group | size | date | filename ----------------------+------------+-------+-------+-------+-------------------------------+------------ [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-7 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-5 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-4 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-3 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-6 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-2 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-9 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-1 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-10 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 file-8 [username:~]
Uh. That's not at all what I wanted. It's missing the leading 'files-to-archive/' part. So I read the manpage. Again. (read read read...) OK, let's try this. I think I understand what I did wrong. The fs-root option doesn't do what I thought it did. Since fs-root defaults to '.' I don't need that option any more.
Let's see what happens when I invoke it "correctly":
[username:~] dar --create files-to-archive -g files-to-archive -v Cannot read directory contents: /home/username/lost+found : Error opening directory: /home/username/lost+found : Permission denied Adding file to archive: /home/username/files-to-archive Adding file to archive: /home/username/files-to-archive/file-7 Adding file to archive: /home/username/files-to-archive/file-5 Adding file to archive: /home/username/files-to-archive/file-4 Adding file to archive: /home/username/files-to-archive/file-3 Adding file to archive: /home/username/files-to-archive/file-6 Adding file to archive: /home/username/files-to-archive/file-2 Adding file to archive: /home/username/files-to-archive/file-9 Adding file to archive: /home/username/files-to-archive/file-1 Adding file to archive: /home/username/files-to-archive/file-10 Adding file to archive: /home/username/files-to-archive/file-8 Writing archive contents... -------------------------------------------- 11 inode(s) saved with 0 hard link(s) recorded 0 inode(s) changed at the moment of the backup 0 inode(s) not saved (no inode/file change) 0 inode(s) failed to save (filesystem error) 122 inode(s) ignored (excluded by filters) 0 inode(s) recorded as deleted from reference backup -------------------------------------------- Total number of inode considered: 133 -------------------------------------------- EA saved for 0 inode(s) -------------------------------------------- [username:~]
Pretty much the same as last time. Let's see what's in the archive:
[username:~] dar --list files-to-archive [data ][ EA ][compr] | permission | user | group | size | date | filename ----------------------+------------+-------+-------+-------+-------------------------------+------------ [Saved] [-----] drwxr-xr-x username users 0 Sun Jun 15 21:27:53 2008 files-to-archive [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-7 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-5 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-4 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-3 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-6 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-2 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-9 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-1 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-10 [Saved] [ ] -rw-r--r-- username users 0 Sun Jun 15 21:27:53 2008 files-to-archive/file-8 [username:~]
OK, that's better, but that invocation is by no means intuitive. Wait a minute, what's this bit about lost+found? Why does it care about lost+found, which is not part of the tree I asked dar to operate on? Maybe it has to do with how dar recurses. Let's try the cpio approach, wherein the archiver gets it's list of filenames on stdin. Something like this:
find files-to-archive | dar --create files-to-archive --include-from-file -
(Omitting more-or-less duplicate output and one erroneous invocation...)
That doesn't work. I have to use /dev/stdin instead of '-'. Ugh. Fine. When I re-invoke, however, I still get the grump about lost+found, though. I don't understand why dar thinks it needs to stat any file that it's not going to archive. It knows it's not going to archive lost+found, by virtue of the fact that it's clearly not present in the list of input paths! I consider this to be a bug. Imagine how horrible the performance might be if you wanted to archive 30 files out of 100,000. dar would stat every one of them! Consider the ramifications of this design choice in the following situations:
- a damaged filesystem (let's say I know I can access A and B but when I access C I encounter a bad block...)
- NFS or something equally expensive
- bad NFS mount points
Moving on again, let's skip to a pet-peeve, hard links. Handling hard links badly is a show-stopper for me, so let's see how dar handles this:
[username:~] rm -rf foo [username:~] mkdir foo [username:~] dd if=/dev/zero of=foo/a bs=100k count=1 [username:~] ln foo/a foo/b [username:~] ln foo/a foo/c [username:~] ln foo/a foo/d [username:~] find foo -ls 3342147 4 drwxr-xr-x 2 username users 4096 Jun 21 09:32 foo 3342148 0 -rw-r--r-- 4 username users 102400 Jun 21 09:32 foo/d 3342148 0 -rw-r--r-- 4 username users 102400 Jun 21 09:32 foo/c 3342148 0 -rw-r--r-- 4 username users 102400 Jun 21 09:32 foo/a 3342148 0 -rw-r--r-- 4 username users 102400 Jun 21 09:32 foo/b [username:~] find foo | dar -c foo --include-from-file=/dev/stdin Cannot read directory contents: /home/username/lost+found : Error opening directory: /home/username/lost+found : Permission denied -------------------------------------------- 2 inode(s) saved with 3 hard link(s) recorded 0 inode(s) changed at the moment of the backup 0 inode(s) not saved (no inode/file change) 0 inode(s) failed to save (filesystem error) 5 inode(s) ignored (excluded by filters) 0 inode(s) recorded as deleted from reference backup -------------------------------------------- Total number of inode considered: 7 -------------------------------------------- EA saved for 0 inode(s) -------------------------------------------- [username:~]
Whoah. Wait a minute! There are 4 files and one directory, but dar says it saved 2 inodes with 3 links recorded. Hmmmm. OK, yeah. 2 inodes - 1 file and 1 directory, and 3 hard links to that file.:
[username:~] dar --list foo [data ][ EA ][compr] | permission | user | group | size | date | filename ----------------------+------------+-------+-------+-------+-------------------------------+------------ [Saved] [-----] drwxr-xr-x username users 0 Sat Jun 21 09:32:33 2008 foo [Saved] [ ] hrw-r--r-- username users 102400 Sat Jun 21 09:32:27 2008 foo/d [Saved] [ ] hrw-r--r-- username users 102400 Sat Jun 21 09:32:27 2008 foo/c [Saved] [ ] hrw-r--r-- username users 102400 Sat Jun 21 09:32:27 2008 foo/a [Saved] [ ] hrw-r--r-- username users 102400 Sat Jun 21 09:32:27 2008 foo/b [username:~]
Alright. So far so good. The size of the archive (102547 bytes) shows it's only storing one copy of the data. Good! dar handles hardlinks just fine.
Taking a step back, I've been beating up on dar pretty badly. dar does some things I don't care for, and some of those things may even be show stoppers. Is it worth continuing the investigation given these annoyances? dar is pretty sophisticated. The archive format itself is well-developed (and appears at least as capable as any of the tar formats), including support for extended attributes (I couldn't care less), acls (ditto), huge filenames, etc.. dar also supports (by no means an exhaustive list):
- splitting - merging (cool!, with lots of file-selection smarts, too) - encryption (both cruddy and good) - storing the table-of-contents (aka, "catalog") only and using *that* to create the following: - incremental/differential archives
It would appear that you can even merge differentials/incremental archives with each other and "full" archives to create new archives, all using a fairly powerful file selection mechanism. Furthermore, dar can record which files were removed between one archive and another, something that nothing else I've run across can do, which also happens to be a big issue for me.
dar looks enormously capable and does seem to be working properly. The issues I encountered:
- pathname reporting on archive (UI issue)
- stat(2)'ing files outside of the requested source tree
- the 'basename' archive filename imposition
won't dissuade me from investigating its use more thoroughly, but it's not going to happen with as much enthusiasm as I would have hoped.
Saturday, June 21, 2008
rdiff-backup
I was recently exploring alternate disk-to-disk backup strategies when I ran across rdiff-backup. I liked what I saw, so I gave it a try. Quite frankly, it was awesome. It worked out-of-the-box (it even comes with openSUSE making it easy for me to install) and worked very well with no surprises.
rdiff-backup makes good use of librsync to only tranfer the changed portions of files, and stores the most-current backup tree otherwise unchanged, storing previous backups as reverse deltas against the current tree. This has several advantages for me:
- Huge data sets which change infrequently or in only minor ways can be backed up much more quickly.
- The resulting tree is easy to search, restore from, etc.. provided you are looking for the "most recent" data. You get to use all of the tools that you use all day every day. You don't have to learn a new tool to get at your data, if the most recent backup will suffice.
- Even getting at older data is not much harder, but you do have to use rdiff-backup. Unfortunately, neither rdiff-backup -h not --help work, but the manpage is pretty good, and the "how do I restore stuff" sections are straightforward and don't seem to contain any gremlins. A simple rdiff-backup --restore-as-of 10D /backups/home/user/data /home/user/data.10daysago will suffice.
- Storing reverse deltas is a huge bonus as most often I'm interested in the most-recent data, and my interest drops off sharply the further back you go. This has the added advantage (versus forward deltas) that you never need to perform more than one full backup, and each backup is essentially only what has changed since the last backup, making the backups FAST and very space efficient.
The only real difference that I can see between the tree that got backed up and the resulting tree is the addition of an 'rdiff-backup-data' directory, which stores a bunch of rdiff-backup meta-data and the reverse deltas (called 'incrementals') relating to previous backups.
I managed to get rdiff-backup up and running on 5 machines in 5-10 minutes each. As far as a productivity win goes, this is a big one. I've never had anything work so well out of the box and with so little fuss.
Thursday, June 12, 2008
NSIS installer for WinAoE
On Mar 10, 2008 I became aware of an AoE driver for Windows ( here ) which was under an Open Source license (GPLv3). Of course, I jumped at it and played with it a bit. However, I immediately felt that one missing bit was that all of the hoop-jumping to get it installed. I grabbed Inno Setup, a Windows Installer Maker that I've used before. It's very nice! However, I couldn't get it to do what I wanted it to, so I turned to NSIS, and found a NSIS plugin here that was just right for this job. After much fiddling, I got everything working reasonably well. I tried contacting the author several times and in several ways, but never had any luck, and eventually my interests found themselves elsewhere for a while. Some people are just hard to get ahold of!
A few days ago, I thought I'd post my results to the aoe-discuss mailing list, and several people expressed some interest, and that's what this post is for.
First, a list of the components and their licenses with comments:
- WinAoE - GPLv3
- NSIS - A Bunch of Licenses, see http://nsis.sourceforge.net/License
- The InstDrv NSIS plugin - similar to 3-clause BSD ("as-is")
- an Installer.nsi file - unfortunately, blogger doesn't let you host any files (not even .txt files!) other than audio and video.
- Download and Install NSIS
- Download the InstDrv NSIS plugin
- Unpack the InstDrv NSIS plugin
- Copy the file InstDrv.dll from Plugins into the NSIS Plugins directory ( typically C:\Program Files\NSIS\Plugins\ )
- Download WinAoE, and unpack it somewhere
- Copy the Installer.nsi *contents* (except for the very bottom - do not include the links that Google Docs includes) to the clipboard
- Open up notepad and paste the contents into a new document
- Save the document into the bin directory of the unpacked WinAoE source and call it Installer.nsi
- Right-click the Installer.nsi file and choose "Compile NSIS Script". It should compile fine. If it doesn't, make sure you copied-and-pasted correctly.
Wednesday, June 4, 2008
Acer 7720-6569 - Overheating Problems
One of the laptops that I work on from time-to-time is a friend's Acer 7720-6569. It's got an absolutely beautiful 17" display, nice sound, and overall the specs are very nice, especially for the money. It's a bit big, and there are some things which don't work quite right.
One of them is the fan:
I've been lead to believe (by this page) that Acer (among others) uses ACPI to control the fan instead of native fan control. Whether it is true or not, I've noticed two things:
- the Acer's fan never seems to go very fast
- when really being worked, sometimes the laptop just shuts off.
Make a bootable USB disk with FreeDOS and VirtualBox
I was working on a friend's computer which needed a BIOS update. In it's infinite wisdom, the manufacturer provided update files which consume more space than can fit on a conventional 1.44MB floppy. Furthermore, the computer is a laptop without a floppy drive, like most laptops these days. How did I solve this problem?
I'll skip all of the horrible, gory details of the 500 things that I tried that did not work, including the several dozen attempts at making either a USB bootable HDD or getting my 512MB USB key fob to work. The last approach I tried was to use a FreeDOS boot disk and copy it to the flash device:
zcat /usr/share/dosbootdisk/floppy.gz > /dev/sdb
This worked, but the "floppy" still only appeared to be 1.44MB. I tried a few things to enlarge it but eventually gave up.
Riding this partial success, I took a different approach. I acquired from the FreeDOS project the smallest (about 8MB) bootable ISO, and using VirtualBox (the OSE edition) I created a virtual machine with a 200MB hard disk, and configured it to boot the ISO. After booting, I used the ISO to partition and format the virtual hard disk and perform a basic installation. Then I shut the virtual machine down. Then I used *vditool* to convert the virtual hard disk from the .vdi format to one suitable for using with dd:
vditool copydd .VirtualBox/VDI//NewHardDisk1.vdi freedos_usb.img
As root, I copied the freedos_usb.img file to /dev/sdb:
cat freedos_usb.img > /dev/sdb; sync; sync; sync
I ejected and re-inserted the USB key fob which caused it to be mounted under /media/NewDisk or some such, and I simply copied the BIOS updating files to a new bios/ subdirectory. After ejecting the device, I inserted it into the other machine and turned it on. Pressing F12 lets me choose another boot device, and the 'USB-HDD' showed up, as it had done before. This time, however, it booted perfectly and I was able to update the BIOS quite easily.
The BIOS update did not resolve the issue I was having, but I learned quite a bit and will save this image somewhere, I'm sure it will come in handy again someday.
In short:
- obtain FreeDOS bootable ISO (the small 8MB image is just fine)
- use VirtualBox (you could probably use other emulation software, of course) to create a machine with a virtual hard drive, anything from 5MB to a few hundred MB.
- boot the FreeDOS ISO, partition and install FreeDOS to the new virtual hard drive
- convert the virtual hard drive to a suitable format for 'dd'
- copy the image to a USB key fob, and force it to be rescanned (eject and re-insert it)
- copy the additional contents you desire to the newly mounted DOS partition
- eject and use the key fob
Saturday, March 1, 2008
openSUSE 10.3 - Some Post-Install Setup
I really like the KDE desktop. It's clean, consistent, and very important to me - doesn't prevent me from making choices - it presents them, has sensible defaults, but they can still be changed. Additionally, everything seems to work together so smoothly and without-fuss. However, the I do typically change a number of the defaults and here are some of them:USB printers that "can't be found" (famously with hplip), and so on.
That's about it for now. As I remember stuff I might put more in here.
- Use sax2 to get that screen resolution just right. This is hugely important on LCDs.
- I turn font hinting, using "slight". This I do through the "Personal Settings" program.
- On openSUSE 10.3, I also switch the "SUSE" menu style to "KDE".
- On an LCD, I manually edit $HOME/.fonts.conf and set the rgba mode to rgb. This makes a big difference - I remember that the KDE font configuration program used to be able to do this but it doesn't appear to work any more - it is disabled for some reason. Why?
- For my personal machines, I switch to "focus follows mouse"
- I install and setup smart. On 10.3, zypper is much too slow and I've encountered weird dependency resolution issues with it. smart is very consistent, faster, and - big in my book - it "just works". It should be noted that 11.0 promises a vastly improved zypper. We'll see.
That's about it for now. As I remember stuff I might put more in here.
Wednesday, February 13, 2008
Acer Laptop: Sound Problems
Tonight I managed to get the sound working on the Acer Aspire 7720-6569 laptop a friend recently purchased. The long and short of it is that I had to acquire updated alsa drivers (and libasound) and userland. openSUSE 10.3 ships with 1.0.14 and it would appear that 1.0.15 (minimum) is required. A stumbling block I'll mention that may help somebody else is that I had to manually rmmod all of the snd* modules and then rcalsasound restart. Additionally, I had to add "model=acer" to the /etc/modprobe.d/sound file that yast wrote. After that the sound worked perfectly.
This is the link I used that may be help to you, too.
openSUSE 10.3: Install problems on Acer Laptop
Last night I ran into some weird problems installing openSUSE 10.3 on a new Acer Aspire laptop - the laptop kept shutting off partway through the install!
The laptop is actually a friend's, and he asked me to "set it up" which means throw openSUSE 10.3 on it, all the updates, a few tweaks and so on. The laptop is an Acer Aspire 7720-6569, which has a mobile Core 2 Duo T5450 at just shy of 1.7 GHz, a big 17" display, 2G of RAM, 160G HDD, blah blah blah. Initially, I was impressed by the light weight (for a physically large laptop), the huge display, and so on. Until you clean up the fonts the display only looks OK but after you fix up the fonts the display is great!
However, partway through the install process (which was insanely fast, estimated 15-20 minutes for a complete install using the KDE desktop), the laptop shut off. I happened to be in another room, so I wasn't exactly sure what happened. Maybe one of the kids did it? Power fluctuation? Then it happened twice more, the last time right in front of me. I had begun to suspect a heat issue, but it wasn't especially warm. For the last boot I made sure to pay attention to the power management system startup and noted that it did not work correctly - apparently the install media has an incomplete or broken power savings subsystem. I posited that, if I could complete an install and reboot that perhaps a new kernel or some bit that isn't on the install media might work better. I managed to get a base system installed and this time the power management stuff worked fine. The laptop did not have another spontaneous shutdown again, over the next 2 days.
What's the upshot of all of this?
- I'll be sending a bug report in to Novell. I've had mixed success with bug reports to Novell, but overall good responses. I'll be updating this entry with the relevant bug number later.
- Acer's BIOS sucks - there was no diagnostic information (temperature, fan speeds, ways to control them, etc...) and in the absence of O/S control, the BIOS just shuts off the computer instead of, oh, turning on or increasing the fan speeds.
UPDATE:
I noticed last night after only half of an hour off of A/C that the LED switched from green to red. What's up with that? According to the battery applet I had over 2 hours left and the machine was totally idle! Chalk that up to a sub-standard BIOS, I guess.
Intel Wireless and openSUSE 10.3
I've worked on two notebooks recently where the built-in Intel Wireless did not work.
Typically, I resorted to using a pair of USB 802.11g adapters I purchased from Zyxel. In this case, the chipset of the USB devices is ZD1211RW. I plug them in and they work perfectly.
However, since one of the laptops was destined for a new user of Linux, I thought - why should this person have to buy/borrow an adapter when the (well-regarded!) Intel wireless is built in?
The built-in stuff seemed to work but it was never able to associate, and repeated attempts merely caused the device to go off into Never-Never Land and Never Come Back.
A brief bit of Googlin' brought me a possible answer - there are two drivers which appear to work, but only one of them actually does. I don't know (or care) about the details in this case, so suffice it to say that if you are having a problem making your Intel Wireless go, and you are running openSUSE 10.3 or can translate the appropriate commands, try this:
rpm -e ipw3945-kmp-default ipw3945d
and make sure you have iwl3945-ucode and iwlwifi-kmp-default installed (along with all dependencies, of course).
Best of Luck!
Friday, December 21, 2007
Coffee
Coffee coffee coffee coffee coffee coffee coffee coffee yummy yummy yummy coffee coffee coffee coffee.
Thursday, December 20, 2007
Why Python?
Why Python?
In a weekend.
Seriously.
If you already know how to program (or think you do), you can pick up enough Python in a day or two to be productive. And productive is the key word, here - it's one of the biggest reasons why I continued with Python. I am more productive in Python than with any other language (insert caveats here). I like the syntax. I like the flow. I like the conciseness of the language. Statements that do what you think they do. No (or few, anyway) hidden surprises. The Python Standard Library is practically the definition of Batteries Included. The Standard Library is solid and hugely comprehensive. I can write an XML-RPC client in 3 lines of code. I can write an XML-RPC server in under a dozen. I can change that server's behavior from one-at-a-time to threaded or forked with one more line. I can crack open an XML file, parse it into a DOM tree, and go rooting about in it in under 5 lines. Sure, the library has some warts, too, but over time many of those warts will be (are being!) removed. The library helps you get real-world things done by making it easy for you to do them: internet protocols galore, interfacing with the filesystem, processing data. The language makes it easy to write my algorithm and structure the program in such a way that, in a year, I can come back to it and be confident that I'll be able to make changes and predict what those changes do. I can hear you saying now that "But that's just a matter of good programming, regardless of the language." and you'd be right. However, the language itself has a say in how you get there! Python doesn't do everything, and what it does it doesn't always do perfectly. Speed is fast, but not C fast. Fast enough that the time saved in development often hugely outweighs the reduced out-and-out speed of the program.
I am a programmer. I have been programming professionally for over 8 years, and I come by it by way of the usual route - I discovered computers (an Apple IIe), learned BASIC, then Pascal, then C. I completed high school "knowing" enough C to be dangerous. I also new a bit of forth and assembly. While in college I learned Shell, Java, JavaScript, Perl, Scheme (a favorite!) and some others. I took classes about programming languages.
Fast Forward a few more years and now I'm working full-time. I'm a Systems Administrator but I also write php, perl, and shell. One of my co-workers introduces me to Python over the course of a lunch one day. I respected this person's opinion quite a bit, but at the time didn't really give it much thought. Right before I was ready to leave the job and take a new one, I took a weekend and learned Python.
Like that.In a weekend.
Seriously.
If you already know how to program (or think you do), you can pick up enough Python in a day or two to be productive. And productive is the key word, here - it's one of the biggest reasons why I continued with Python. I am more productive in Python than with any other language (insert caveats here). I like the syntax. I like the flow. I like the conciseness of the language. Statements that do what you think they do. No (or few, anyway) hidden surprises. The Python Standard Library is practically the definition of Batteries Included. The Standard Library is solid and hugely comprehensive. I can write an XML-RPC client in 3 lines of code. I can write an XML-RPC server in under a dozen. I can change that server's behavior from one-at-a-time to threaded or forked with one more line. I can crack open an XML file, parse it into a DOM tree, and go rooting about in it in under 5 lines. Sure, the library has some warts, too, but over time many of those warts will be (are being!) removed. The library helps you get real-world things done by making it easy for you to do them: internet protocols galore, interfacing with the filesystem, processing data. The language makes it easy to write my algorithm and structure the program in such a way that, in a year, I can come back to it and be confident that I'll be able to make changes and predict what those changes do. I can hear you saying now that "But that's just a matter of good programming, regardless of the language." and you'd be right. However, the language itself has a say in how you get there! Python doesn't do everything, and what it does it doesn't always do perfectly. Speed is fast, but not C fast. Fast enough that the time saved in development often hugely outweighs the reduced out-and-out speed of the program.
Programming is really about solving problems, and in Python the tools to solve your problems are all part of the language. Take iteration, for example. Iteration is a base concept and the language provides iteration as a base feature. Python makes iteration easy. Python makes easy things scandalously easy and hard things easier.
Things in Python are easy to use. No pointers, no references, no hoop-jumping. I hate hoop-jumping. Hoop-jumping is all of that extra stuff that you have to do in order to do something. In C you have memory allocation, tracking, de-allocation. Pointers, arrays, arrays of arrays, and so on (and on and on!) and pretty soon you've got line noise. Segfaults. Buffer overruns.
Python gets out of my way and lets me solve the problem instead of concentrating on the syntax and hoop-jumpery. It lets me spend my precious brain cycles on solving the important stuff - the algorithm, the structure, etc... instead of spending so many ultimately wasted cycles on syntax.
I'm excited and a little frightened by what Python 3000 might bring. I've seen very wonderful changes from Python 1.5 through 2.6, and I hope for more in the future.
Wednesday, December 19, 2007
Some RAID10 performance numbers
I've been using Linux software raid (md) for a very long time - more or less since the beginning, and it's quite honestly been great to me. I've always deployed in a raid5 configuration and never gave much thought to the other levels. Recently, raid10 became available. Not raid1+0 but was is considered (by some) to be a non-standard raid10 implementation which allows a non-even number of components to comprise the (usual) raid1 portion. See the wikipedia article and 'man md' for more details.
Some notes about the hardware and setup:
An alternate chart (thanks due to suggestions received on the Linux-RAID mailing list.), and assuming 'x' is 70 MiB/s, or the speed of one component:
Chart:
Footnotes:
- the kernel is 2.6.22.12, openSUSE "default" kernel
- the CPU is an AMD x86-64, x2 3600+ in power-saving mode (1000 MHz)
- the motherboard is an EPoX MF570SLI which uses the nVidia MCP55 SATA controller (PCIe)
- the drives are 3 different makes of SATA II, 7200 rpm
- each drive is capable of not more than 75 MiB/s (at best - the outermost tracks) and closer to 70 MiB/s the portions of the disk involved in these tests
- each drive is partitioned, identically, into 4 partitions. This test involves the third partition, 4 GiB in size, which is 2 GiB from the start.
- the system was largely idle but does other things
- the system has 1 GiB of RAM
- the raid was created with:
mdadm --create /dev/md2 --level=${level} --raid-devices=3 --spare-devices=0 --layout=${format} --chunk=256 --assume-clean --metadata=1.0 ${DEVICES}
- the deadline I/O scheduler was used on each component drive
- the stripe_sizes and caches, queues sizes, and flusher parameters were left at their defaults
- the caches were dropped before each invocation of 'dd':
echo 3 > /proc/sys/vm/drop_caches - the 'write' portion consisted of this dd invocation:
dd if=/dev/zero of=/dev/md2 bs=256K count=15000 conv=fdatasync - the 'read' portion consisted of this dd invocation:
dd if=/dev/md2 of=/dev/null bs=256K count=15000 - I did not test any chunk size other than 256K but probably will in the future.
- I can supply the entire test script if necessary (I intend on doing this in the future, after some additional refinement.)
| level | format | Writing | Reading | Writing (Degraded) | Reading (Degraded) |
| raid5 | left-asymmetric | 55 | 129 | 46 | 124 |
| raid5 | left-symmetric | 54 | 123 | 50 | 122 |
| raid5 | right-asymmetric | 54 | 124 | 49 | 124 |
| raid5 | right-symmetric | 54 | 128 | 49 | 116 |
| raid10 | n2 | 103 | 95 | 103 | 104 |
| raid10 | o2 | 102 | 94 | 100 | 102 |
| raid10 | f2 | 97 | 162 | 97 | 51 |
| raid0 | - | 205 | 186 | n/a | n/a |
| level | format | Writing | Reading | Writing (Degraded) | Reading (Degraded) |
| raid5 | left-asymmetric | 0.8 | 1.8 | 0.7 | 1.8 |
| raid5 | left-symmetric | 0.8 | 1.8 | 0.7 | 1.7 |
| raid5 | right-asymmetric | 0.8 | 1.8 | 0.7 | 1.8 |
| raid5 | right-symmetric | 0.8 | 1.8 | 0.7 | 1.7 |
| raid10 | n2 | 1.5 | 1.4 | 1.4 | 1.4 |
| raid10 | o2 | 1.5 | 1.4 | 1.4 | 1.4 |
| raid10 | f2 | 1.4 | 2.3 | 1.4 | 0.7 |
| raid0 | - | 3.0 | 2.8 | n/a | n/a |
What do these numbers tell us?
(Of course, these observations only apply for 3 drives in this configuration. Caveat, handwaving.)- Degraded read speed on raid5 is 85-90% of non-degraded. That's pretty good.
- Degraded writing on raid5 is virtually indistinguishable for non-degraded.
- raid10 near and offset performance, reading or writing, degraded or not, is very consistent.
- raid10 far layout has awesome read performance (non-degraded) - I'd ballpark it near raid0 performance. Degraded, however, shows much worse performance. Why?
- raid10 far layout has no discernable performance difference when writing in degraded mode.
Footnotes:
- Whenever possible, always use conv=fdatasync for tests like this. I'll explain why in a future installment.
- I did not use iflag=direct (which sets O_DIRECT).
A few notes on Linux
I've been doing Linux since late 1994 when I defected from FreeBSD. I tried RedHat, Debian, and Slackware. I settled into (not the same as settling for) Debian, and didn't really use any other distro until 2000 or so. I played with Gentoo and a host of others. I learned a great many things - expanded my horizons as they say. RedHat, no offense to their great work, never really did much for me. SuSE, on the other hand, seemed to really fit the bill and I've been using SuSE more or less exclusively since 9.1 or so. Like everybody, they've made some missteps but their openSUSE project, now at 10.3, shaped up very nicely. If you want to be really impressed, go look at their Build Service.
Some people get really passionate about their distros - I have certainly been one of them. These days, if you squint your eyes a bit, most of them do most of the same things mostly the same way. Most of the time. This, I think, is a great validation of the system. Proof, if you will, that mere hobbyists can do really great things. Somebody had an itch and it got scratched. Maybe all they could do was file a bug saying "I wish this thing worked better." and somebody else fixed it. Maybe they work on the kernel, or some bit of software, or whatever. The whole idea is that a jillion people doing one small thing really adds up.
I should say that "Hobbyists" is not intended to disparage the really outstanding work that many, many commercial entities have put into Linux, both front-stage and behind the curtain. Too many people conflate the perceived ideologies of Freedom-loving Folk with anti-commercial sentiments. I have nothing against commercial involvement in Linux or even *gasp* paying for boxed copies of Linux (or games or books or ...). True, I downloaded the isos like everybody else, and I'll probably never use those manuals that came with it, but I've put my money where I thought it would do some good.
The point is not to say "my OS is better than your OS" - not at all. That's like saying that my choice of iced cream is more valid than yours. However, Linux is more than a philosophical choice, it's also a practical one - I can get more done in less time with less "in my way" with the Linux distro of my choice than I could with anything else, and that results in less frustration (a happier Jon), more work done (yay!). Hopefully along the way I can do my bit, too.
Intro
This blog is an experiment. I've been debating having a blog for, oh, nigh on a year now. More, perhaps. This blog may contain personal opinions, technology, grumps and gripes, but will probably involve Python, Linux, Software RAID (md), LVM, Block Devices, shell, and probably a whole host of other things, too.
Subscribe to:
Posts (Atom)
