Well, that was unproductive.
It's hard for me to know what changed, from a Software perspective, between your successful/fast runs and the slow ones. If the answer is "nothing", it seems quite likely that the problem is some kind of bizarre hardware failure.
I'd do any subsequent backups with Console open to the System log, and look in there for any unusual activity.
To diagnose hardware, I'd try to isolate aspects of the drive first -- verify its firmware version, swap out the FireWire cable, try a different port.
If none of that helps, it sounds like something in the iMac is problematic. That isn't something I've heard of in the rev. b units -- but anything is possible. If you've got another potential destination volume, it might be helpful to try backing up to that.
I'd also try booting in Safe mode -- power off and keep Shift held down until you get to the login panel or the desktop -- and then attempt the backup from there. That will help to isolate any low-level software that'd be running.
Sorry that's a bit rambling and scattershot, but it's what I can come up with!