So is there really a big difference between the "regular" HD24 and the HD24XR other than 96K capable?
As I know you're definitely a man not to want your signal paths compromised or degraded if at all possible , then I'd say yes - the converters in the HD24XR are worth the extra outlay (even for 44.1-48kHz work). Even at current prices, you'll get 24 AD/DA and a 24 track recorder at the same price that similar (or even lesser!) converters alone are going for...
Of course, if you're doing your conversion before hitting the recorder, then there would be no advantage in the XR over the standard model.
Just make sure you factor in the FirePort, as you'll need that for the transfer of tracks to a PC (other methods are painfully slow in comparison).
"It's not pretty, also you can't dance to it." - Frank Zappa