Do you need a click track?
If so, are you happy to have your backings in Mono?
If No, or Yes/Yes then stereo is all you need and an iPod would do the job. You could save the files as 44.1kHz/16bit .wav or .aiff for higher quality (though iTunes lossless system is pretty good). If you have an iPod touch you could use the 'SoundCue' App for single track playback.
Next step up would be a dedicated flash drive player, which would enable you to use a foot pedal for start/stop.
Or as has been said a Mac with an interface.
I'm hoping someone will develop a generic programmable foot switch for iPhone/iPad as the SoundCue app on iPad would be ideal for my purposes.