I have the optical cable now to connect the DAT player to the MacBook, but I can't get a signal. Same problem when I hook it up to the line in on my Sony PCM-D50 recorder, which also has optical in. After reading around, I'm starting to think the problem is that the tapes were recorded on long play and thus at a lower rate that's not supported by the MacBook or the recorder (both of which have floors of 44.1 kHz).
The long-play mode on DAT machines uses a 32kHz sample rate with 12-bit non-linear encoding and a frequency response that extended to about 14.5kHz.
The issue may well be to do with the 32kHz sample rate being incompatible with your Mac, but I wonder also if the machine can output S/PDIF in this format at all because of the need to transcode between 12-bit non-linear and the 16 bit linear format required by everything else.
Edited to add: The other possibility is that the recording has been flagged as copyrighted and the SCMS system is preventing anything else from recording the data digitally.
Given the inherently limited quality of the source recordings in long-play mode, I would suggest that taking the analogue outputs (most DAT machines can replay long-play DATs even if they can't record them) and re-recording through a normal computer interface via the analogue domain might be the most pragmatic way forward.
Technical Editor, Sound On Sound