Note: This isn't actually that hard. It took me quite a bit of fux0ring about though because of some user stupidity (ie, my stupidity).
The Epson Perfection 1660 is a nice cheap-ish ($400) USB2 scanner. It works perfectly with Linux. It's well supported by recent versions of the 
SANE Epson backend.
The USB2.0 layer in Linux 2.4.x isn't exactly rock solid. I'd class it as experimental so YMMV. Make sure you have a reasonable recent kernel. I'm working with 2.4.18 (RedHat's current on RH7.1). I had some very weird problems trying to use an old USB extension cable with the new scanner. It locked up my box and when it didn't there were big delays in detecting devices and stuff. I have a VIA KT333 motherboard with VIA USB2 controller. Check out http://www.linux-usb.org for more info about USB support in Linux.
For linux to recognize that I have USB2 on my motherboard I have to load the ehci-hcd USB driver with:
modprobe ehci-hcd
With redhat you can add an alias to /etc/modules.conf like this:
alias usb-controller-1 ehci-hcd
The "scanner" USB driver in 2.4.18 (and I think later versions up to 2.4.20) doesn't recognized the Epson's USB device ID's out of the box. So you have to force the issue with driver options. The options are:
modprobe scanner vendor=0x04b8 product=0x011e
Or with /etc/modules.conf options:
options scanner vendor=0x04b8 product=0x011e
See the linux kernel docs (usr/src/linux-2.4/Documentation/usb/scanner.txt) for more info on the module options.
If you have a recent version of SANE installed you should be able to run sane-find-scanner as root and have it correctly identify the scanner as attached to /dev/usb/scanner0.
I figured that the sane RPM's I had (Ximian Gnome 1.4) would be a little out of date so I built my own from source. I grabbed some existing RH8.0 SRPMS and tweaked the .spec files to build the newer sane-backends and sane-frontends on my RH7.1 and RH7.3 systems. I also patched the backends package with the very latest Epson backend from the driver developer's site.
The build seemed to work well but somehow I managed to delete the epson backend .so's. I have no idea how this happened but it took me forever to figure out why I couldn't talk to the scanner. I just re-installed my RPMS and things started to work again. But I digress...
Using the SANE_DEBUG environment variables is extremely useful in finding out what is going wrong (if things aren't working). eg:
export SANE_DEBUG_DLL=128; export SANE_DEBUG_EPSON=128; scanimage -L
Use lower numbers than 128 to reduce debugging output detail.
Anyway, now that everything is functioning as it should I'm very impressed with the scanner. It's very fast (USB2 is cool) with good picture quality and resolution (1600dpi). It's much better than my old Agfa SnapScan? 1212U. Though the SnapScan? also works quite well with Linux.
Wowzers. Epson actually provide a Linux Scanning app! It's called "Image Scan! for Linux". You can get it here: http://www.epkowa.co.jp/english/linux_e/linux.html
I believe it's licensed under the GPL and is actually just a front end to SANE. I think it has a nicer UI than XSane. Not as feature packed but better for newbies or if you want to use LinuxForYourMum.
Part of HardwareEpson
One page links to HardwareEpsonPerfection1660:
