[ale] need web spider

[Armsby John-G16665]

Anyone have any recommendations for a web spider which indexes filenames, and content?  I have several large file systems (NT) which are part of a web document file system (no html, just a big bag if files), some classic web sites, and some HPUX web sites composed of a bunch of raster, pdf, and Office files.
My volume is probably about 500,000 files.

I see a lot of stuff out there but....... 


John


---
This message has been sent through the ALE general discussion list.
See http://www.ale.org/mailing-lists.shtml for more info. Problems should be 
sent to listmaster at ale dot org.