The Web-walking part of a search engine that collects pages for indexing in the search engine's database. Also called a bot. The best-known spider is Scooter, the web-walker for the Alta Vista search engine.
Posted by Jargon File at May 15, 2003 10:29 AM