A web crawler, sometimes called a spider or spiderbot and often shortened to crawler, is an Internet bot that systematically browses the World Wide Web and that is typically operated by search engines for the purpose of Web indexing (web spidering).
- A software robot, called a spider or crawler, automatically fetches sites all over the Web, reading pages and following associated links. 一个 叫做 蜘蛛 或 爬虫 的 软件 机器人 自动地 在 整个 Web 上 取出 站点， 阅读 页面 和 跟随 相关 的 链接。
A web crawler, also known as a spider, is an automated program or bot predominantly used by search engines like Google and Bing to explore and catalog web content across the Internet.
What is a web crawler? | How web spiders work - Cloudflare
crawler noun [C] (INTERNET) internet & telecoms specialized (also web crawler) a computer program that automatically searches for information on the internet, usually in order to index (= list) internet content
A web crawler, crawler or web spider, is a computer program that's used to search and automatically index website content and other information over the internet.
What is a Web Crawler? Everything you need to know from ... - TechTarget
A web crawler is a digital search engine bot that uses copy and metadata to discover and index site pages. Also referred to as a spider bot, it "crawls" the world wide web (hence "spider" and "crawler") to learn what a given page is about.
An example of a chassis is a vehicle frame, the underpart of a motor vehicle, on which the body is mounted; if the running gear such as wheels and transmission, and sometimes even the driver's seat, are included, then the assembly is described as a rolling chassis.