Bingbot

Bingbot is a web-crawling robot (type of internet bot), deployed by Microsoft October 2010 to supply Bing.^[1] It collects documents from the web to build a searchable index for the Bing (search engine). It performs the same function as Google's Googlebot.

A typical user agent string for Bingbot is " Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)". This appears in the web server logs to tell the webmaster who is requesting a file. Each webmaster is able to use the included agent identifier, "bingbot", to disallow or allow access to their site (by default access is allowed). If they don't want to grant access they can use the Robots Exclusion Standard to block it (relying on the assumed good behaviour of bingbot), or use other server specific means (relying on the web server to do the blocking).

References

↑ "BingBot Crawl Activity Surging?". Retrieved 2016-07-16.

External links

Web crawlers

Internet bots designed for Web crawling and Web indexing

Active	80legs bingbot Fetcher Googlebot Heritrix HTTrack Pandemonium_(Webcrawler) PHP-Crawler PowerMapper Wget

Discontinued	FAST Crawler msnbot RBSE TkWWW robot Twiceler Yahoo! Slurp

Types	Distributed web crawler Focused crawler ICDL crawler

This article is issued from Wikipedia - version of the 7/29/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.