Fetch as Bingbot is a very useful tool for troubleshooting. You will find this tool in the Diagnostics & Tools section in the navigation menu. Fetch as Bingbot allows you to see exactly what the Bingbot sees when downloading a page from your site. As such, it’s a great tool for determining if a page on your site can be crawled. To use it, simply enter in the URL from your site and click Fetch. This tool will send the request to Bingbot for processing, so it’s normal that it takes a few seconds to run your request. Just below the URL entry form, the URL you requested (along with any previous requests) will be shown. When Bingbot is finished with your request, the status for the URL will change to Completed. To see what Bingbot found at your URL, simply click the Completed link and scroll down the page to view the page source that Bingbot found.
See what Bingbot sees
Fetch as Bingbot shows you exactly what your HTTP Headers and page source look like to Bingbot. Viewing your page’s source in this manner ensures you know exactly what Bingbot is seeing when it crawls the URL. This is especially useful in helping find areas when you want to make sure content can be indexed or in cases where your site may have been compromised and is for example sending different HTML to Bingbot then to users. By scanning through the page source, you could note things like links injected into blog posts or any other manner of additive items you did not place on your website.
Fetch as Bingbot is also a reliable way to test if a URL is being blocked by your robots.txt file. It’s polite, so sending it to fetch a URL blocked via robots.txt will get you a notice explaining that this barrier is in place.
Also, if Fetch as Bingbot is not able to crawl a page because of other blocks or politeness constraints, for example the maximum amount of crawls we are allowed to make in a given time window, the tool will tell you too.