How To Choose Your Google Crawl Rate

Writing by Nick Stamoulis Tweet

Did you know you can change your Google crawl rate? It’s true and you can do it in Google Webmaster Tools if you have an account.

The default setting for the Google crawl rate is to let Google determine how often it crawls your website. However, if you have special needs then you can set your own crawl rate. But I wouldn’t recommend changing this unless you understand the special needs of your server and you are an advanced webmaster.

To change your crawl rate, log in to your Google Webmaster Tools account and click on Settings. In these settings you can establish a geographic targeting area, set a preferred domain (using www or without www), and choosing a customized crawl rate. Find the crawl rate button and set it to “Set custom crawl rate”. Below the crawl rate button you’ll see two settings – requests per second and seconds between requests. Select the settings you desire that are faster or slower than the default rate and click “Save”.

If you are experiencing higher than expected volumes of traffic then you’ll want to have Google crawl your site less often. You may even want to slow down the requests per second. Slowing down your crawl rate will allow you to handle the increased traffic better on your site and this is a particularly handsome way to let Google know that you are seeing a spike in traffic due to increased PPC marketing or link bait that is working. If your site is not getting crawled or it is taking an extra long time to get indexed then you can increase your settings and invite Google to spend more time on your site to increase the number of your indexed pages.