Google has updated its crawler help documentation to add a new section for HTTP caching, which explains how Google's crawlers handle cache control headers. Google also posted a blog post begging us to ...

Google announced an update to their crawler documentation, adding more information about caching which should help better understand how to optimize for Google’s crawler. By following the new ...