Googlebot News
SearchVerify Googlebot News IP Address
Verify if an IP address truly belongs to Google, using official verification methods. Enter both IP address and User-Agent from your logs for the most accurate bot verification.
Googlebot-News is Google’s crawler dedicated to discovering and indexing news content for Google News and Top Stories. The bot focuses on timely, high-quality journalism, scanning article pages, structured data, headlines, timestamps, authorship, and metadata to assess relevance and freshness. Crawling is more frequent than standard Googlebot, reflecting the need for rapid updates. Its role is to ensure accurate, real-time coverage of news sources across Google’s search and news platforms. RobotSense.io verifies Googlebot News using Google’s official validation methods, ensuring only genuine Googlebot News traffic is identified.
User Agent Examples
Googlebot Smartphone: Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot Desktop: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Googlebot/2.1; +http://www.google.com/bot.html) Chrome/W.X.Y.Z Safari/537.36
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
Googlebot/2.1 (+http://www.google.com/bot.html)Robots.txt Configuration for Googlebot News
Googlebot-NewsUse this identifier in your robots.txt User-agent directive to target Googlebot News.
Recommended Configuration
Our recommended robots.txt configuration for Googlebot News:
User-agent: Googlebot-News
Allow: /Completely Block Googlebot News
Prevent this bot from crawling your entire site:
User-agent: Googlebot-News
Disallow: /Completely Allow Googlebot News
Allow this bot to crawl your entire site:
User-agent: Googlebot-News
Allow: /Block Specific Paths
Block this bot from specific directories or pages:
User-agent: Googlebot-News
Disallow: /private/
Disallow: /admin/
Disallow: /api/Allow Only Specific Paths
Block everything but allow specific directories:
User-agent: Googlebot-News
Disallow: /
Allow: /public/
Allow: /blog/Set Crawl Delay
Limit how frequently Googlebot News can request pages (in seconds):
User-agent: Googlebot-News
Allow: /
Crawl-delay: 10Note: This bot officially honors the Crawl-delay directive.