Google-Agent
AI AgentVerify Google-Agent IP Address
Verify if an IP address truly belongs to Google, using official verification methods. Enter both IP address and User-Agent from your logs for the most accurate bot verification.
Google-Agent is a user-agent used by AI agents running on Google infrastructure to navigate the web and perform actions on behalf of users. It appears in scenarios where agent-based systems (such as experimental projects like Project Mariner) fetch webpages, interact with content, or complete tasks requested by users. This traffic is user-driven and task-specific, not a general-purpose crawler or indexing bot. Requests are typically targeted and may involve multi-step interactions rather than simple page fetches. It's activity is usually low to moderate depending on user demand. It does not directly contribute to Google Search indexing. Google-Agent bot does not respect robots.txt rules. RobotSense.io verifies Google-Agent using Google’s official validation methods, ensuring only genuine Google-Agent traffic is identified.
User Agent Examples
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; Google-Agent; +https://developers.google.com/crawling/docs/crawlers-fetchers/google-agent) Chrome/W.X.Y.Z Safari/537.36Robots.txt Configuration for Google-Agent
No Robots.txt Identifier
Google-Agent does not have a unique robots.txt User-Agent identifier, which means this bot cannot be specifically targeted in your robots.txt file.
Looking to detect or manage this bot? RobotSense.io provides real-time bot detection and management beyond robots.txt, helping you identify and control bots that cannot be blocked through traditional means.
Frequently Asked Questions
- What is Google-Agent, and why is it visiting my website?
- Google-Agent is a user-agent associated with AI-driven agents operated by Google that perform web actions on behalf of users. These agents fetch webpages, navigate content, or complete multi-step tasks triggered by explicit user requests. Unlike traditional crawlers, it does not perform broad indexing and instead targets specific URLs relevant to a task. Google-Agent traffic is expected on public websites but is typically limited and user-driven.
- Is Google-Agent a legitimate bot, or is it commonly spoofed?
- Google-Agent is an official Google user-agent, but it can be spoofed like other well-known bots. Attackers may imitate it to bypass bot filters or disguise automated activity as legitimate traffic. Because of this, the presence of the Google-Agent user-agent string alone is not sufficient to confirm authenticity. Verification must rely on network-level checks rather than headers. You can use Google's recommended methods mentioned below to verify a legitimate visit, or use RobotSense.io API to easily verify Google-Agent visits.
- How can I verify that a request is really coming from Google-Agent?
- You can use Google's recommended official methods to verify Google-Agent bot visits, these include: - IP range checks - Reverse DNS → forward DNS Do not use User-Agent based detection as that can be easily spoofed. Alternatively, you can use RobotSense.io API to easily verify Google-Agent bot and all other bots from Google.
- Should I allow or block Google-Agent on my website?
- Allowing Google-Agent is generally optional and depends on whether you want your content accessible to user-driven AI agents. It does not impact search indexing or SEO directly. Blocking may be appropriate if: - Your infrastructure cannot handle automated interactions - Content is sensitive or not intended for automated access - APIs or transactional endpoints should not be accessed by agents For most public content, allowing it poses minimal risk, but control may be needed for dynamic or sensitive systems.
- How can I control or block Google-Agent using robots.txt or other methods?
- You cannot add a rule in your robots.txt to control Google-Agent bot, as this crawler has no specific robots.txt user-agent. However, you can use controls in your WAF, or in RobotSense enforcement settings to manage the bot behavior.
- How often does Google-Agent crawl websites, and can it impact server performance?
- Google-Agent operates in a user-driven, event-based manner rather than continuous crawling. Requests occur when users trigger tasks that require fetching or interacting with web content. As a result: - Request frequency varies based on user demand - Traffic is typically low to moderate - Impact on bandwidth and server load is usually minimal However, multi-step interactions may generate multiple sequential requests for a single task. Some administrators choose to rate-limit or restrict it.
- What happens if I block Google-Agent? SEO, visibility, and feature impact explained.
- Blocking Google-Agent does not affect search rankings or indexing in Google Search. However, it may limit how Google-powered agents interact with your site: - Users may not be able to complete agent-driven tasks involving your website - Reduced compatibility with emerging AI-assisted browsing features The impact is primarily on user experience within agent-based systems, not visibility in search results.
- Does Google-Agent collect, scrape, or use my content for training or reuse?
- Google-Agent accesses page content as needed to complete user-requested tasks, such as navigation or information retrieval. It is not a general-purpose indexing crawler and does not maintain a search index. There is no public documentation confirming its use for AI training or dataset collection. Its activity is limited to task-specific content retrieval, and any data usage is tied to fulfilling the user’s request rather than broad content reuse.
Other Google Bots
Google operates other crawlers you may also need to configure.
AdsBot
AdsAdsBot-Google is Google’s crawler responsible for evaluating landing pages used in Google Ads campaigns. It performs desktop-focused checks on page quality, load speed, relevance, and policy compliance. These assessments directly influence ad quality scores, cost efficiency, and overall eligibility. Blocking AdsBot prevents Google from reviewing landing pages, which can degrade or disable ad performance. Crawl activity is selective and tied to active or recently modified ad campaigns rather than broad indexing. Its purpose is to ensure that advertisers maintain fast, trustworthy, and policy-compliant landing pages. It ignores the global user agent (*) rule. RobotSense.io verifies AdsBot/AdsBot-Google using Google’s official validation methods, ensuring only genuine AdsBot/AdsBot-Google traffic is identified.
AdsBot Mobile Web
Ads[This crawler is officially retired as per Google] AdsBot-Google-Mobile bot is Google’s mobile-focused crawler used to evaluate the landing page experience for Google Ads. It simulates mobile device conditions to assess page quality, load performance, mobile usability, and policy compliance. These evaluations directly influence Google Ads quality scores and ad eligibility. If you run ads, blocking it may negatively affect ad performance because Google cannot verify the mobile landing page experience. Crawl activity is targeted and low-volume, triggered when ads are created, updated, or actively running. Its purpose is ensuring advertisers provide fast, compliant, and user-friendly mobile pages. It ignores the global user agent (*) rule. RobotSense.io verifies AdsBot Mobile Web/AdsBot-Google-Mobile using Google’s official validation methods, ensuring only genuine AdsBot Mobile Web/AdsBot-Google-Mobile traffic is identified.
AdSense / Mediapartners-Google
AdsMediapartners-Google is Google’s crawler dedicated to evaluating webpages for Google AdSense. It scans pages to understand content, layout, and context so Google can deliver relevant ads and optimize revenue for publishers. Unlike Googlebot, this crawler does not index content for Search - its role is purely advertising-related. Blocking it may prevent AdSense from analyzing pages and serving targeted ads effectively. Crawl activity is generally light and focused on pages where AdSense code is present, helping Google match ad inventory with page themes and user interests. It ignores the global user agent (*) rule. RobotSense.io verifies AdSense / Mediapartners-Google using Google’s official validation methods, ensuring only genuine AdSense / Mediapartners-Google traffic is identified.
APIs-Google
Developer ToolsAPIs-Google is a service crawler used by Google to verify and interact with endpoints tied to various Google APIs. It is typically triggered when applications, scripts, or integrations using Google services need to fetch or validate external resources. Common use cases include OAuth flows, link previews, data validation, push notification messages, and API-driven checks performed on behalf of Google products. Crawl activity is usually low-volume and event-driven, reflecting specific API operations rather than broad crawling or indexing associated with Google Search. It ignores the global user agent (*) rule. RobotSense.io verifies APIs-Google using Google’s official validation methods, ensuring only genuine APIs-Google traffic is identified.
Chrome Web Store / Google-CWS
Developer ToolsChrome Web Store fetcher or Google-CWS is a Google user-agent associated with Chrome Web Services, typically used for link preview generation, safe browsing checks, and content fetching triggered by Chrome features. It performs lightweight requests to retrieve metadata, page titles, favicons, and safety signals from URLs that developers provide in the metadata of their Chrome extensions and themes. This bot is not a search crawler and does not influence Google Search indexing or rankings. Activity occurs when Chrome or Google services need to quickly inspect a URL for previews, safety evaluation, or rendering behavior. It ignores robots.txt rules. RobotSense.io verifies Chrome Web Store fetcher using Google’s official validation methods, ensuring only genuine Chrome Web Store fetcher traffic is identified.
DuplexWeb-Google
Others[This crawler is officially retired as per Google] DuplexWeb-Google is a Google crawler associated with Duplex and Assistant-related technologies that fetch web content to help generate conversational responses and perform task-oriented actions. It retrieves page information needed to understand structured data, business details, menus, appointment flows, and other interactive elements. Crawl activity is selective and generally tied to user-initiated tasks or systems that prepare content for automated assistance. Its purpose is to support natural-language interactions by ensuring Google’s assistant technologies can interpret and use real-time webpage information accurately. It ignores the global user agent (*) rule. RobotSense.io verifies DuplexWeb-Google using Google’s official validation methods, ensuring only genuine DuplexWeb-Google traffic is identified.
Similar Bots
Other AI Agent bots from different operators.