APIs-Google

AdsBot-Google is Google’s crawler responsible for evaluating landing pages used in Google Ads campaigns. It performs desktop-focused checks on page quality, load speed, relevance, and policy compliance. These assessments directly influence ad quality scores, cost efficiency, and overall eligibility. Blocking AdsBot prevents Google from reviewing landing pages, which can degrade or disable ad performance. Crawl activity is selective and tied to active or recently modified ad campaigns rather than broad indexing. Its purpose is to ensure that advertisers maintain fast, trustworthy, and policy-compliant landing pages. It ignores the global user agent (*) rule. RobotSense.io verifies AdsBot/AdsBot-Google using Google’s official validation methods, ensuring only genuine AdsBot/AdsBot-Google traffic is identified.

[This crawler is officially retired as per Google] AdsBot-Google-Mobile bot is Google’s mobile-focused crawler used to evaluate the landing page experience for Google Ads. It simulates mobile device conditions to assess page quality, load performance, mobile usability, and policy compliance. These evaluations directly influence Google Ads quality scores and ad eligibility. If you run ads, blocking it may negatively affect ad performance because Google cannot verify the mobile landing page experience. Crawl activity is targeted and low-volume, triggered when ads are created, updated, or actively running. Its purpose is ensuring advertisers provide fast, compliant, and user-friendly mobile pages. It ignores the global user agent (*) rule. RobotSense.io verifies AdsBot Mobile Web/AdsBot-Google-Mobile using Google’s official validation methods, ensuring only genuine AdsBot Mobile Web/AdsBot-Google-Mobile traffic is identified.

Mediapartners-Google is Google’s crawler dedicated to evaluating webpages for Google AdSense. It scans pages to understand content, layout, and context so Google can deliver relevant ads and optimize revenue for publishers. Unlike Googlebot, this crawler does not index content for Search - its role is purely advertising-related. Blocking it may prevent AdSense from analyzing pages and serving targeted ads effectively. Crawl activity is generally light and focused on pages where AdSense code is present, helping Google match ad inventory with page themes and user interests. It ignores the global user agent (*) rule. RobotSense.io verifies AdSense / Mediapartners-Google using Google’s official validation methods, ensuring only genuine AdSense / Mediapartners-Google traffic is identified.

Chrome Web Store fetcher or Google-CWS is a Google user-agent associated with Chrome Web Services, typically used for link preview generation, safe browsing checks, and content fetching triggered by Chrome features. It performs lightweight requests to retrieve metadata, page titles, favicons, and safety signals from URLs that developers provide in the metadata of their Chrome extensions and themes. This bot is not a search crawler and does not influence Google Search indexing or rankings. Activity occurs when Chrome or Google services need to quickly inspect a URL for previews, safety evaluation, or rendering behavior. It ignores robots.txt rules. RobotSense.io verifies Chrome Web Store fetcher using Google’s official validation methods, ensuring only genuine Chrome Web Store fetcher traffic is identified.

[This crawler is officially retired as per Google] DuplexWeb-Google is a Google crawler associated with Duplex and Assistant-related technologies that fetch web content to help generate conversational responses and perform task-oriented actions. It retrieves page information needed to understand structured data, business details, menus, appointment flows, and other interactive elements. Crawl activity is selective and generally tied to user-initiated tasks or systems that prepare content for automated assistance. Its purpose is to support natural-language interactions by ensuring Google’s assistant technologies can interpret and use real-time webpage information accurately. It ignores the global user agent (*) rule. RobotSense.io verifies DuplexWeb-Google using Google’s official validation methods, ensuring only genuine DuplexWeb-Google traffic is identified.

Feedfetcher is Google’s crawler responsible for retrieving RSS and Atom feeds used in Google News, Google Reader (historically), and other syndication-based services. It fetches feed URLs rather than full webpages. The bot does not index content for Google Search and does not follow links within feeds; its role is purely to collect updates for subscribed users or Google systems that aggregate feed content. Most publishers allow it to ensure timely distribution of updates. Crawl activity is periodic and lightweight, triggered when feed subscribers or internal services request refreshes. It ignores robots.txt rules. RobotSense.io verifies Feedfetcher using Google’s official validation methods, ensuring only genuine Feedfetcher traffic is identified.