Yandex

YaDirectFetcher is a Yandex crawler used to fetch landing pages and assets for advertising purposes within the Yandex.Direct ecosystem. It retrieves content to evaluate ad relevance, page availability, redirects, and compliance with advertising policies. This crawler supports both initial ad review and ongoing campaign monitoring. It is not a general search crawler and does not index content for Yandex Search. Blocking it may disrupt ad approval or delivery. Crawl activity is targeted and low-volume, typically triggered by ad creation, updates, or periodic quality checks across Yandex.Direct campaigns. It does not use the robots.txt at all and thus global robots.txt user agent (*) rule and its own directives are completely ignored. RobotSense.io verifies YaDirectFetcher using Yandex’s official validation methods, ensuring only genuine YaDirectFetcher traffic is identified.

YandexAccessibilityBot is a crawler operated by Yandex to evaluate websites for accessibility and usability compliance. It scans pages to assess factors such as semantic structure, text alternatives, contrast, and navigability for assistive technologies. The insights gathered help Yandex improve accessibility-related features and guidance within its products. This bot does not perform full search indexing and does not directly influence Yandex search rankings. It ignores the global robots.txt user agent (*) rule. Crawl activity is generally lightweight and focused on representative pages rather than comprehensive site-wide crawling. As per Yandex, it sends up to 3 requests to the site per second. The robot ignores the setting in Yandex Webmaster. RobotSense.io verifies YandexAccessibilityBot using Yandex’s official validation methods, ensuring only genuine YandexAccessibilityBot traffic is identified.

YandexAdNet is Yandex’s advertising-related crawler used to evaluate webpages that participate in the Yandex Advertising Network. It fetches pages to analyze content, layout, ad placement, and policy compliance. These checks help Yandex determine ad relevance, safety, and monetization eligibility for publisher sites. The bot is not a general search crawler and does not index content for Yandex Search. Blocking it may limit ad review or delivery. Crawl activity is targeted and low-volume, typically triggered by publisher onboarding, configuration changes, or ongoing ad quality assessments. RobotSense.io verifies YandexAdNet using Yandex’s official validation methods, ensuring only genuine YandexAdNet traffic is identified.

YandexAdditional is a Yandex service crawler used to interpret and apply robots.txt rules that control whether indexed page content can be used in Yandex AI-generated responses. It operates only on pages that have already been indexed by Yandex’s primary crawler and does not request new pages or trigger indexing. The bot analyzes access directives to ensure content usage complies with site owner preferences in AI features. It does not perform independent crawling. Its activity is internal and policy-driven, supporting correct enforcement of robots-based restrictions for Yandex’s AI response systems. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexAdditional using Yandex’s official validation methods, ensuring only genuine YandexAdditional traffic is identified.

YandexAdditionalBot is a Yandex service crawler used to interpret and apply robots.txt rules that control whether indexed page content can be used in Yandex AI-generated responses. It operates only on pages that have already been indexed by Yandex’s primary crawler and does not request new pages or trigger indexing. The bot analyzes access directives to ensure content usage complies with site owner preferences in AI features. Its activity is internal and policy-driven, supporting correct enforcement of robots-based restrictions for Yandex’s AI response systems. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexAdditionalBot using Yandex’s official validation methods, ensuring only genuine YandexAdditionalBot traffic is identified.

YandexBlogs is Yandex’s crawler dedicated to discovering and indexing blog-style content for Yandex’s blog and social content aggregations. It focuses on comments in posts, articles, feeds, and frequently updated pages rather than full websites. The bot analyzes text, timestamps, authorship, and update frequency to surface fresh and relevant blog content. Crawl activity is periodic and content-focused, prioritizing sources that publish regularly. Its purpose is to keep Yandex’s blog-oriented services current and reflective of ongoing discussions across the web. RobotSense.io verifies YandexBlogs using Yandex’s official validation methods, ensuring only genuine YandexBlogs traffic is identified.

YandexBot is Yandex’s primary web crawler, responsible for discovering, fetching, and indexing content for Yandex Search. It crawls pages to analyze text, links, metadata, structured data, and media assets. Crawl frequency depends on site popularity, update cadence, and internal Yandex signals. YandexBot supports multiple verticals, including images, videos, and news, through specialized variants. Its role is comparable to Googlebot or Bingbot - keeping Yandex’s search index fresh, accurate, and aligned with user queries across Yandex’s ecosystem. YandexBot also has a specialized mode called MirrorDetector that is used to determine the site's alternate addresses across the web. It likely fetches page content, structure, and metadata to compare similarities between domains and identify sites that replicate content with minimal variation. This analysis likely helps Yandex combat content duplication, mirror networks, and abusive SEO practices. This mode is not used for general search indexing and does not surface content directly in search results. Crawl activity with this mode is targeted and investigative, focusing on suspected duplicates rather than comprehensive site-wide crawling. RobotSense.io verifies YandexBot using Yandex’s official validation methods, ensuring only genuine YandexBot traffic is identified.

YandexCalendar is a Yandex crawler used to retrieve event-related information for calendar and scheduling features within Yandex services. It downloads calendar files by users' requests. These files are often located in directories prohibited from indexing. It may fetch pages that contain event details such as dates, times, locations, descriptions, and structured data (for example, event schema or calendar feeds). These requests are typically purpose-driven, triggered when events are submitted, referenced, or discovered for inclusion in Yandex’s products. Crawl activity is low-volume and focused, aimed at keeping event listings accurate, timely, and up to date rather than performing broad web indexing. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexCalendar using Yandex’s official validation methods, ensuring only genuine YandexCalendar traffic is identified.

YandexComBot is a Yandex crawler responsible for indexing web content for Yandex Search in languages other than Russian. It crawls pages to discover and index multilingual content for international search results. The bot can index pages unless there is an explicit robot-specific restriction defined in robots.txt. It respects standard crawl directives and follows Yandex’s indexing policies for non-Russian content. Crawl activity is comparable to YandexBot but focused on international sites and language variants. Its purpose is to keep Yandex’s global search index accurate and accessible across multiple languages and regions. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexComBot using Yandex’s official validation methods, ensuring only genuine YandexComBot traffic is identified.

YandexDirect is a Yandex crawler used to evaluate landing pages associated with ads running on the Yandex.Direct advertising platform. It fetches pages to assess content relevance, load speed, redirects, layout, and compliance with Yandex advertising policies. These checks directly influence ad approval, quality scores, and delivery performance. The bot is not a general search crawler and does not index content for Yandex Search. Blocking it may affect ad review or campaign effectiveness. Crawl activity is targeted and low-volume, typically triggered when ads are created, updated, or periodically reviewed for policy compliance. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexDirect using Yandex’s official validation methods, ensuring only genuine YandexDirect traffic is identified.

YandexDirectDyn is a Yandex crawler used for dynamic ad processing within the Yandex.Direct advertising platform. It fetches landing pages to analyze content, structure, and product or service details used in dynamically generated ads. The bot helps Yandex automatically create and update ad creatives based on page content, pricing, and availability. It performs targeted, ad-specific fetches rather than broad crawling. Blocking it may limit dynamic ad generation or affect campaign performance. Crawl activity is low-volume and triggered by dynamic ad setups, feed updates, or automated optimization cycles within Yandex.Direct. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexDirectDyn using Yandex’s official validation methods, ensuring only genuine YandexDirectDyn traffic is identified.

YandexFavicons is a Yandex crawler dedicated to fetching website favicon images for use across Yandex Search results, browsers, and related services. It retrieves small icon files such as favicon.ico or declared alternative icons in HTML markup. This crawler does not index page content or influence search rankings; its role is purely visual enrichment. Crawl activity is minimal and typically triggered when Yandex discovers a new site, detects favicon changes, or refreshes visual assets to ensure accurate and up-to-date representation in Yandex interfaces. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexFavicons using Yandex’s official validation methods, ensuring only genuine YandexFavicons traffic is identified.

YandexImageResizer is a Yandex service crawler that retrieves images to generate resized, optimized, or cached versions for use in Yandex search results and related products. It fetches original image files to create thumbnails or alternative resolutions suited for different devices and interfaces. This crawler does not perform full page indexing and does not independently rank content. Crawl activity is lightweight and asset-specific, typically triggered when Yandex needs to update or regenerate image variants to improve load performance and visual presentation across its platforms. YandexImageResizer honours the global robots.txt user agent (*) rule. RobotSense.io verifies YandexImageResizer using Yandex’s official validation methods, ensuring only genuine YandexImageResizer traffic is identified.

YandexImages is Yandex’s crawler responsible for discovering and indexing image content for Yandex Images and related visual search services. It fetches image files along with surrounding page context, captions, alt text, and structured data to understand relevance and quality. Crawl activity is typically lightweight and asset-focused, revisiting images based on page updates and popularity signals. Its purpose is to keep Yandex’s image search index accurate, fresh, and useful for users searching visually across Yandex’s ecosystem. It observes the global robots.txt user agent (*) rule. RobotSense.io verifies YandexImages using Yandex’s official validation methods, ensuring only genuine YandexImages traffic is identified.

YandexMarket is a Yandex crawler used to fetch and validate product information for the Yandex Market shopping platform. It retrieves product pages, pricing, availability, images, and structured data to ensure accurate listings. These fetches support price comparison, catalog updates, and merchant verification within Yandex Market. The bot is not a general search crawler and does not index content for Yandex Search. Blocking it may prevent product updates or visibility on Yandex Market. Crawl activity is targeted and low-volume, typically driven by merchant feed submissions, catalog refreshes, or product data changes. YandexMarket crawler ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexMarket using Yandex’s official validation methods, ensuring only genuine YandexMarket traffic is identified.

YandexMedia is a Yandex crawler used to discover and process media content such as audio, video, and rich media assets for Yandex’s media-related services. It fetches media files along with associated metadata, descriptions, and structured data. This crawler supports content discovery and enrichment across Yandex’s media platforms rather than general web search indexing. Crawl activity is typically lightweight and asset-focused, triggered when Yandex detects new or updated media content or refreshes existing media libraries to keep listings accurate and up to date. It honours the global robots.txt user agent (*) rule. RobotSense.io verifies YandexMedia using Yandex’s official validation methods, ensuring only genuine YandexMedia traffic is identified.

YandexMetrika is a Yandex service user-agent associated with the Yandex Metrica analytics platform. It appears when Yandex systems verify counters, process analytics-related requests, or validate site configurations tied to Metrica features. This traffic is not a web crawler and does not index content or collect pages for search. Requests are typically lightweight and focused on analytics endpoints, scripts, or verification resources. Blocking it may interfere with analytics functionality or data accuracy. Activity is low-volume and event-driven, triggered by site visits, analytics setup changes, or periodic system checks related to measurement and reporting. YandexMetrika does not observe the global or user-agent specific rules in robots.txt file. RobotSense.io verifies YandexMetrika using Yandex’s official validation methods, ensuring only genuine YandexMetrika traffic is identified.

YandexMobileBot is Yandex’s mobile-focused web crawler used to evaluate and index content as it appears on smartphones and other mobile devices. It simulates mobile browsing conditions to assess layout, performance, usability, and content accessibility. The bot fetches pages, structured data, and media assets to support Yandex’s mobile search results and ranking systems. Crawl frequency depends on site popularity and update cadence, with emphasis on pages optimized for mobile users. Its role is to ensure accurate, mobile-friendly representation of content within Yandex Search. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexMobileBot using Yandex’s official validation methods, ensuring only genuine YandexMobileBot traffic is identified.

YandexMobileScreenShotBot is a Yandex crawler used to capture mobile-rendered screenshots of webpages for use in Yandex Search and preview features. It simulates a smartphone browser to render pages as mobile users would see them. The bot fetches HTML, executes basic rendering, and captures visual snapshots to support rich previews and usability assessments. Crawl activity is targeted and relatively low-volume, focused on pages where visual representation or mobile preview quality is important within Yandex’s ecosystem. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexMobileScreenShotBot using Yandex’s official validation methods, ensuring only genuine YandexMobileScreenShotBot traffic is identified.

YandexOntoDB is a Yandex crawler used to extract and analyze structured knowledge from webpages for Yandex’s ontology and knowledge graph systems. It processes content to detect entities, relationships, attributes, and semantic patterns. This crawler supports Yandex’s understanding of people, organizations, places, and concepts across the web rather than performing standard search indexing. Crawl activity is typically focused and analytical, targeting pages rich in factual or structured information that can enhance Yandex’s internal knowledge databases. YandexOntoDB bot honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexOntoDB using Yandex’s official validation methods, ensuring only genuine YandexOntoDB traffic is identified.

YandexOntoDBAPI is a Yandex crawler and service agent used to retrieve and process webpage content for Yandex’s ontology and knowledge graph APIs. It fetches structured and semi-structured data to support semantic analysis, entity extraction, and API-driven knowledge services. This traffic is not part of general search indexing and is oriented toward data enrichment and knowledge engineering tasks. Crawl activity is typically low-volume and targeted, focusing on authoritative sources relevant to Yandex’s internal and partner-facing semantic platforms. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexOntoDBAPI using Yandex’s official validation methods, ensuring only genuine YandexOntoDBAPI traffic is identified.

YandexPagechecker is a Yandex crawler used to fetch webpages for technical validation, primarily through Yandex’s Structured Data Validator and related diagnostic tools. It retrieves pages to analyze schema markup, microdata, JSON-LD, and other structured elements. The bot checks syntax, compliance, and data consistency to help site owners ensure their pages are correctly interpreted by Yandex services. This crawler is not used for broad search indexing. It's crawl activity is typically low-volume and strictly on-demand, triggered when validation or inspection tools are run by users or internal systems. It honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexPagechecker using Yandex’s official validation methods, ensuring only genuine YandexPagechecker traffic is identified.

YandexPartner is a Yandex crawler associated with partner integrations and content distribution agreements. It fetches webpage content and metadata to support Yandex services that rely on licensed or syndicated data from external publishers. This traffic is not general search crawling and does not directly affect Yandex Search rankings. Crawl activity is typically targeted and low-volume, focused on partner-approved resources and data synchronization workflows. Its purpose is to ensure accurate, timely content delivery between Yandex and its publishing or platform partners. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexPartner using Yandex’s official validation methods, ensuring only genuine YandexPartner traffic is identified.

YandexRCA is a Yandex crawler used to collect webpage data for generating content previews across Yandex services. It fetches pages to extract titles, descriptions, images, and other metadata required to render previews in search results, recommendations, and partner surfaces. The bot does not perform full search indexing and does not analyze pages for ranking purposes. Crawl activity is lightweight and targeted, typically triggered when Yandex needs to create or refresh a preview for a specific URL. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexRCA using Yandex’s official validation methods, ensuring only genuine YandexRCA traffic is identified.

YandexRenderResourcesBot is a Yandex crawler used to load external resources - such as JavaScript, CSS, and related assets - required for full page rendering. It supports Yandex’s ability to correctly render and evaluate pages that rely on client-side code. The bot accesses resources only when the HTML page that references them is allowed for Yandex bots. If a page is disallowed in robots.txt, the bot will not fetch its resources. However, if the HTML page is accessible, YandexRenderResourcesBot may ignore robots.txt rules for individual resources. Crawl activity is tightly scoped and occurs solely to support JavaScript-based rendering and accurate page interpretation. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexRenderResourcesBot using Yandex’s official validation methods, ensuring only genuine YandexRenderResourcesBot traffic is identified.

YandexScreenshotBot is a Yandex crawler used to capture visual screenshots of webpages for previews and diagnostic purposes. It renders pages to take static images that represent how content appears to users. These screenshots may be used in search previews, Webmaster tools, or internal evaluation systems. The bot does not index content for search ranking and does not perform broad crawling. Crawl activity is targeted and low-volume, typically triggered when Yandex needs to create or refresh a visual snapshot of a specific page. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexScreenshotBot using Yandex’s official validation methods, ensuring only genuine YandexScreenshotBot traffic is identified.

YandexSearchShop is a Yandex crawler used to download product catalogs provided in YML (Yandex Market Language) format at the request of users. These catalog files are typically used for shopping and comparison services and are often stored in directories that are otherwise prohibited from search indexing. The bot fetches only the specified YML files rather than crawling entire websites. This activity is not related to general search indexing. Crawl behavior is targeted and request-driven, focusing solely on retrieving product feeds needed to process user-submitted catalog data. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexSearchShop using Yandex’s official validation methods, ensuring only genuine YandexSearchShop traffic is identified.

YandexSitelinks is a Yandex crawler used to verify the availability and accessibility of pages selected for sitelinks in Yandex Search results. It performs targeted checks to confirm that sitelink URLs return valid responses and remain accessible. The bot does not crawl sites broadly or index new content; its role is strictly validation. It's crawl activity is lightweight and periodic, triggered when sitelinks are generated, updated, or rechecked. Its purpose is to ensure that sitelinks shown in search results lead to functioning, reliable pages for users. It honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexSitelinks using Yandex’s official validation methods, ensuring only genuine YandexSitelinks traffic is identified.

YandexSpravBot is Yandex’s business-focused crawler used to collect and verify information for Yandex Business and related local services. It fetches webpages to gather business details such as names, addresses, phone numbers, opening hours, categories, and other local signals. This bot supports business profiles, maps, and directory listings rather than general web search indexing. Crawl activity is targeted and typically low-volume, focused on pages relevant to business information accuracy and updates within Yandex’s local and commercial ecosystem. It honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexSpravBot using Yandex’s official validation methods, ensuring only genuine YandexSpravBot traffic is identified.

YandexTracker is a Yandex service crawler associated with Yandex Tracker, a work management and issue-tracking platform. It fetches webpage content when users attach URLs or configure integrations that require retrieving external resources. This traffic is user-initiated and not part of web crawling or search indexing. The bot retrieves only the specific URLs referenced in Tracker tasks or workflows. Crawl activity is low-volume and strictly event-driven, triggered by user actions within Yandex Tracker rather than automated discovery. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexTracker using Yandex’s official validation methods, ensuring only genuine YandexTracker traffic is identified.

YandexUserproxy is a Yandex service user-agent that proxies real user actions across Yandex services. It sends requests when users click buttons, open links, or trigger features such as online page translation, content previews, or embedded tools. This traffic is explicitly user-driven and does not represent autonomous crawling or indexing. The agent identifies itself with the YandexUserproxy user-agent and fetches only the URLs required to complete a user-requested action. Request volume is typically low and bursty, reflecting individual user interactions rather than systematic data collection. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexUserproxy using Yandex’s official validation methods, ensuring only genuine YandexUserproxy traffic is identified.

YandexVerticals is a Yandex crawler used by Yandex Verticals (Classifieds), including Auto.ru, Yandex Realty, Yandex Jobs, and Yandex Reviews. It fetches webpages to collect listings, offers, reviews, and structured data required to populate these category-specific platforms. The bot is not a general search crawler and does not index content for Yandex Search. Crawl activity is targeted and driven by vertical-specific needs, such as updating listings, verifying availability, or refreshing classified data. Its purpose is to keep Yandex’s classifieds and review platforms accurate, current, and reliable for users. It honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexVerticals using Yandex’s official validation methods, ensuring only genuine YandexVerticals traffic is identified.

YandexVertis is a Yandex crawler used to support Yandex’s specialized search verticals, such as real estate, jobs, auto listings, and other category-focused services. It fetches webpages to collect structured and semi-structured data relevant to these verticals, including listings, prices, attributes, and availability. This bot is not a general-purpose search crawler and does not perform broad web indexing. Crawl activity is targeted and use-case driven, focusing on pages that supply data needed to power Yandex’s vertical search and marketplace-style experiences. It honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexVertis using Yandex’s official validation methods, ensuring only genuine YandexVertis traffic is identified.

YandexVideo is Yandex’s crawler responsible for discovering and indexing video content for Yandex Video search. It fetches pages and video files to analyze video metadata, thumbnails, captions, structured data, and surrounding context. This crawler determines how videos are categorized and displayed in Yandex’s video search results. It allows site owners to control video indexing independently from standard web crawling. Crawl activity is asset-focused and periodic, prioritizing pages that host or reference video content. Its role is to keep Yandex’s video search index accurate, fresh, and aligned with user video queries. It honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexVideo using Yandex’s official validation methods, ensuring only genuine YandexVideo traffic is identified.

YandexVideoParser is a Yandex crawler used to parse and index video content for display in Yandex Video search. It focuses on extracting video-related signals such as file URLs, metadata, thumbnails, embeds, transcripts, and structured data. The bot works alongside YandexVideo to interpret how videos should appear in search results. Crawl activity is targeted and asset-focused, concentrating on pages that contain or reference video content. Its purpose is to ensure accurate understanding and presentation of videos within Yandex’s video search ecosystem. It ignores the global robots.txt user agent (*) rule. RobotSense.io verifies YandexVideoParser using Yandex’s official validation methods, ensuring only genuine YandexVideoParser traffic is identified.

YandexWebmaster is a Yandex crawler used in connection with Yandex Webmaster tools to perform site diagnostics, verification, and technical checks. It fetches webpages when site owners use Webmaster features such as URL inspection, indexing checks, error analysis, or configuration validation. This traffic is on-demand and user-initiated, not part of broad search crawling or discovery. Crawl activity is typically low-volume and focused on specific URLs submitted through Webmaster tools to help site owners monitor and optimize their sites for Yandex Search. It honors the global robots.txt user agent (*) rule. RobotSense.io verifies YandexWebmaster using Yandex’s official validation methods, ensuring only genuine YandexWebmaster traffic is identified.