iTMS (iTunes Crawler)
OthersVerify iTMS (iTunes Crawler) IP Address
Verify if an IP address truly belongs to Apple, using official verification methods. Enter both IP address and User-Agent from your logs for the most accurate bot verification.
iTMS (iTunes Crawler) is Apple’s crawler used to fetch and validate webpages associated with content distributed through Apple services such as the iTunes Store, Apple Podcasts, and related media platforms. It retrieves metadata, feeds, and linked resources required for listing, previewing, or validating content. This crawler does not perform general web indexing and does not influence search rankings outside Apple’s ecosystem. Crawl activity is typically targeted and low-volume, triggered when content is submitted, updated, or refreshed within Apple’s media services. As per official documentation, iTMS does not respects robots.txt rules. RobotSense.io verifies iTMS using Apple's official validation methods, ensuring only genuine iTMS traffic is identified.
User Agent Examples
iTMSRobots.txt Configuration for iTMS (iTunes Crawler)
iTMSUse this identifier in your robots.txt User-agent directive to target iTMS (iTunes Crawler).
Recommended Configuration
Our recommended robots.txt configuration for iTMS (iTunes Crawler):
User-agent: iTMS
Allow: /Completely Block iTMS (iTunes Crawler)
Prevent this bot from crawling your entire site:
User-agent: iTMS
Disallow: /Completely Allow iTMS (iTunes Crawler)
Allow this bot to crawl your entire site:
User-agent: iTMS
Allow: /Block Specific Paths
Block this bot from specific directories or pages:
User-agent: iTMS
Disallow: /private/
Disallow: /admin/
Disallow: /api/Allow Only Specific Paths
Block everything but allow specific directories:
User-agent: iTMS
Disallow: /
Allow: /public/
Allow: /blog/Set Crawl Delay
Limit how frequently iTMS (iTunes Crawler) can request pages (in seconds):
User-agent: iTMS
Allow: /
Crawl-delay: 10Note: This bot does not officially mention about honoring Crawl-Delay rule.
Put these rules to work
Frequently Asked Questions
- What is iTMS bot, and why is it visiting my website?
- iTMS (iTunes Crawler) is Apple's crawler used to retrieve and validate content associated with Apple media services, including Apple Podcasts, the iTunes Store, and related content platforms. It typically visits websites to fetch podcast feeds, media metadata, artwork, and linked resources needed for content listings, previews, and validation. Crawl activity is usually triggered when content is submitted, updated, refreshed, or reviewed within Apple's ecosystem. For websites that distribute content through Apple services, iTMS traffic is expected and generally low in volume.
- Is iTMS a legitimate bot, or is it commonly spoofed?
- iTMS is a legitimate crawler operated by Apple. As with many well-known crawlers, its User-Agent can be spoofed by automated tools, scrapers, or malicious actors attempting to disguise their traffic. Attackers may impersonate iTMS to avoid bot filtering or gain access to content that receives special treatment. Because User-Agent strings can be easily forged, they should not be relied upon as the sole method of verification. You can use Apple's recommended methods mentioned below to verify a legitimate visit, or use RobotSense.io API to easily verify iTMS bot visits.
- How can I verify that a request is really coming from iTMS bot?
- You can use Apple's recommended official methods to verify iTMS bot visits, these include: - IP range checks - Reverse DNS → forward DNS Do not use User-Agent based detection as that can be easily spoofed. Alternatively, you can use RobotSense.io API to easily verify iTMS bot and all other bots from Apple.
- Should I allow or block iTMS on my website?
- For websites that publish podcasts, media feeds, or content distributed through Apple services, allowing iTMS is generally beneficial because it enables content validation and metadata retrieval. If your site does not participate in Apple's media ecosystem, the crawler may be neutral and provide little direct benefit. Blocking may be appropriate when: - The content should not appear in Apple services. - The site hosts sensitive or restricted resources. - Internal APIs should not be accessed by external crawlers. - Traffic management or compliance requirements require stricter controls.
- How can I control or block iTMS using robots.txt or other methods?
- iTMS bot does not honor robots.txt directives. You can use controls in your WAF, or in RobotSense enforcement settings to manage the bot behavior.
- How often does iTMS crawl websites, and can it impact server performance?
- iTMS generally uses event-driven and refresh-based crawling rather than broad continuous web crawling. Requests are typically triggered when Apple services need to validate, update, or refresh content associated with a listing. For most websites, impact is minimal because: - Request volumes are usually low. - Bandwidth consumption is modest. - Crawl activity is targeted rather than site-wide. Large media catalogs or frequently updated podcast networks may observe periodic bursts of crawler activity in server logs.
- What happens if I block iTMS? SEO, visibility, and feature impact explained.
- Blocking iTMS does not affect rankings in Google, Bing, or other general-purpose search engines because iTMS is not a search indexing crawler. Potential impacts include: - Podcast feeds may not validate correctly. - Content updates may not be refreshed in Apple services. - Media listings, previews, or metadata may become outdated. - New content submissions may fail verification checks. - Apple-specific content discovery features may be limited. General web SEO and search visibility outside Apple's ecosystem are typically unaffected.
- Does iTMS collect, scrape, or use my content for training or reuse?
- iTMS retrieves content required to validate and display media listings within Apple services. This may include podcast feeds, metadata, artwork, episode information, and other linked resources necessary for content presentation. Documented uses include: - Content validation. - Metadata retrieval. - Listing generation. - Preview and catalog maintenance. There is no public documentation indicating that iTMS is used for general web indexing, SEO databases, or AI model training. Its activity is focused on supporting Apple media services rather than broad content collection across the web.
Other Apple Bots
Apple operates other crawlers you may also need to configure.
Similar Bots
Other Others bots from different operators.
Amazonbot
Othersby Amazon
[Amazon Bots can take upto 30 days to read your Robots.txt updates.] Amazonbot is Amazon’s official web crawler, used to discover and fetch webpage content for applications such as Alexa, product-related features, and Amazon’s AI and search systems. Crawl activity varies based on Amazon services that rely on external web content, but it is generally moderate and focused on structured data, text content, and page metadata. Its purpose is to enhance Amazon’s search, AI models, and user-facing features. It ignores the global user agent (*) rule. RobotSense.io verifies Amazonbot using Amazon’s official validation methods, ensuring only genuine Amazonbot traffic is identified.
Amzn-User
Othersby Amazon
[Amazon Bots can take upto 30 days to read your Robots.txt updates.] Amzn-User is a bot associated with Amazon services that fetch webpage content on behalf of end users or Amazon applications rather than acting as a general-purpose crawler. It typically appears when Amazon apps, devices, or internal systems request metadata, previews, or content needed for features like link expansion, in-app browsing, or contextual analysis. The traffic is user-driven, not designed for large-scale indexing or scraping. Amzn-User usually performs lightweight, targeted fetches limited to specific URLs users interact with. Its purpose is to support Amazon product experiences by retrieving just enough page data to power user-facing functionality. It ignores the global user agent (*) rule. RobotSense.io verifies Amzn-User using Amazon’s official validation methods, ensuring only genuine Amzn-User traffic is identified.
BingPreview
Othersby Microsoft
BingPreview is Microsoft’s rendering and compatibility crawler used to evaluate how webpages appear in browsers and Bing search features. It fetches pages to test layout, mobile responsiveness, JavaScript rendering, and visual elements. These checks help Bing understand how content will display in search results and improve snippet generation and ranking signals tied to user experience. Crawl activity is moderate and often concentrated on pages important to Bing’s index. Its purpose is to simulate real-browser behavior and refine Bing’s presentation quality. RobotSense.io verifies BingPreview using Microsoft’s official validation methods, ensuring only genuine BingPreview traffic is identified.
DuplexWeb-Google
Othersby Google
[This crawler is officially retired as per Google] DuplexWeb-Google is a Google crawler associated with Duplex and Assistant-related technologies that fetch web content to help generate conversational responses and perform task-oriented actions. It retrieves page information needed to understand structured data, business details, menus, appointment flows, and other interactive elements. Crawl activity is selective and generally tied to user-initiated tasks or systems that prepare content for automated assistance. Its purpose is to support natural-language interactions by ensuring Google’s assistant technologies can interpret and use real-time webpage information accurately. It ignores the global user agent (*) rule. RobotSense.io verifies DuplexWeb-Google using Google’s official validation methods, ensuring only genuine DuplexWeb-Google traffic is identified.
FacebookExternalHit
Othersby Meta / Facebook
FacebookExternalHit is Facebook’s (Meta’s) crawler used to fetch webpage content for link previews across Facebook, Messenger, Instagram, and other Meta surfaces. It retrieves metadata such as Open Graph tags, titles, descriptions, images, and structured data. These requests are user-triggered, occurring when someone shares or pastes a URL on a Meta platform. The bot does not index or rank websites and has no connection to search algorithms. Blocking it may prevent accurate link previews. Crawl activity is lightweight and focused on fetching just enough content to generate rich social previews. It ignores the global user agent (*) rule. RobotSense.io verifies FacebookExternalHit using Meta’s official validation methods, ensuring only genuine FacebookExternalHit traffic is identified.
Google Messages
Othersby Google
Google Messages uses a fetcher to retrieve webpage data for generating link previews when users send URLs in chat conversations. It fetches metadata such as titles, descriptions, images, and structured tags (e.g., Open Graph) to render rich previews inside messages. This traffic is strictly user-triggered, occurring only when a link is shared. It is not a crawler for indexing or discovery and has no impact on Google Search rankings. Blocking it may prevent previews from displaying correctly. Activity is lightweight and targeted, focused solely on enhancing the messaging experience with accurate link previews. Google Messages bot does not respect robots.txt rules. RobotSense.io verifies Google Messages using Google’s official validation methods, ensuring only genuine Google Messages traffic is identified.