What is Meta-ExternalFetcher, and why is it visiting my website?

Meta-ExternalFetcher is a crawler operated by Meta that retrieves webpage content for link previews, metadata extraction, and content enrichment across Facebook, Instagram, Messenger, Threads, and related Meta products. It typically fetches page titles, descriptions, images, Open Graph tags, and structured data when users share links or when Meta refreshes preview information. Crawl activity is generally user-driven and focused on specific URLs rather than broad website indexing. For publicly accessible websites that are shared on Meta platforms, this bot traffic is expected.

Is Meta-ExternalFetcher a legitimate bot, or is it commonly spoofed?

Meta-ExternalFetcher is a legitimate crawler operated by Meta. Like many widely recognized bots, its User-Agent string can be spoofed by scrapers, scanners, or malicious actors attempting to disguise automated traffic. Attackers may impersonate Meta bots because some websites allow them access to content or bypass certain filtering rules. User-Agent strings alone cannot reliably verify authenticity and should always be supplemented with additional verification methods. You can use Meta's recommended methods mentioned below to verify a legitimate visit, or use RobotSense.io API to easily verify Meta-ExternalFetcher bot visits.

How can I verify that a request is really coming from Meta-ExternalFetcher?

You can use Meta's recommended official methods to verify Meta-ExternalFetcher bot visits, these include: - IP range checks Do not use User-Agent based detection as that can be easily spoofed. Alternatively, you can use RobotSense.io API to easily verify Meta-ExternalFetcher bot and all other bots from Meta.

Should I allow or block Meta-ExternalFetcher on my website?

For most public websites, allowing Meta-ExternalFetcher is beneficial because it enables accurate link previews when content is shared across Meta platforms. Without access to page metadata and images, preview quality may be reduced or unavailable. Blocking may be appropriate when: - Content is private or access-controlled. - Sensitive resources should not be fetched by external services. - Internal applications or APIs are not intended for public sharing. - Traffic management policies require stricter crawler controls. For websites that rely on social sharing, allowing the crawler is generally recommended.

How can I control or block Meta-ExternalFetcher using robots.txt or other methods?

You can add a rule in your robots.txt, as given above to control (crawl-delay) or disallow Meta-ExternalFetcher bot. The Meta-ExternalFetcher bot honors it's own specific robots.txt directives, but does not honor global directives. Also, you can use further controls in your WAF, or in RobotSense enforcement settings to manage the bot behavior.

How often does Meta-ExternalFetcher crawl websites, and can it impact server performance?

Meta-ExternalFetcher primarily performs event-driven fetching. Requests commonly occur when users share links on Meta platforms, when previews are generated, or when metadata needs to be refreshed. For most websites, performance impact is minimal because: - Requests are limited to specific URLs. - Bandwidth usage is typically low. - Crawl activity is not site-wide. Large publishers with frequently shared content may see higher request volumes, but the crawler generally produces much less load than major search engine bots.

What happens if I block Meta-ExternalFetcher? SEO, visibility, and feature impact explained.

Blocking Meta-ExternalFetcher does not affect traditional search engine rankings because it is not a search indexing crawler. Potential impacts include: - Missing or incomplete link previews on Facebook, Instagram, Messenger, or other Meta products. - Missing page titles, descriptions, or preview images when links are shared. - Reduced content enrichment within Meta's ecosystem. - Delays or failures when metadata needs to be refreshed. Blocking does not directly affect: - Google indexing. - Bing indexing. - Organic search rankings. - Most SEO databases or analytics tools.

Does Meta-ExternalFetcher collect, scrape, or use my content for training or reuse?

Meta-ExternalFetcher retrieves webpage content and metadata required to generate previews and enrich shared links. This commonly includes page titles, descriptions, Open Graph metadata, structured data, and preview images. Documented uses include: - Link preview generation. - Metadata extraction. - Content enrichment for shared URLs. - Preview refresh and validation. There is no public documentation indicating that Meta-ExternalFetcher is used for general web indexing or AI training. Its documented purpose is focused on content retrieval needed to support sharing and preview functionality across Meta products.

Meta-ExternalFetcher

Name: Meta-ExternalFetcher
Author: Meta / Facebook

Others

Operated by Meta / FacebookOthers

Visit Bot Homepage

Verify Meta-ExternalFetcher IP Address

Verify if an IP address truly belongs to Meta / Facebook, using official verification methods. Enter both IP address and User-Agent from your logs for the most accurate bot verification.

Meta-ExternalFetcher is a Meta crawler that retrieves webpage content to support link previews, metadata extraction, and other external content processing tasks across Facebook, Instagram, and related Meta products. It fetches titles, descriptions, images, and structured data required for rendering shared links or enriching user interactions. These requests are typically user-driven but may also support automated metadata refreshes. Crawl volume is lightweight and focused, targeting only the URLs needed for previews or content enrichment within Meta’s ecosystem. It ignores the global user agent (*) rule. RobotSense.io verifies Meta-ExternalFetcher using Meta’s official validation methods, ensuring only genuine Meta-ExternalFetcher traffic is identified.

This bot does not honor Crawl-Delay rule.

User Agent Examples

Contains: meta-externalfetcher/1.1 (+https://developers.facebook.com/docs/sharing/webmasters/crawler)

Contains: meta-externalfetcher/1.1

Robots.txt Configuration for Meta-ExternalFetcher

Robots.txt User-Agent:Meta-ExternalFetcher

Use this identifier in your robots.txt User-agent directive to target Meta-ExternalFetcher.

Recommended Configuration

Our recommended robots.txt configuration for Meta-ExternalFetcher:

User-agent: Meta-ExternalFetcher
Allow: /

Completely Block Meta-ExternalFetcher

Prevent this bot from crawling your entire site:

User-agent: Meta-ExternalFetcher
Disallow: /

Completely Allow Meta-ExternalFetcher

Allow this bot to crawl your entire site:

User-agent: Meta-ExternalFetcher
Allow: /

Block Specific Paths

Block this bot from specific directories or pages:

User-agent: Meta-ExternalFetcher
Disallow: /private/
Disallow: /admin/
Disallow: /api/

Allow Only Specific Paths

Block everything but allow specific directories:

User-agent: Meta-ExternalFetcher
Disallow: /
Allow: /public/
Allow: /blog/

Set Crawl Delay

Limit how frequently Meta-ExternalFetcher can request pages (in seconds):

User-agent: Meta-ExternalFetcher
Allow: /
Crawl-delay: 10

Note: This bot does not officially mention about honoring Crawl-Delay rule.

Frequently Asked Questions

What is Meta-ExternalFetcher, and why is it visiting my website?: Meta-ExternalFetcher is a crawler operated by Meta that retrieves webpage content for link previews, metadata extraction, and content enrichment across Facebook, Instagram, Messenger, Threads, and related Meta products. It typically fetches page titles, descriptions, images, Open Graph tags, and structured data when users share links or when Meta refreshes preview information. Crawl activity is generally user-driven and focused on specific URLs rather than broad website indexing. For publicly accessible websites that are shared on Meta platforms, this bot traffic is expected.
Is Meta-ExternalFetcher a legitimate bot, or is it commonly spoofed?: Meta-ExternalFetcher is a legitimate crawler operated by Meta. Like many widely recognized bots, its User-Agent string can be spoofed by scrapers, scanners, or malicious actors attempting to disguise automated traffic. Attackers may impersonate Meta bots because some websites allow them access to content or bypass certain filtering rules. User-Agent strings alone cannot reliably verify authenticity and should always be supplemented with additional verification methods. You can use Meta's recommended methods mentioned below to verify a legitimate visit, or use RobotSense.io API to easily verify Meta-ExternalFetcher bot visits.
How can I verify that a request is really coming from Meta-ExternalFetcher?: You can use Meta's recommended official methods to verify Meta-ExternalFetcher bot visits, these include: - IP range checks Do not use User-Agent based detection as that can be easily spoofed. Alternatively, you can use RobotSense.io API to easily verify Meta-ExternalFetcher bot and all other bots from Meta.
Should I allow or block Meta-ExternalFetcher on my website?: For most public websites, allowing Meta-ExternalFetcher is beneficial because it enables accurate link previews when content is shared across Meta platforms. Without access to page metadata and images, preview quality may be reduced or unavailable. Blocking may be appropriate when: - Content is private or access-controlled. - Sensitive resources should not be fetched by external services. - Internal applications or APIs are not intended for public sharing. - Traffic management policies require stricter crawler controls. For websites that rely on social sharing, allowing the crawler is generally recommended.
How can I control or block Meta-ExternalFetcher using robots.txt or other methods?: You can add a rule in your robots.txt, as given above to control (crawl-delay) or disallow Meta-ExternalFetcher bot. The Meta-ExternalFetcher bot honors it's own specific robots.txt directives, but does not honor global directives. Also, you can use further controls in your WAF, or in RobotSense enforcement settings to manage the bot behavior.
How often does Meta-ExternalFetcher crawl websites, and can it impact server performance?: Meta-ExternalFetcher primarily performs event-driven fetching. Requests commonly occur when users share links on Meta platforms, when previews are generated, or when metadata needs to be refreshed. For most websites, performance impact is minimal because: - Requests are limited to specific URLs. - Bandwidth usage is typically low. - Crawl activity is not site-wide. Large publishers with frequently shared content may see higher request volumes, but the crawler generally produces much less load than major search engine bots.
What happens if I block Meta-ExternalFetcher? SEO, visibility, and feature impact explained.: Blocking Meta-ExternalFetcher does not affect traditional search engine rankings because it is not a search indexing crawler. Potential impacts include: - Missing or incomplete link previews on Facebook, Instagram, Messenger, or other Meta products. - Missing page titles, descriptions, or preview images when links are shared. - Reduced content enrichment within Meta's ecosystem. - Delays or failures when metadata needs to be refreshed. Blocking does not directly affect: - Google indexing. - Bing indexing. - Organic search rankings. - Most SEO databases or analytics tools.
Does Meta-ExternalFetcher collect, scrape, or use my content for training or reuse?: Meta-ExternalFetcher retrieves webpage content and metadata required to generate previews and enrich shared links. This commonly includes page titles, descriptions, Open Graph metadata, structured data, and preview images. Documented uses include: - Link preview generation. - Metadata extraction. - Content enrichment for shared URLs. - Preview refresh and validation. There is no public documentation indicating that Meta-ExternalFetcher is used for general web indexing or AI training. Its documented purpose is focused on content retrieval needed to support sharing and preview functionality across Meta products.