What is Dotbot, and why is it visiting my website?

Dotbot is the primary web crawler operated by Moz to build and maintain Moz's link index and web graph. It crawls websites to discover backlinks, anchor text, page relationships, redirects, and other structural signals used in Moz's SEO and link intelligence products. Crawl activity is driven by Moz's indexing processes and link discovery efforts rather than user visits. For publicly accessible websites, Dotbot traffic is normal and expected if the site is connected to the broader web.

Is Dotbot a legitimate bot, or is it commonly spoofed?

Dotbot is a legitimate crawler operated by Moz. Like most well-known crawlers, its User-Agent string can be spoofed by scrapers, scanners, or malicious actors attempting to make their requests appear trustworthy. Attackers may impersonate Dotbot to bypass bot protections or gain access to content that receives preferential treatment. Because User-Agent headers can be easily forged, User-Agent matching alone cannot reliably verify authenticity. Unfortunately, Moz does not provide any method beyond user-agent for legitimate bot detection.

How can I verify that a request is really coming from Dotbot?

Unfortunately, Moz does not support any method beyond user-agent for bot verification. This potentially makes Dotbot a highly abused bot.

Should I allow or block Dotbot on my website?

Allowing Dotbot is generally beneficial if you want your website represented accurately within Moz's SEO tools and backlink datasets. Its crawl data contributes to link analysis, authority metrics, competitive research, and other SEO insights used by Moz customers. Blocking may be appropriate when: - The site contains sensitive or proprietary content. - Server resources are limited. - Internal applications or APIs should not be crawled. - You do not want your site included in third-party SEO datasets. - Or, if you are seeing unusual traffic from Dotbot For most public websites, allowing Dotbot is optional but often useful.

How can I control or block Dotbot using robots.txt or other methods?

You can add a rule in your robots.txt, as given above to control (crawl-delay) or disallow Dotbot. The Dotbot honors robots.txt directives. Also, you can use further controls in your WAF, or in RobotSense enforcement settings to manage the bot behavior.

How often does Dotbot crawl websites, and can it impact server performance?

Dotbot performs continuous and periodic crawling to maintain Moz's web graph and backlink index. Crawl frequency depends on factors such as site size, link popularity, crawl accessibility, and the site's importance within the web's link structure. Potential impacts include: - Increased bandwidth usage. - Moderate to high request volumes on large websites. - Additional load on dynamic pages and database-driven content. For smaller websites, the impact is usually minimal. Larger sites with extensive content may observe more noticeable bot traffic in server logs.

What happens if I block Dotbot? SEO, visibility, and feature impact explained.

Blocking Dotbot does not directly affect rankings in Google, Bing, or other search engines because Dotbot is not a search engine crawler. Potential impacts include: - Reduced backlink visibility in Moz tools. - Less complete link profile reporting. - Lower accuracy of Domain Authority and related Moz metrics. - Reduced representation in Moz's SEO datasets and competitive research tools. Blocking Dotbot does not directly affect: - Search engine indexing. - Organic search rankings. - Social media previews. - AI training systems. The primary impact is on Moz's ability to analyze and report information about your website.

Does Dotbot collect, scrape, or use my content for training or reuse?

Dotbot collects webpage content and metadata necessary to build Moz's link index and web graph. This includes hyperlinks, anchor text, redirects, canonical tags, page titles, metadata, and structural signals used for SEO analysis. Documented uses include: - Backlink discovery and indexing. - Link graph construction. - Domain Authority and link metric calculations. - SEO research and competitive analysis. There is no public documentation indicating that Dotbot is used for AI model training. Its primary role is link intelligence and SEO data collection rather than machine learning dataset creation or public search indexing.

Dotbot

Name: Dotbot
Author: Moz

SEO Tools

Operated by MozSEO Tools

Visit Bot Homepage

Verify Dotbot IP Address

Verify if an IP address truly belongs to Moz, using official verification methods. Enter both IP address and User-Agent from your logs for the most accurate bot verification.

Dotbot is the web crawler operated by Moz, primarily used to build and maintain Moz’s link index and web graph. It crawls webpages to discover backlinks, anchor text, page relationships, and structural signals used in Moz’s SEO and link intelligence products. Dotbot is not a public search engine crawler and does not influence search rankings directly. Crawl activity can be moderate to high depending on site size and connectivity, reflecting its role in mapping link relationships across the web for SEO analysis and competitive research.

This bot officially honors Crawl-Delay rule.

User Agent Examples

Contains: dotbot

Robots.txt Configuration for Dotbot

Robots.txt User-Agent:dotbot

Use this identifier in your robots.txt User-agent directive to target Dotbot.

Recommended Configuration

Our recommended robots.txt configuration for Dotbot:

User-agent: dotbot
Allow: /

Completely Block Dotbot

Prevent this bot from crawling your entire site:

User-agent: dotbot
Disallow: /

Completely Allow Dotbot

Allow this bot to crawl your entire site:

User-agent: dotbot
Allow: /

Block Specific Paths

Block this bot from specific directories or pages:

User-agent: dotbot
Disallow: /private/
Disallow: /admin/
Disallow: /api/

Allow Only Specific Paths

Block everything but allow specific directories:

User-agent: dotbot
Disallow: /
Allow: /public/
Allow: /blog/

Set Crawl Delay

Limit how frequently Dotbot can request pages (in seconds):

User-agent: dotbot
Allow: /
Crawl-delay: 10

Note: This bot officially honors the Crawl-delay directive.

Frequently Asked Questions

What is Dotbot, and why is it visiting my website?: Dotbot is the primary web crawler operated by Moz to build and maintain Moz's link index and web graph. It crawls websites to discover backlinks, anchor text, page relationships, redirects, and other structural signals used in Moz's SEO and link intelligence products. Crawl activity is driven by Moz's indexing processes and link discovery efforts rather than user visits. For publicly accessible websites, Dotbot traffic is normal and expected if the site is connected to the broader web.
Is Dotbot a legitimate bot, or is it commonly spoofed?: Dotbot is a legitimate crawler operated by Moz. Like most well-known crawlers, its User-Agent string can be spoofed by scrapers, scanners, or malicious actors attempting to make their requests appear trustworthy. Attackers may impersonate Dotbot to bypass bot protections or gain access to content that receives preferential treatment. Because User-Agent headers can be easily forged, User-Agent matching alone cannot reliably verify authenticity. Unfortunately, Moz does not provide any method beyond user-agent for legitimate bot detection.
How can I verify that a request is really coming from Dotbot?: Unfortunately, Moz does not support any method beyond user-agent for bot verification. This potentially makes Dotbot a highly abused bot.
Should I allow or block Dotbot on my website?: Allowing Dotbot is generally beneficial if you want your website represented accurately within Moz's SEO tools and backlink datasets. Its crawl data contributes to link analysis, authority metrics, competitive research, and other SEO insights used by Moz customers. Blocking may be appropriate when: - The site contains sensitive or proprietary content. - Server resources are limited. - Internal applications or APIs should not be crawled. - You do not want your site included in third-party SEO datasets. - Or, if you are seeing unusual traffic from Dotbot For most public websites, allowing Dotbot is optional but often useful.
How can I control or block Dotbot using robots.txt or other methods?: You can add a rule in your robots.txt, as given above to control (crawl-delay) or disallow Dotbot. The Dotbot honors robots.txt directives. Also, you can use further controls in your WAF, or in RobotSense enforcement settings to manage the bot behavior.
How often does Dotbot crawl websites, and can it impact server performance?: Dotbot performs continuous and periodic crawling to maintain Moz's web graph and backlink index. Crawl frequency depends on factors such as site size, link popularity, crawl accessibility, and the site's importance within the web's link structure. Potential impacts include: - Increased bandwidth usage. - Moderate to high request volumes on large websites. - Additional load on dynamic pages and database-driven content. For smaller websites, the impact is usually minimal. Larger sites with extensive content may observe more noticeable bot traffic in server logs.
What happens if I block Dotbot? SEO, visibility, and feature impact explained.: Blocking Dotbot does not directly affect rankings in Google, Bing, or other search engines because Dotbot is not a search engine crawler. Potential impacts include: - Reduced backlink visibility in Moz tools. - Less complete link profile reporting. - Lower accuracy of Domain Authority and related Moz metrics. - Reduced representation in Moz's SEO datasets and competitive research tools. Blocking Dotbot does not directly affect: - Search engine indexing. - Organic search rankings. - Social media previews. - AI training systems. The primary impact is on Moz's ability to analyze and report information about your website.
Does Dotbot collect, scrape, or use my content for training or reuse?: Dotbot collects webpage content and metadata necessary to build Moz's link index and web graph. This includes hyperlinks, anchor text, redirects, canonical tags, page titles, metadata, and structural signals used for SEO analysis. Documented uses include: - Backlink discovery and indexing. - Link graph construction. - Domain Authority and link metric calculations. - SEO research and competitive analysis. There is no public documentation indicating that Dotbot is used for AI model training. Its primary role is link intelligence and SEO data collection rather than machine learning dataset creation or public search indexing.