PerfBee checks your site with its own crawler, PerfBeeBot. This guide shows how to add your site to PerfBee and how to let PerfBeeBot through your firewall, so that every page gets checked.
1. Add your site to My Sites
In the dashboard, open My Sites and add your site's address. A site in My Sites:
- gets a first Fast scan when you add it (see the credits at the end);
- gets its own PerfBee token. PerfBeeBot sends it in the
X-PerfBee-Tokenheader on its requests to that site (not on its video, audio and iframe checks), and to no other site.
Only add sites you own or are allowed to audit.
Verify that the site is yours, with a DNS record, a file or a meta tag. Verification is not needed to scan. On the Free plan it lets one crawl read more than 500 pages, and on every plan it unlocks two owner-only options: Ignore robots.txt for your own scans, and Staging access for a site behind a password. See Verifying Your Site.
2. How PerfBeeBot crawls
PerfBeeBot always says who it is:
User-Agent: Mozilla/5.0 (compatible; PerfBeeBot/1.0; +https://perfbee.com/bot)
From: support@perfbee.com
By default it crawls politely. It sends about two requests per second to a host. It follows your robots.txt, including Crawl-delay. It slows down when your server answers 429 or 503. If a firewall refuses it, PerfBeeBot does not try to get around the block. Those pages show as Not checked in the report and cost no credits.
3. Let PerfBeeBot through your firewall
PerfBeeBot crawls from one IP address. It resolves back to crawler.perfbee.com. The current list is always at perfbee.com/bot/ips.json.
169.58.21.218
Allow this address. Until 11 October 2026 PerfBeeBot also used 173.212.236.185. If you allowed that one, you can remove it. To find your site's token, open Site Crawler, pick the site, and in the Crawler access card open How to let PerfBeeBot in. The same steps, filled in with your IP and token, are also in My Sites.
Cloudflare, Free plan
- Open your site in the Cloudflare dashboard and go to Security → WAF → Tools.
- Under IP Access Rules, add the PerfBee IP address with the action Allow.
On the Free plan this is the only rule that also gets PerfBeeBot past Bot Fight Mode. A rule that matches the token header does not.
Cloudflare, Pro plan and above
You can use a custom rule instead. Go to Security → WAF → Custom rules and create a rule. Choose Edit expression and paste this, with your own token:
(ip.src eq 169.58.21.218 and any(http.request.headers["x-perfbee-token"][*] eq "YOUR-PERFBEE-TOKEN"))
Choose the action Skip. Then tick Super Bot Fight Mode, rate limiting rules, managed rules, Browser Integrity Check and Security Level.
Always match the token's value together with our IP address. A rule that only checks whether the header is present lets anyone who sends that header past your firewall.
Nginx
If your server block uses deny rules, add our address above them, inside server {} or location /:
allow 169.58.21.218;
Then test the configuration and reload Nginx:
sudo nginx -t && sudo systemctl reload nginx
Apache
If your configuration restricts access, allow our address in .htaccess or your VirtualHost:
<RequireAny>
Require ip 169.58.21.218
Require all denied
</RequireAny>
Then reload Apache with sudo systemctl reload apache2.
Shopify
Shopify does not let stores allowlist an IP address. It uses crawler signatures instead. In your Shopify admin, open Online Store → Preferences → Crawler access and create a signature for your domain. Paste the Signature-Input and Signature values into Shopify crawler access for the site in My Sites. A signature lasts up to three months. We remind you before it expires.
Other firewalls and hosts
Steps for Akamai, AWS WAF, Vercel, Sucuri, Imperva, DataDome, Wordfence, Kinsta, WP Engine, Imunify360, Plesk, SiteGround and Linux firewalls are in the dashboard, under Crawler access and in My Sites.
4. Test access
In Site Crawler, pick the site and click Test access in the Crawler access card. PerfBeeBot requests the site's address the way a scan does. It tells you whether it got in. If it did not, it names the protection that refused it.
5. Is Crawl-delay slowing your scans?
If your robots.txt sets a Crawl-delay for all bots, PerfBeeBot waits that long between pages, up to 10 seconds. A longer delay counts as 10 seconds. At 10 seconds PerfBeeBot checks about 360 pages an hour, so a 1,000-page site takes about three hours. You can speed up PerfBee without changing what other bots see. Add a group for PerfBeeBot alone, without a Crawl-delay:
User-agent: PerfBeeBot
Disallow: /admin
A bot follows only the group that names it. Copy your Disallow lines into the PerfBeeBot group, or PerfBeeBot will be allowed everywhere. The scan report suggests this group, built from your own rules.
6. Ask for the fast lane
Once your firewall lets PerfBeeBot in, email support@perfbee.com. We switch your site to the fast lane: up to five requests per second instead of about two, and no robots.txt Crawl-delay, because you have let us in. Large sites then finish in a fraction of the time.
Credits
| Scan | Credits |
|---|---|
| Fast scan, per page | 1 credit |
| Detailed scan, per page | 1 credit |
| JavaScript rendering, per rendered page | 2 credits |
A verified site costs the same credits as any other site. With smart JavaScript rendering, only the pages that need it are rendered. The other pages cost the scan's normal price. Pages PerfBeeBot could not check cost nothing.