What does the crawler check?
+
PerfBeeBot starts from your sitemap, follows internal links and checks every page and link it finds. It can also check external links, images, CSS, JS, video, iframe and audio sources. A detailed crawl adds each page's titles, descriptions, headings, security headers, structured data, hreflang and near-duplicate text, plus the AI Readiness report.
Should I run a fast crawl or a detailed crawl?
+
Fast checks links and status codes. Detailed adds indexing, content and asset findings for every page, plus the AI Readiness report.
Does it check hreflang, structured data and near-duplicate pages?
+
Yes, in a detailed crawl. Hreflang is read from each page and from your XML sitemap, and invalid values, missing return links and broken alternates are listed. Structured data (JSON-LD, Microdata, RDFa-Lite) is compared with the properties Google documents for 15 types of markup. Pages with almost the same text are listed as near-duplicates. This compares your markup with Google's documentation; it does not say whether Google will show a rich result.
What happens when a site blocks PerfBeeBot?
+
The URL is marked Not checked, with the reason: a bot wall, 401, 403, 429, 430, 503, 999, robots.txt, a timeout or a dropped connection. It is never counted as broken and never billed. PerfBee does not get past bot walls; for your own site there are allowlist guides for Cloudflare, Shopify, Akamai, AWS WAF, Vercel and others.
Can it pull custom data or find text on every page?
+
Yes. Custom extraction reads a value from each page with a CSS selector, an XPath or a regex. Custom search lists the pages that contain a phrase or a regex, or the pages missing it. Test a rule on one page first. Each has its own report tab and CSV or Excel export.
What happens to the issues a crawl finds?
+
Every finding becomes a fix in Fix Planner: one fix for a page template, the whole site, or the separate pages that have the issue. A broken link is one fix, with the pages that link to it. Fixes are ranked by severity, pages affected, how well linked they are and whether one change fixes them all, and each fix says why it ranks there. With Search Console linked, a fix that affects pages with more clicks ranks higher. Made the change? Click Check if fixed. A crawl reads the pages of that fix again, and only those pages are billed.
Can it monitor my SEO over time?
+
Yes. For SEO monitoring, schedule a crawl, weekly or daily, and each crawl is compared with the previous one in the same mode. The Changes tab lists pages that are new, gone or changed in status code, title, meta description, indexability or canonical, and broken links that are new or fixed. A fast crawl compares broken links only. A changed title or canonical shows in the Changes tab and is not sent as an alert. A scheduled crawl tells you when it finishes, finds broken links or fails: in the app, by email and by signed webhook.
Can I see Google Analytics numbers next to my pages?
+
Yes, on every plan, with no credits. Connect Google Analytics read-only and link a Google Analytics 4 property to a site in My Sites. Site Pages then shows each page's sessions and key events for the last 28 days, the change against the 28 days before, and how many sessions came from AI assistants as GA4 reports them. The Conversion tab uses the revenue Analytics reports for a landing page. PerfBee does not read users or visitor identifiers, and it never writes to Analytics.
Can I see Search Console numbers next to my pages?
+
Yes, with no credits. Connect Google Search Console read-only and link a property to a site in My Sites. Site Pages then shows each page's clicks, impressions and average position for the last 28 days, and the change in clicks against the 28 days before. It also shows Google's index status. Inspect now gives 10 index answers a day for each site. PerfBee reads search queries only if the workspace owner agrees to it. The numbers stay in your dashboard and in your own downloads.
Can it write my redirects?
+
Yes. The Redirects tab lists every page that answers 404 or 410, most linked first, with a suggested new address: the same page name in another folder, a close name that keeps the model code, or the nearest parent folder, never the homepage. Change or skip any row. Download them for Apache, Nginx, Cloudflare Bulk Redirects or Shopify, or as CSV and Excel. Check published redirects then reads each old URL once, billed like a fast crawl.
Can it make my sitemap?
+
Yes. The Sitemap tab builds a sitemap.xml from the pages that work and may be indexed, and lists what to remove from the sitemap you publish today, such as pages that answer 404, redirect or say noindex. It writes no lastmod, because PerfBee does not know when a page last changed. Download it as a file.
Can I see my site as folders?
+
Yes, with no credits. The Structure tab shows each folder with its pages, errors, broken links and click depth, as a folder tree, a depth chart and a treemap. Errors and warnings are known only for pages a detailed crawl read, and folders without such pages say so. Download the tree as CSV or Excel.
Can I group URLs into segments and compare two pages?
+
Yes, with no credits. A segment is a named group of URLs made from rules, such as every page of your blog, and Site Pages can show one at a time. Compare with another page reads two pages of your site live and shows what differs: status, title, description, canonical, headings, structured data and words. Neither is in the REST API, the assistant tools or live links.
Can I run my own AI prompts on my pages?
+
Yes, on Business and up, with your own key from OpenAI, Anthropic or Google Gemini. In Site Pages you select pages, write an instruction and run it. The model sees only what Site Pages stores for each page: the title, meta description, headings, a few technical fields and, if you tick it, the main text of the page. It never sees the images or the links, so review every answer. Your provider bills your account and PerfBee uses no credits. PerfBee never applies an answer to your site.
Can it test accessibility?
+
Yes, as a sample. Accessibility by template tests one example page of each of your biggest templates, up to 10 pages by default, against automated rules and groups the findings by rule and template. You see the pages and the credits first and confirm before anything is spent. Passing every rule does not make a site accessible.
Where do the third-party script numbers come from?
+
From your performance tests. Lighthouse loads the page in a real browser, so tags that Google Tag Manager adds after load are counted; a crawl of the HTML cannot see them. The Scripts tab adds up the newest test of each page: how much each service downloads and how long it keeps the main thread busy. The crawl adds scripts your pages still ask for that answer an error.
Does it render JavaScript and check JavaScript SEO?
+
Yes. Choose Off, Auto or Every page in the scan form; Off, plain HTML, is the default. Auto renders 5 sample pages at each URL depth with and without JavaScript, and renders the rest of that depth only if it finds at least 3 links the HTML lacked. Pages that are empty without JavaScript are always rendered. For rendered pages you can choose Desktop or Mobile, add an extra wait, or wait for an element to appear. The report shows where a rendered page differs from the server HTML, and the JavaScript errors it logged.
Can it crawl a staging site or pages behind a password?
+
Yes, on a site you verified in My Sites. Add a user name and password, or a request header such as a Vercel protection bypass. PerfBee sends them only to that site, over HTTPS. It does not log in through a form, does not get past bot protection and cannot reach a private network or VPN.
How big a site can it crawl?
+
There is no page limit. A crawl can run for up to 6 days, in 6-hour parts that pick up where the last one stopped. You can start inside one folder, exclude URL patterns, set a page cap for each URL pattern, such as 200 product pages, or check a pasted or uploaded list of up to 2,000 URLs. Be gentler slows a crawl to about one request a second.
Can I use it from code, a Google Sheet, Claude or ChatGPT?
+
Yes. The REST API, signed webhooks and the MCP server for Claude and ChatGPT are available. Reading costs no credits; a crawl you start through them spends credits like one in the dashboard. None of them can change schedules, and the API and webhooks only work on sites already in My Sites. Search Console numbers, accessibility results and segments are not in them. Live links keep a Google Sheet up to date with your newest crawl.
Can I send a client a white-label report?
+
Yes, on Starter and up. Set your name, colour and logo once in Settings, under Branding. A public share link to a finished crawl shows them and opens without an account; it is valid for 7, 30 or 90 days and you can revoke it. PDF and Excel exports carry your branding, and CSV file names start with your name. Impact Reports puts the same branding on a report of what you fixed and the clicks before and after.