Title: Forge AI Bot Log – Block AI Bots, Crawler Detection & Verification Author: forgecreative Published: 28 Luj 2026 Last modified: 09 Otover 2026 --- Search plugins ![](https://ps.w.org/forge-ai-bot-log/assets/banner-772x250.png?rev=3629318) ![](https://ps.w.org/forge-ai-bot-log/assets/icon.svg?rev=3629318) # Forge AI Bot Log – Block AI Bots, Crawler Detection & Verification By [forgecreative](https://profiles.wordpress.org/forgeonline/) [Download](https://downloads.wordpress.org/plugin/forge-ai-bot-log.1.6.4.zip) * [Details](https://lmo.wordpress.org/plugins/forge-ai-bot-log/#description) * [Reviews](https://lmo.wordpress.org/plugins/forge-ai-bot-log/#reviews) * [Installation](https://lmo.wordpress.org/plugins/forge-ai-bot-log/#installation) * [Development](https://lmo.wordpress.org/plugins/forge-ai-bot-log/#developers) [Support](https://wordpress.org/support/plugin/forge-ai-bot-log/) ## Description **Any script can claim to be GPTBot. This plugin checks whether it really is.** Most AI bot trackers read the user-agent string and take it at face value — so a scraper that sends `User-agent: GPTBot` gets logged as OpenAI. Forge AI Bot Log verifies each visit against the crawler’s own published infrastructure — the IP- range lists OpenAI, Anthropic and Perplexity publish, and forward-confirmed reverse DNS for Google, Apple, Amazon and Meta — and labels every hit **Verified**, **Spoofed** or **Unverified**. You find out which AI traffic genuinely came from OpenAI, and which was something else wearing its name. Verification covers every crawler that publishes a way to check it, including OpenAI, Anthropic, Google, Perplexity, Amazon, Apple and Meta. Bots with no published verification method are labelled Unverified rather than guessed at. Beyond verification, **Forge AI Bot Log** shows you exactly which AI crawlers and assistants are visiting your WordPress site and explains what each one actually does. The bundled directory covers 26 bots from 16 companies, including GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, Google-Extended, GoogleOther, PerplexityBot, GrokBot, CCBot, Bytespider, Amazonbot, Applebot, Applebot-Extended, meta-externalagent, cohere-ai, Diffbot, YouBot, MistralAI-User and DuckAssistBot. #### Key features * **Live dashboard** — bot visits for the last 24 hours, 7 days, 30 days, plus a complete searchable log. * **Bot directory** — click any bot for a plain-English explanation of who owns it, whether it trains AI models on your pages, powers AI search, or fetches pages live, and whether it obeys robots.txt. * **Most-crawled content** — see which of your pages AI bots are reading most. * **Verified / Spoofed labelling** — every tracked hit is checked against the vendor’s published IP ranges or by forward-confirmed reverse DNS, whichever that crawler’s owner publishes, so impersonation is flagged instead of trusted. Results are cached and throttled to keep it off the critical path. * **Access control** — block any bot with one click. Blocking adds a `Disallow:/` directive for that crawler to your robots.txt, and there is a hard-block `.htaccess` snippet for the bots that ignore robots.txt. * **Monthly email summary** — sent to the site admin on the 1st of each month at 9am, covering the previous month. * **New-bot alerts** — optional email the first time a brand-new AI bot appears. * **Reliable capture** — an in-PHP logger, an optional must-use early logger, and a server access-log importer for sites behind full-page caching or a CDN. * **Privacy-friendly** — only logs bots, not human visitors, with optional IP anonymisation and configurable data retention. * **CSV export** — download the full or filtered activity log for your own analysis. #### Why it matters AI crawlers may use your content as training data, cite you in AI search answers, or read your pages on a user’s behalf. Which of those you want is your call — but you cannot make that call without seeing the traffic first. Forge AI Bot Log makes AI bot activity visible and gives you the controls to manage it. #### Who this is for Publishers and bloggers deciding whether to opt out of LLM training. Site owners who suspect content scraping and want evidence. Agencies monitoring client sites, who need to monitor crawler activity across a portfolio. Anyone who has read that they should block AI bots but wants to know which bots are actually visiting before blocking anything. If you want an AI bot blocker that shows you its work rather than a blocklist you have to trust, this is it. Google-Extended and Applebot-Extended are opt-out tokens rather than crawlers in their own right, and the plugin treats them as such — so you can decline LLM training without touching normal search crawling. #### Powered and maintained by Forge Forge AI Bot Log is powered and maintained by [forge.co.nz](https://forge.co.nz/), a New Zealand web design and development studio. We build and support the plugin, keep the bundled crawler list current with each release, and read every support thread. ### External services Crawler IP-range lists. To confirm that a visit claiming to be an OpenAI, Anthropic or Perplexity crawler really came from that company, the plugin downloads the IP- range list each vendor publishes for exactly this purpose, and compares the visiting IP against it. The lists are fetched at most once every 24 hours and cached in your database; only the request for the public list is made, and no site, user or visitor data is transmitted. This happens only while bot verification is enabled in Settings. The lists fetched are: * OpenAI — https://openai.com/gptbot.json, https://openai.com/searchbot.json and https://openai.com/chatgpt-user.json. Terms: https://openai.com/policies/terms- of-use/ — Privacy: https://openai.com/policies/privacy-policy/ * Anthropic — https://claude.com/crawling/bots.json. Terms: https://www.anthropic. com/legal/consumer-terms — Privacy: https://www.anthropic.com/legal/privacy * Perplexity — https://www.perplexity.ai/perplexitybot.json and https://www.perplexity. ai/perplexity-user.json. Terms: https://www.perplexity.ai/hub/legal/terms-of- service — Privacy: https://www.perplexity.ai/hub/legal/privacy-policy Plugin news opt-in. If you tick “Plugin news” in Settings or the first-run setup screen, your site’s admin email address is sent once to Forge’s own server (forge. co.nz), which adds it to our mailing list so we can email you about new features and updates. Nothing is sent unless you check this box, and it is unticked by default. Terms: https://forge.co.nz/terms/ — Privacy: https://forge.co.nz/privacy/ ## Screenshots [⌊The dashboard: bot visits for the last 24 hours, 7 days and 30 days, a visits- per-day chart split by purpose, and your most active bots and most-visited pages.⌉⌊ The dashboard: bot visits for the last 24 hours, 7 days and 30 days, a visits-per- day chart split by purpose, and your most active bots and most-visited pages.⌉[ The dashboard: bot visits for the last 24 hours, 7 days and 30 days, a visits-per- day chart split by purpose, and your most active bots and most-visited pages. [⌊The full activity log — every AI bot visit with time, page, status and verification result: Verified, Spoofed, Unverified or No check.⌉⌊The full activity log — every AI bot visit with time, page, status and verification result: Verified, Spoofed, Unverified or No check.⌉[ The full activity log — every AI bot visit with time, page, status and verification result: Verified, Spoofed, Unverified or No check. [⌊The bot directory: all 26 bots with who owns them, what they do, and whether they're blocked.⌉⌊The bot directory: all 26 bots with who owns them, what they do, and whether they're blocked.⌉[ The bot directory: all 26 bots with who owns them, what they do, and whether they’re blocked. [⌊Access Control — block any bot with one click. Blocking adds a robots.txt rule automatically, and a ready-to-paste .htaccess snippet covers bots that ignore it.⌉⌊ Access Control — block any bot with one click. Blocking adds a robots.txt rule automatically, and a ready-to-paste .htaccess snippet covers bots that ignore it.⌉[ Access Control — block any bot with one click. Blocking adds a robots.txt rule automatically, and a ready-to-paste .htaccess snippet covers bots that ignore it. [⌊Settings — choose what's recorded, configure email summaries and alerts, and import your server's access log.⌉⌊Settings — choose what's recorded, configure email summaries and alerts, and import your server's access log.⌉[ Settings — choose what’s recorded, configure email summaries and alerts, and import your server’s access log. ## Installation 1. Upload the `forge-ai-bot-log` folder to `/wp-content/plugins/`, or install via Plugins Add New. 2. Activate the plugin. 3. Visit **Forge AI Bot Log** in the admin menu. 4. (Optional, for cached sites) Copy `mu-plugin/forge-ai-bot-log-mu.php` into `/wp- content/mu-plugins/` for earlier capture, or use Settings Import from server access log. ## FAQ ### How do I stop ChatGPT from using my content? Block GPTBot to stop OpenAI collecting your pages as training data for its models, and block OAI-SearchBot and ChatGPT-User if you also want to keep your content out of ChatGPT’s search and live page-fetching. The plugin lists all three separately, because most people want to opt out of training without disappearing from AI search. Blocking writes the relevant robots.txt directives for you. ### Can I tell if a bot claiming to be GPTBot is fake? Yes — that is the main reason this plugin exists. A user-agent string is trivial to forge, so any scraper can send `GPTBot` and appear in your logs as OpenAI. On each tracked visit the plugin checks the visiting IP against whichever proof that crawler’s owner publishes. For OpenAI, Anthropic and Perplexity that is their official IP-range list. For Google, Apple, Amazon and Meta it is a reverse DNS lookup on the visiting IP, re-resolving the hostname it gets back to confirm it maps to the same address and to the crawler’s published domain. Genuine crawlers pass; spoofing does not. Hits are labelled Verified, Spoofed or Unverified so you can see the difference. ### How do I block AI bots in robots txt? Tick the bots you want to block on the plugin’s Access page. The directives are merged into the robots.txt WordPress serves, so you never edit a file by hand. One caveat worth knowing: if a real `robots.txt` file exists in your site root, your web server serves that file directly and WordPress never runs — so the plugin’s rules will not appear. Delete or fold in that physical file if you want the plugin to manage your AI crawler directives. ### Does blocking a bot really stop it? robots.txt is advisory. Well-behaved bots obey it; some (Bytespider, Perplexity- User and Grok’s fetchers, in the bundled list) are reported to ignore it. For those the plugin generates an `.htaccess` hard-block snippet that rejects them at the server level, on Apache. ### Does blocking Google-Extended remove me from Google Search? No. Google-Extended controls whether your content is used to train and ground Google’s Gemini models. Normal Google Search crawling and indexing is done by Googlebot, which this plugin does not touch, so your search rankings are unaffected. Note also that blocking Google-Extended does not remove you from AI Overviews, which are generated from Google’s regular search index. ### Which AI crawlers can it detect? 26 bots from 16 companies — OpenAI, Anthropic, Google, Perplexity, xAI, Amazon, Apple, Meta, ByteDance, Cohere, Mistral AI, DuckDuckGo, You.com, Diffbot, Timpi and Common Crawl — split into AI training crawlers, AI search indexers and live retrieval agents. Each has its own entry explaining what it does with your pages. The crawlers behind the big assistants are all covered, so ChatGPT, Claude, Gemini, Grok and Perplexity traffic is identified by name rather than lumped together. ### Grok read my site but nothing was logged — why? xAI is the least transparent of the major AI companies. Its documented user agents( GrokBot, xAI-Grok, Grok-DeepSearch) are in the bundled directory and are logged when they appear, but independent monitoring consistently finds that much of Grok’s real fetching arrives under generic Chrome or Safari user-agent strings from rotating IP addresses, which no user-agent-based tracker can attribute. Grok can also answer questions from its search partner’s existing index without visiting your site at all. So treat Grok’s numbers in any tool — this one included — as a minimum. ### Does this slow down my site? No. Logging is limited to matched AI bots and runs on an early hook, and verification is throttled and cached — vendor IP-range lists are fetched at most once a day and stored locally. LLMs are not consulted at any point; detection is local pattern matching against the bundled list. ### Will it catch bots if I use a caching plugin or CDN? Full-page caches can serve bots without running PHP. Use the included must-use early logger and/or the access-log importer to capture those hits. ## Reviews ![](https://secure.gravatar.com/avatar/f7a9f3421ae687e092636c470e7c7d8235d3b513e5555fd8c616f09c6e579ee1? s=60&d=retro&r=g) ### 󠀁[Excellent, easy-to-use AI bot dashboard](https://wordpress.org/support/topic/excellent-easy-to-use-ai-bot-dashboard/)󠁿 [Perry Bernard](https://profiles.wordpress.org/pfbernardnz/) 25 Setember 2026 This plugin provides both a fascinating and useful feedback point on AI bot traffic that is otherwise not easily observable. I'll be using this going forward to get a better understanding of how our site IA engineering is affecting access frequency and will likely influence how we apply GEO-specific changes. [ Read all 1 review ](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/) ## Contributors & Developers “Forge AI Bot Log – Block AI Bots, Crawler Detection & Verification” is open source software. The following people have contributed to this plugin. Contributors * [ forgecreative ](https://profiles.wordpress.org/forgeonline/) [Translate “Forge AI Bot Log – Block AI Bots, Crawler Detection & Verification” into your language.](https://translate.wordpress.org/projects/wp-plugins/forge-ai-bot-log) ### Interested in development? [Browse the code](https://plugins.trac.wordpress.org/browser/forge-ai-bot-log/), check out the [SVN repository](https://plugins.svn.wordpress.org/forge-ai-bot-log/), or subscribe to the [development log](https://plugins.trac.wordpress.org/log/forge-ai-bot-log/) by [RSS](https://plugins.trac.wordpress.org/log/forge-ai-bot-log/?limit=100&mode=stop_on_copy&format=rss). ## Changelog #### 1.6.4 * Added: a one-screen setup wizard shown once right after activation, offering three starting choices (check bots are genuine, shorten IP addresses, plugin news), each already available in Settings. Skippable, and never reshown. * Added: an optional “Plugin news” opt-in in Settings and the setup wizard. Unticked by default; only when checked does your site admin email get sent once to Forge’s mailing list. See External services. #### 1.6.3 * Added: a dedicated “Pro” page in the menu describing what the separate Pro add- on adds, alongside the existing short mentions in Settings and in the emails. Disappears automatically once Pro is active. * Changed: the Pro add-on mentions now link to its pricing page instead of straight to checkout. #### 1.6.2 * Fixed: Access Control blocks are now built once at plugin load and reach the robots.txt served to bots, not just the preview shown in the admin. A build taken during development had regressed to admin-only construction, so blocked bots were never actually kept out. * Fixed: reverse-DNS verification no longer labels a genuine bot “Spoofed” when the address checked belongs to an unrecognised proxy rather than the bot itself; it’s reported “Unverified” instead, as in earlier releases. * Fixed: uninstalling now also clears the scheduled new-bot-alert event, so nothing is left pending after removal. * Performance: the reverse-DNS verification cache is larger (500 to 5,000 entries) and results are trusted for longer (1 hour to 6 hours), so busy sites crawled by Google, Amazon, Apple or Meta from many different addresses don’t force repeated live DNS lookups. * Added: a mention of the Forge AI Bot Log Pro add-on — a fuller description in a panel at the bottom of Settings, and a single quiet line at the bottom of the Dashboard, Activity Log, Bot Directory and Access Control screens, plus the monthly summary and new-bot alert emails. Hidden automatically once Pro is active, and can be turned off everywhere at once with the `forge_ai_bot_log_show_pro_notice` filter. #### 1.6.1 * Added: extension hooks (actions and filters) so add-ons such as Forge AI Bot Log Pro can attach reports and capture without duplicating the core plugin. No change to tracking behaviour. * Added: the access-log importer now exposes each parsed line, including the referrer, to add-ons, and no longer counts a bot visit twice when it was already recorded as it happened. A line for the same bot and page within two minutes of an existing record is skipped. * Added: a warning when a caching plugin or a managed host (WP Engine, Kinsta, Flywheel, Pantheon) is active, explaining that bot visits to cached pages are not seen by WordPress, with the bot names to paste into the cache’s exclusion setting or the alternative of importing the server log file. * Redesigned: every screen now uses one consistent layout of headings, number cards, panels, tables and coloured tags, shared with add-ons, with a “?” icon beside every number and column that needs explaining. Hover, tab to it or tap it for a plain-language explanation. * Added: the dashboard chart splits bot visits per day into live reads, search crawls, training crawls and agents. * Changed: bot types are named in plain words: Live read, Search crawl, Training crawl, Agent. * Changed: “Most-visited pages” counts address variants of the same page together, so /about, /about/ and /about/?ref=1 are one row. * Fixed: Google-Extended and Applebot-Extended are now labelled as opt-out names rather than bots. They are names used in robots.txt to control AI training and never visit a site themselves. They can still be blocked under Access Control but are no longer offered for recording. * Fixed: tables no longer squash neighbouring panels on narrower screens; wide tables scroll sideways instead. * Fixed: when the older pre-release “AI Bot Log” plugin was also active, its stylesheet loaded on this plugin’s screens instead of this plugin’s own, so help text showed inline and the layout looked broken. This plugin now uses its own asset names and switches the older plugin’s styles off on its screens. #### 1.3.0 * Added: xAI’s Grok crawlers — GrokBot, xAI-Grok and Grok-DeepSearch — to the bot directory, with an honest note that much of Grok’s fetching uses disguised browser user agents no tracker can attribute. * Security: forwarding headers (X-Forwarded-For, X-Real-IP, CF-Connecting-IP) are no longer trusted from arbitrary clients — previously a request could forge a vendor IP in a header and have a fake crawler labelled Verified. Cloudflare’s header is now honoured only when the request genuinely arrives from Cloudflare’s published ranges, and generic proxy headers only behind a local reverse proxy. * Fixed: hits captured by the optional early logger (mu-plugin) now record the visitor’s IP address and are labelled “No check” instead of being mislabelled“ Unverified”. * Fixed: the early logger now honours the excluded-paths and IP-anonymisation settings, matching the main logger. * Performance: the early logger no longer reads the signature list for ordinary human traffic. * Removed: the cross-promotion email settings and the optional remote promo feed. Existing promo settings are cleaned up automatically. #### 1.2.4 * Fixed: OpenAI, Anthropic and Perplexity crawlers are now verified against the vendors’ published IP-range lists instead of reverse DNS, which those vendors don’t support — genuine ClaudeBot/GPTBot hits were previously shown as Spoofed/ Unverified. #### 1.2.3 * Documentation: expanded and clarified the readme, with new FAQs on blocking, verification and robots.txt handling. * Documentation: corrected the verification description — hits are labelled Verified, Spoofed or Unverified, not only the first two. * Documentation: clarified that robots.txt directives are merged into the robots. txt WordPress serves, and that a physical robots.txt file in the site root overrides them. * Internal version constant bumped to match the plugin header, so browsers re-fetch the plugin’s CSS and JavaScript once. No other functional changes. #### 1.2.2 * Fixed: genuine bots visiting over IPv6 are no longer incorrectly marked “Spoofed” by DNS verification. #### 1.2.1 * Removed the remote bot-list auto-updater; the bundled bot list now updates via plugin releases. #### 1.2.0 * Added an External services disclosure for the optional promo feed. * Added a Settings shortcut to the Plugins screen. * New-bot alert emails are now sent on a background schedule instead of during the visit. Older entries (1.0.0 – 1.1.1) are in `changelog.txt` in the plugin folder. ## Meta * Version **1.6.4** * Last updated **6 days ago** * Active installations **70+** * WordPress version ** 5.8 or higher ** * Tested up to **7.1.3** * PHP version ** 7.4 or higher ** * Language * [English (US)](https://wordpress.org/plugins/forge-ai-bot-log/) * Tags * [LLM](https://lmo.wordpress.org/plugins/tags/llm/)[robots.txt](https://lmo.wordpress.org/plugins/tags/robots-txt/) [scraper](https://lmo.wordpress.org/plugins/tags/scraper/)[Training Data](https://lmo.wordpress.org/plugins/tags/training-data/) * [Advanced View](https://lmo.wordpress.org/plugins/forge-ai-bot-log/advanced/) ## Ratings 5 out of 5 stars. * [ 1 5-star review ](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/?filter=5) * [ 0 4-star reviews ](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/?filter=4) * [ 0 3-star reviews ](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/?filter=3) * [ 0 2-star reviews ](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/?filter=2) * [ 0 1-star reviews ](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/?filter=1) [Your review](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/#new-post) [See all reviews](https://wordpress.org/support/plugin/forge-ai-bot-log/reviews/) ## Contributors * [ forgecreative ](https://profiles.wordpress.org/forgeonline/) ## Support Got something to say? Need help? [View support forum](https://wordpress.org/support/plugin/forge-ai-bot-log/)