Bad bots are a waste of server resources and can skew Google Analytics data.
Blocking bad bots in WordPress can speed up your site and prevent useless bots from constantly hitting your site. You would never know if spam bots were hitting your site unless you checked. In this tutorial, I will show you how to find bad bots using Wordfence, then block them either using Wordfence, Cloudflare Firewall Rules, or the Blackhole For Bad Bots plugin.
I don’t recommend Wordfence since it can be a slow plugin itself. Cloudflare Firewall Rules only let you block 5 bots (with the free plan) which can be a great start for most WordPress sites, and the Blackhole For Bad Bots plugin should block all spam bots not obeying the nofollow rule.
What Are Bad Bots And Why Should I Block Them?
Bad bots are any bot that hit your website at no benefit to you. These bots consume server resources especially if they hit your website or wp-login page excessively. Blocking them can put less stress on your server and potentially save you bandwidth, hosting costs, and speed up your site. This can can also prevent bad bots from appearing in your Google Analytics data.
1. Install Wordfence
Wordfence shows you all bots hitting your website in real-time.
You don’t need to leave this plugin enabled permanently; we will strictly be using it’s Live Traffic report to identify which bots are hitting your site and whether they should be blocked. Once we know the bad bots, we can block them without having to leave Wordfence installed.
2. View Your Live Traffic Report
Go to Wordfence → Tools → Live Traffic.
The live traffic report shows all bots hitting your site in real-time.
3. Identify Bad Bots Hitting Your Site
Observe your live traffic report for a few minutes to see if any suspicious bots are hitting your site repetitively. Make a list of their hostnames (shown in Wordfence) then Google their hostnames to see if other people reported are reporting them as a bad bot (you will need to do some research and make sure it’s actually a spam bot). Googlebot and other legitimate bots are OK and shouldn’t be blocked, but keep an eye out for suspicious ones and add them to your list.
After creating a list of all your spam bot’s hostnames, you have a few options for blocking them. I recommend the Blackhole For Bad Bots plugin since it’s automatic and will block any new bad bots in the future (which may not currently be on your list). Or if you only have a few primary spam bots hitting your site (up to 5 hostnames), you can also use Cloudflare’s Firewall Rules. Wordfence is great at blocking bad bots, but the plugin itself can cause a slow WordPress site.
4. Block Bad Bots With Wordfence
Wordfence has a variety of blocking options for blocking bad bots, but the plugin itself can make your WordPress site a little slower and you risk blocking legitimate humans/crawlers if the plugin isn’t configured correctly. Only use Wordfence if you’re comfortable configuring it.
Block Bad Bots By Hostname
- Go to the Blocking Settings and create a blocking rule
- Add the hostname of a bad bot you would like to block
- Use an asterisk (as shown below) to block all variations of that bot
- Create blocking rules for all bad bot hostnames from your live traffic report
Block Bad Bots With Rate Limiting
- Go to Wordfence → Firewall → Rate Limiting
- Configure the settings to limit “requests” and “pages viewed” by crawlers
- Be careful not to block legitimate bots/humans who don’t follow your rate limiting rules
Configure Wordfence Brute Force Protection
- Go to Wordfence → Firewall → Brute Force Protection
- Enable limit login attempts and prevent “admin” usernames
- Configure these settings to further secure your WP admin area
See A Log Of Spam Bots Being Blocked – once you configure Wordfence to block bad bots, you can see a login of all bots being blocked from your site, their hostnames, and their block count.
5. Block Bad Bots With Cloudflare Firewall Rules
Cloudflare Firewall Rules lets you block a maximum of 5 hostnames on the free plan.
Login to your Cloudflare Dashboard and go to Firewall → Firewall Rules → Create A Firewall Rule. Copy the bad bot’s hostnames (from Wordfence) and add it here in the “Value” field. Since you can create 5 rules, you would repeat this step for your 5 worst bad bots from Wordfence.
- Field = Hostname
- Operator = Contains
- Value = the hostname of the bad bot you found in Wordfence
You can see bots being blocked by Cloudflare in the Firewall Events tab:
6. Install The Blackhole For Bad Bots Plugin
The Blackhole For Bad Bots plugin stops bad bots by adding a hidden trigger link to the footer of your website that tells bots not to follow it. If they do, they will be blocked immediately from your website. Any legitimate bots (eg. Googlebot) will follow your rule and will not be blocked.
Step 1: Install The Blackhole For Bad Bots Plugin.
Step 2: In the plugin settings, copy the Robots Rules.
Step 3: Add the Robots Rules to your robots.txt file.
Step 4: Once you added the rule, go to your homepage and view the source code. Search the word “blackhole” and you should see the link created by the plugin. It should look like this:
<a rel="nofollow" style="display:none;" href="https://onlinemediamasters.com/?blackhole=2de810ae57" title="Blackhole for Bad Bots">Do NOT follow this link or you will be banned from the site!</a>
Step 5: In the plugin’s “Bad Bots” settings, you can view all bots that have been blocked.
7. Move Your WP Login Page
Some bad bots will try accessing your wp-login page. Even if they fail to gain access, they will still attempt this numerous times which is a waste of server resources. Since most spam bots aren’t complex, moving your WP Login page should help prevent the bots from even hitting it.
- Hide Login Page
- IThemes Security
- Remove Dashboard Access
- Some security plugins also have this feature
8. Limit Login Attempts
Limiting login attempts will lock out users and bots with too many failed login attempts on your wp-login page. This is just another way help block spam bots from excessively hitting your site.
- Limit Login Attempts Reloaded
- Some security plugins also have this feature
- Wordfence → Firewall → Brute Force Protection settings
Frequently Asked Questions
👾 What are bad bots exactly?
Bad bots are any bot that hit your website without any benefit, leading to a waste of server resources and possibly even skewed Google Analytics data.
👾 How do I check if bad bots are hitting my site?
Wordfence's Live Traffic report shows you all bots hitting your website in real-time. Here, you can determine whether there are suspicious bots hitting your site.
👾 How do I block them?
You can block them in Wordfence's Blocking and Rate Limiting settings. Alternatively if you don't want to use Wordfence, you block 5 hostnames using Cloudflare Firewall Rules, or install the Blackhole For Bad Bots plugin.
👾 Should I use Wordfence to block bad bots?
Only use Wordfence if you know what you're doing and are comfortable configuring the settings. It has excellent blocking options and can stop almost every bad bot, but the wrong configuration may block legitimate crawlers and even human visitors.
👾 Which plugins are best for stopping bad bots?
The Blackhole For Bad Bots plugin does an excellent job in stopping bad bots by adding a hidden trigger link to the footer of your website telling bots not to follow the link. If they do, they will be banned.
I hope this guide was helpful and that you’re able to block those pesky bots! If you have any questions whatsoever, leave me a comment below and I will get back to you as soon as I can.
See Also: How I Got 100% GTmetrix Scores