hello I have this quick task to do to reduce my hosting resources usage
I have investigated your case and I found that the hight CPU usage is due to bots. You can block all bots in your [login to view URL] file using the following code:
Code:
User-agent: *
Disallow: /
For the ones that does not check this file, you can set an .htaccess rule. In your specific case you can set:
Code:
RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^bot\*$
RewriteRule .* - [F,L]
This will block the access of a bot identified by the name " bot* ".
For an empty user agent string you can use the following rule:
Code:
RewriteCond %{HTTP_USER_AGENT} ^$
RewriteRule ^(.*)$ - [F,L]
The bad bots you can block via the following .htaccess' firewall:
Code:
# 6G FIREWALL/BLACKLIST
# @ [login to view URL]
# 6G:[QUERY STRINGS]
RewriteEngine On
RewriteCond %{QUERY_STRING} (eval\() [NC,OR]
RewriteCond %{QUERY_STRING} (127\.0\.0\.1) [NC,OR]
RewriteCond %{QUERY_STRING} ([a-z0-9]{2000}) [NC,OR]
RewriteCond %{QUERY_STRING} (javascript:)(.*)(;) [NC,OR]
RewriteCond %{QUERY_STRING} (base64_encode)(.*)(\() [NC,OR]
RewriteCond %{QUERY_STRING} (GLOBALS|REQUEST)(=|\[|%) [NC,OR]
RewriteCond %{QUERY_STRING} (<|%3C)(.*)script(.*)(>|%3) [NC,OR]
RewriteCond %{QUERY_STRING} (\|\.\.\.|\.\./|~|`|<|>|\|) [NC,OR]
RewriteCond %{QUERY_STRING} (boot\.ini|etc/passwd|self/environ) [NC,OR]
RewriteCond %{QUERY_STRING} (thumbs?(_editor|open)?|tim(thumb)?)\.php [NC,OR]
RewriteCond %{QUERY_STRING} ('|")(.*)(drop|insert|md5|select|union) [NC]
RewriteRule .* - [F]
# 6G:[REQUEST METHOD]
RewriteCond %{REQUEST_METHOD} ^(connect|debug|delete|move|put|trace|track) [NC]
RewriteRule .* - [F]
# 6G:[REFERRERS]
RewriteCond %{HTTP_REFERER} ([a-z0-9]{2000}) [NC,OR]
RewriteCond %{HTTP_REFERER} ([login to view URL]|todaperfeita) [NC]
RewriteRule .* - [F]
# 6G:[REQUEST STRINGS]
RedirectMatch 403 (?i)([a-z0-9]{2000})
RedirectMatch 403 (?i)(https?|ftp|php):/
RedirectMatch 403 (?i)(base64_encode)(.*)(\()
RedirectMatch 403 (?i)(=\'|=\%27|/\'/?)\.
RedirectMatch 403 (?i)/(\$(\&)?|\*|"|\.|,|&|&?)/?$
RedirectMatch 403 (?i)(\{0\}|\(/\(|\.\.\.|\+\+\+|\"\")
RedirectMatch 403 (?i)(~|`|<|>|:|;|,|%|\|\s|\{|\}|\[|\]|\|)
RedirectMatch 403 (?i)/(=|\$&|_mm|cgi-|etc/passwd|muieblack)
RedirectMatch 403 (?i)(&pws=0|_vti_|\(null\)|\{\$itemURL\}|echo(.*)kae|etc/passwd|eval\(|self/environ)
RedirectMatch 403 (?i)\.(aspx?|bash|bak?|cfg|cgi|dll|exe|git|hg|ini|jsp|log|mdb|out|sql|svn|swp|tar|rar|rdf)$
RedirectMatch 403 (?i)/(^$|(wp-)?config|mobiquo|phpinfo|shell|sqlpatch|thumb|thumb_editor|thumbopen|timthumb|webshell)\.php
# 6G:[USER AGENTS]
SetEnvIfNoCase User-Agent ([a-z0-9]{2000}) bad_bot
SetEnvIfNoCase User-Agent ([login to view URL]|binlar|casper|checkpriv|choppy|clshttp|cmsworld|diavol|dotbot|extract|feedfinder|flicky|g00g1e|harvest|heritrix|httrack|kmccrew|loader|miner|nikto|nutch|planetwork|postrank|purebot|pycurl|python|seekerspider|siclab|skygrid|sqlmap|sucker|turnit|vikspider|winhttp|xxxyy|youda|zmeu|zune) bad_bot
Order Allow,Deny
Allow from All
Deny from env=bad_bot
# 6G:[BAD IPS]
Order Allow,Deny
Allow from All
# uncomment/edit/repeat next line to block IPs
# Deny from 123.456.789
More information about it you can find on the link below:
[ur][login to view URL][/code]
You can block multiple bots with such a rule. For example, if you want to add block for google bot in the same RewriteRule just add:
Code:
RewriteCond %{HTTP_USER_AGENT} (googlebot|bingbot|Baiduspider) [NC]
You can also tweak the Googlebot crawl rate from your Google Webmaster Tools account.
Hey,
I'm an experienced WordPress developer and can assist you reduce your website hosting resources usage. Apart from blocking bots, you need to set up caching. Caching, also helps in reducing hosting resources.
Kind Regards,
Anthony
Hey Friend,
Myself having has 10 + years experience of web site development.
Expert in Custom programming in php / Mysql / Jquery / Wordpress/ Magento / Joomla / OSC / OpenCart.
Developed hundred's of websites and software's.
and many more....
Lets Discuss on project.
I run in to this issue on a regular basis. There are two solutions to this problem. The more complicated solution is to do exactly what you have listed. I can alter the htaccess file with the setting you provided. The better option is to install a few security plugins which will implement those rules for you. This way you can turn the rules on/off with out the risk of making a mistake. I'm open to doing this either way.
I have 10+ years development experience with Wordpress. 15+ years development experience with PHP, MySQL and Javascript. Wrote my first web page way back in 1995. I have the experience you need to get this done efficiently and quickly. I can definitely fix this for you.
Thanks,
-Dave
I have to advise you about this. Using ModRewrite in Apache to do this is actually very CPU intensive compared to other methods. I've built and managed enough sites to know.
You'll get better results by using a reverse proxy such as Cloudflare.
For the best effectiveness I recommend placing your site behind a reverse proxy which will offload the CPU and traffic being caused by bots. Is your site performing slowly or is it more a situation where your hosting provider is charging you extra because of the high load? If you're interested I can help you do this.
Please feel free to chat with me, your .htaccess code seems there are similarities with the plugin "all-in-one security". Siteground provide additional plugins to maximize CPU usage. other than that, you also have to do some other things based on advice from Siteground to maximize CPU usage. Are you already doing that.
Regards