Handling Malicious Bot Traffic

Bad or malicious bot traffic can easily have a negative impact on the performance of your Relewise solution. To combat malicious bots, we have gathered a few tips below to help you work around the issue.

Please note that we at Relewise are not experts in protection against malicious bots, and that if you are having persistent issues of this nature, we refer to any of the excellent companies serving countermeasures.

For further reading about bots and how to counter the bad ones, refer to these articles:

Adsterra: What Is Bot Traffic: How to Detect and Stop Bots

Cloudflare: What is bot management?

Imperva: 9 Recommendations to Prevent Bad Bots on Your Website

Identifying Malicious Bots

Typically, you will detect malicious bot traffic in your web analytics service such as Google Analytics, on the basis of strange traffic spikes, unusual bounce rates, unusual session lengths, etc. Services like Google Analytics filter out the majority of benign bot traffic by default, which means you will not see a majority of the known and helpful bots in your overview. In order to identify remaining bot traffic, we refer you to this guide which offers a substantial amount of tips on how to identify and combat bot traffic via Google Analytics directly.

Relewise is not designed to combat bots, but you may notice unusual patterns in incoming search terms or user behavior. If you see repeated search queries in your Live Search View section of MyRelewise which contain strange symbols or strings, odds are that you are dealing with a bot testing the security of your system.

Compare Useragent

The simplest way to combat malicious bot use is to compare the useragent to a list of known bad actors. This is not a foolproof method by any means, but can serve to weed out the majority with very little effort.

Be aware that maintaining a useragent list can be very time-consuming, and that site performance can begin to suffer the longer the list becomes.

For pre-made lists that are ready to be installed, we refer to this list on Github or this far larger list, both of which may serve as starting points.

Be mindful that you do not simply block bots for the sake of it; many bots are beneficial and obey robots.txt.

Throttling Requests per Minute

If your setup can handle it, you may also consider throttling the amount of requests you permit per minute per IP. This necessitates that you track all requests made. Throttling in this way can run the risk of accidentally affecting human users, as well as blocking benign bots from performing their beneficial services. It is a tool to be used carefully, according to the needs of your website.

Professional Assistance

Relewise are not experts on combating malicious bots, and we urge you to reach out to a professional service if you find your website under persistent attack. Hiring a professional service can help build a much more sophisticated defense against malicious bot intrusions, as well as leave your hands free to pursue more important work.

Relewise is not affiliated with any bot mitigation services, and cannot offer a list of qualified partners. However, a quick search for "bot protection software" on your favored search engine is bound to lead you in the right direction.