Did you know you’re four times more likely to be hit with bad bots coming from Amazon AWS than any other internet service provider (ISP)? Amazon Web Services, which originates more bad bot traffic than the next five ISPs combined, is the global leader in infrastructure as a service. It is also the ISP of choice for bad bots in 2016 according to the Distil Networks Bad Bot Report 2017.
Data centers, like AWS, are the heavyweights of bad bot traffic generation—60.1% of bad bots come from data centers, 30.5% from residential ISPs such as Comcast and AT&T U-verse, and 9.4% from mobile carriers such as T-Mobile and KPN (a Dutch mobile provider).
The advantage of a data center based bot over that of a personal computer is the flexibility it affords bot operators. Instead of relying on software installed in browsers and operating systems, bot operators can build bad bots within data centers according to their own specifications.
Amazon Web Services the perfect environment for data center based bad bots. Within minutes, bad bot operators can sign up with a credit card, login to a private account, and begin spinning up servers. They can load these servers with open source tools like headless browsers or browser automation tools—along with scrapers, vulnerability scanners, login testing software, etc.
These trends have broadened the scope of bad bot use cases. Advanced persistent bots (APBs) can carry out sophisticated attacks, such as account-based abuse and transaction fraud, which require multiple steps and deeper penetration into the web application.
Why You Don’t Want to Block Amazon Web Services (or any other ISP) On Your Own
In the bad bot report, we recommend blocking data center traffic to lower the number of bad bots hitting your site. The logic being that end users on personal devices connect to websites via residential and mobile networks, not ISPs like Amazon Web Services and Microsoft Azure.
While it is possible to block data centers, it is an enormous task to handle by yourself. In this example we zoom in on the challenges blocking Amazon Web Services poses because it is by far the biggest offending ISP.
As the leader in its space, Amazon Web Services hosts millions of websites and is continuously adding and releasing IP ranges. In order to keep up with this dynamic IP infrastructure you will either have to program complex automated scripts that dynamically update your blacklists or do so manually.
Either way you’re looking at a time consuming and onerous process and taking either action adds complexity into your web traffic flow that could lead to unexpected errors, which would most likely result in downtime.
Assuming you’ve evaluated those risks and have decided to proceed anyway, you can’t just blindly block ranges broadcast by AWS. If you look at one of their main subnets 22.214.171.124, you’ll find the /10 is owned by another entity, Merck and Co.
In this instance Amazon just leased smaller blocks from Merck and Co’s cloud. If you aren’t careful, you will wind up blocking far more than just Amazon’s IP range—you may wind up locking out customers and/or partners.
As of this blog, Amazon has a list of over 1,000 subnets to parse through. Because they use elastic IP addressing, the route broadcast by a partner is liable to change—adding another layer of complexity.
Also, you must be absolutely certain no one in your organization relies on tools or services built on top of AWS—otherwise you will end up disrupting others within your company. AWS is incredibly popular, and there’s a reason half the internet drops offline when AWS has downtime.
How to Block ISPs (and More) Using Distil Networks Universal Access Control Lists
Taking all these risks into account, Distil Networks Universal ACLs (access control lists) enables you to blacklist and whitelist access to your protected web and API domains from ISPs like Amazon AWS. Rather than manually tailoring an ACL for each new domain, Distil offers a GUI based tool for creating universal ACLs for your APIs, websites, and web apps. This features helps to:
- Block all attempts by malicious users
- Allow all attempts by approved users
Blocking Amazon (or any ISP we have listed) is as easy as creating an ACL that blacklists problematic ISPs via organizations. You can even apply it to specific paths in your domain (e.g., http://www.example.com/sign_in) to block requests coming from suspect or temporary ISPs.
In addition to blocking ISPs, the Universal ACL enables blacklisting or whitelisting by: countries, user agent, API token, Distil generated device IDs, and HTTP referer.
Once configured, you can tailor a series of ACLs according to your business needs and practices. For example, create an ACL whitelisting your internal tools via API tokens or IP addresses. That way you don’t accidentally block partners or third-party tools. You can even limit blocking to specific URL paths such as your API-specific URLs (e.g., api.example.com) to ensure that only authorized users have access.
To help you get started, Distil publishes pre-configured ACLs so you can apply Distil-curated and-recommended whitelists and blacklists to your domains.
Would you like to see how it’s done in the Distil Networks customer portal? Here is a fantastic overview video. Want to find out how else our technology can protect your assets from malicious automation? Sign up for a free portal demo.
About the Author
Peter Zavlaris weighs in on various topics around bot mitigation, bot defense sharing white papers, videos and other resources on the topic.More Content by Peter Zavlaris