Today, Distil Networks released its sixth annual report into the trends in the bad bot landscape. Titled the “Bad Bot Report 2019: The Bot Arms Race Continues”, this publication is the industry's leading research into bad bots and their effect on businesses.
Key Findings from the 2019 Bad Bot Report:
- In 2018, bad bots accounted for 1 in 5 website requests (20.4 percent of web traffic). Good bots decreased slightly to make up 17.5 percent of traffic.
- 73.6 percent of bad bots are classified as Advanced Persistent Bots (APBs), which are characterized by their ability to cycle through random IP addresses, enter through anonymous proxies, change their identities, and mimic human behavior.
- Nearly 50 percent (49.9 percent) of bad bots report their user agent as Chrome. Mobile browsers, such as Safari Mobile, Android and Opera increased from 10.4 percent last year to 13.9 percent.
- Amazon is the leading ISP for originating bad bot traffic. In 2018, 18 percent of bad bot traffic originated from Amazon compared with 10.62 percent the previous year.
- Despite the fact that 53.4 percent of bot traffic originates from the United States, Russia and Ukraine combined make up nearly half (48.2 percent) of country-specific IP block requests.
Increasingly, the bot problem is an arms race. Bots are mimicking real human workflows across web applications to “behave” like real users. They are obfuscating their activity by reverse engineering detection systems. Advanced attackers now show definitive behavior that they know about the technology they’re trying to defeat, and they’re continuously learning how to adapt their tactics.
Targeted advanced persistent bots (APBs)include:
- Dedicated scraping tools targeted at avoiding digital rights management processes to gain free access to online books
- Systematic gift card balance checking
- Airline market intelligence operators running a dedicated scraping team unscrupulously attacking airlines in high volume and re-selling the gathered data.
- Airlines see dedicated bots rotating from attacking the website to mobile app to API and back ad nauseum to avoid any blocks impeding their goal.
This year’s report provides a comprehensive breakdown of some of the top industries impacted by bots and the specific challenges they face. Key findings include:
|Industry||Percentage of Bad Bot Traffic||Type of Attacks|
|Financial Services||42.2 percent||Credential stuffing to access or take over user accounts|
|Ticketing||39.3 percent||Scalping bots, seat inventory checkers, credential stuffing|
|Education||37.9 percent||Scraping for research papers, class inventory and user account access|
|Government||29.9 percent||Voter registration account interference, scraping business registration listings|
|Gambling and Gaming||25.9 percent||Scraping ever-changing betting lines, account takeover seeking loyalty points|
|Airlines||25.9 percent||Scraping pricing information, account takeover to empty airline mile balances|
|Ecommerce||18 percent||Price scraping, content scraping, account takeovers, credit card fraud and gift card abuse|
About the AuthorMore Content by Edward Roberts