Bad bots are used by competitors, hackers and fraudsters and are the key culprits behind web scraping, brute force attacks, competitive data mining, online fraud, account hijacking, data theft, spam, digital ad fraud and downtime. In 2017, bad bots accounted for 21.8% of all website traffic, a 9.5% increase over the previous year. Good bots increased by 8.7% to make up 20.4% of all website traffic.
According to Distil Networks’ fifth annual Bad Bot Report, which details the analysis of hundreds of billions of bad bot requests at the application layer, gambling companies and airlines suffer from higher proportions of bad bot traffic than other industries, with 53.1% and 43.9% of traffic coming from bad bots, respectively. E-commerce, healthcare and ticketing websites meanwhile suffer from highly sophisticated bots, which are difficult to detect.
A full 83.2% of bad bots report their user agent as web browsers Chrome, Firefox, Safari or Internet Explorer; 10.4% claim to come from mobile browsers such as Safari Mobile, Android or Opera.
Additionally, 82.7% of bad bot traffic emanated from data centers in 2017, compared to 60.1% in 2016. The availability and low cost of cloud computing explains the dominance of data center use.
“This year bots took over public conversation, as the FBI continues its investigation into Russia’s involvement in the 2016 US presidential election and new legislation made way for stricter regulations,” said Tiffany Olson Jones, CEO of Distil Networks. “Yet as awareness grows, bot traffic and sophistication continue to escalate at an alarming rate. Despite bad bot awareness being at an all-time high, this year’s Bad Bot Report illustrates that no industry is immune to automated threats and constant vigilance is required in order to thwart attacks of this kind.”
For the first time, Russia became the most blocked country, with one in five companies (20.7%) implementing country-specific IP block requests. Last year's leader, China, dropped down to sixth place with 8.3%.