Bad bots are big news largely because of the FBI investigation into Russia's involvement in the 2016 presidential election. But bad bots are a bigger problem than automated tweeting: 42.2% of all website traffic comes from bots; and 21.8% of it is down to bad bots.
Distil Networks' 2018 Bad Bot Report, based on an analysis of hundreds of billions of bad bot requests, shows that bad bot traffic increased by 9.5% in 2017. Bad bots differ from good bots, whose traffic also increased by 8.8% to 20.4%. It means that only -- on average -- 57.8% of visiting traffic comes from a genuine human being interested in the website content.
Good bots are those that all websites require. They include the search engine page indexing bots from Google and Bing, and they bring humans to the site. Bad bots, however, are secretive and nefarious. They come from outright criminals and commercial competitors; and their purpose is to detract and/or steal from the website.
Distil highlights eight different bad bot functions: price scraping, content scraping, account takeover, account creation, credit card fraud, denial of service, gift card balance checking, and denial of inventory. They fall into three primary categories: competitive, organized criminal, and nuisance.
Price scraping and content scraping are generally competitor attacks. Price scraping allows competitors to maintain price levels slightly lower to score more highly in search engine rankings. Content scraping is simply the theft of proprietary content to augment another site's own content.