Distil Networks today published its 2019 Bad Bot Report, which looks at the current state of automated web traffic, both good and bad. The paper highlights some startling figures, including that bots accounted for 37.9 percent of all online traffic during 2018. This is a admittedly a startling number, but it actually represents a welcome decrease on last year’s figures.
This decrease can be found across legitimate bot traffic (like, for example, the scrapers travel sites use to identify flight deals) as well as ill-intentioned bots (like those used to conduct denial-of-service attacks). These slumped by 14.4 percent and 6.4 percent respectively. The share of human traffic, on the other hand, grew by 7.5 percent to 62.1 percent.
It’s not clear why this shift took place, and Distil doesn’t try any qualitative analysis to figure out why. One plausible example could be the increasing levels of Internet access, largely coming from developed nations, which is bolstered by the proliferation of affordable smartphones and low-cost data plans.
The paper does an excellent job of highlighting the challenges faced by those combating bot traffic. The current generation of bad bots are described as Advanced Persistent Bots (APBs), which have characteristics that make it difficult to mitigate against. APBs try to obfuscate their origin by relying upon anonymous proxies and other identity-hiding technologies, while simultaneously trying to appear to target sites as legitimate human traffic.