Bot Traffic and Your KPI

May 2, 2014 Charlie Minesinger

  

While Heartbleed and other catastrophic data breaches are in the news, nearly every website is violated by bot traffic on a daily basis and it is costing website operators billions of dollars.

Do you really have a bot problem?  Consider that one botnet, Pushdo, has more than 4 million IP addresses with computers infected but just about all those IP addresses also have legitimate consumer traffic.  In fact, it is commonplace for consumers and bots to share a device and IP address.  In the Bad Bot Landscape Report, Distil also found about 1,100 commercial ISPs have more than 70% of their traffic is automated user agents.  So have no doubt that launching bots is a large-scale commercial operation and it is pervasive across many industries.  It is not a question of whether or not you have bot traffic today, it is a question of how much.  On average, after removing traffic from search engine bots and monitoring services we see lows of 5% bot traffic, but typically we see 20% to 35% of requests on a website are generated by bots. 

Ask yourself…

  • What is acceptable an amount of bot traffic on your website?
  • How do you measure your bot traffic today?  In real time or after the attack?
  • What Key Performance Indicators (KPI) are affected by bot traffic today?  Revenue?

And then consider how bot traffic might affect your business and your career…

CONTENT THEFT (including IP Infringement)

  • Email harvesting – automating the capture of email addresses provided in lists and derived from contact information and then used for SPAM
  • Data Aggregation – taking data sets, running a set of automated queries to mine data through a website interface using standard consumer UI or generating a series of URLs
  • Content Replication – stealing your content to drive traffic to their site rather than yours and does not provide links back nor attribution (also called plagiarism)

>> KPI: Pagerank, Daily Pageviews, Marketing Costs AND Revenue

COMPETITIVE THREATS

  • Price scraping – Paging through your website to obtain a complete data set on every item you sell, current availability, and the price
  • Theft of Brand Assets – Taking branded images, specifications, and reviews that retailers use under license from the brands and manufacturers they distribute… this is then used for counterfeiting

>> KPI: Margin, Legal AND Revenue

FRAUD

  • Fraudulent registrations – the ability to autocomplete registrations, including pass of CAPTCHA and even phone verification steps to create certified accounts
    • Phone book stuffing on lead forms
    • Fake accounts to gain access to directories – LinkedIn, FB,
  • Form spam, using artificial intelligence to
  • Cookie trafficking, obtaining valid user cookies and

>> KPI: Margin, IT Support Costs, AND Lost Revenue

AUTOMATED TRAFFIC

  • Automated Click fraud – directing bots to click on ads and banners all over the web to drive fake traffic to a website and drive up marketing costs.
  • Brown outs – using automated browsers to make a high volume of requests on a website to slowdown the server
  • Artificial traffic for impression fraud – using bots to drive traffic to a domain to drive up the impression count on those domains

>> KPI:  Conversion rate, Click through rate, CPA, AND Marketing Cost

So, when Distil identifies 99.9% of bot page requests in real time, how can you use that information to improve your online business?  Your KPI?  Your career?

About the Author

Charlie Minesinger

Charlie Minesinger is the Director of Sales at Distil Networks focused on strategic accounts and channels. Charlie has over a decade of experience selling into enterprises and brand accounts, including Sprint, Disney, Nortel, Amazon, Cisco, AIG, Mattel. Charlie brings with him experience in start-ups and selling into new markets.

Follow on Twitter More Content by Charlie Minesinger
Previous Article
Here comes the cavalry
Here comes the cavalry

The engineering team at Distil Networks is growing with new members. Effective teamwork can produce gains t...

Next Article
Click Fraud Prevention: Does Your Directory Site Need It?
Click Fraud Prevention: Does Your Directory Site Need It?

If you own a directory site, click fraud should be a huge concern. By investing in a click fraud prevention...