Understanding Bot Motives with Top Targeted Content Reporting

December 10, 2015 Orion Cassetto

Let’s face it, bots are a tricky problem. Not only are they difficult to stop, but they are also difficult to identify and understand. They don’t trip alarms on traditional security tools and they don’t leave evidence behind in audit logs. Their attacks are insidious, usually depending on automation combined with logical misuse or abuse of a website. This is in direct contrast to the application vulnerability exploit types of attack which WAFs protect against. Adding further complexity to the issue, attack motives are incredibly varied, ranging from competitive intel and content theft, to account takeover and online fraud.

Uncovering the Automated Motives of Bots

To help provide you with context on the bad bot activity on your website, we have recently released a new report in our product. This new report summarizes the top content that was targeted by bad bots during a given period. It is especially useful for understanding:

  • What attackers were after (e.g. login pages indicate account take over, pricing pages indicate competitive data-mining, etc.)
  • The impact bots have on your website.  How much is the targeted data worth to your business?

Without type of report, IT teams would need to sift through mountains of logs to correlate data and identify trends.

Using the Top Targeted Content Report to Investigate Your Bad Bot Problems

The new report is available within the Distil Networks Portal for all customers. To use it, simply select one of your domains and click on the “reports” drop down. At the bottom of this list you’ll find our new report, titled “Targeted Content”.

Once in the dashboard, you’ll be presented with the top 100 paths (URLs) accessed by bad bots, arranged by popularity. A quick search through the list should yield interesting results.  See your support portal listed? Maybe it’s a form spamming bot trying to muddy up your ticketing system. Did you find your product portfolio or your pricing page on this list? It might indicate that your competitor is trying to undercut your prices.

Looking at the screenshot below, we can see that the malicious bots visiting this website were interested in the login page and had visited it 7,191 times in a Target. This is a tell-tale sign of brute force account take over bots.


Distil Networks also provides you with detailed information about these bots including the detection mechanism which caught them and what enforcement step we took against the attacker. To view this info, simply click on the path you’d like to investigate.

Path Arrow

On the next screen you’ll find a chart titled “Threats breakdown by requests”, which essentially shows you a distribution of what detection methods we used to find the bots accessing this part of your website. It can be used to understand the relative sophistication of a bot and it’s methods. For example, we can see that 626 requests failed to load JavaScript. This is a symptom of basic or primitive bots.

Threat Breakdown

The second chart titled “Request Responses”, shows what steps we took to protect against the bots. Here we can see that we served around 773 CAPTCHA tests and 20 hard blocks.

Request Response

Fueled by Feedback

This feature came into existence thanks to your feedback. You asked for more context rich reporting to allow you to better understand the threats bots posed to your web environments and we listened.

Do you have an idea for a report or a feature that would make our product even better? We’d love to hear it. Feel free to reach out to our customer success team by submitting a ticket at http://help.distilnetworks.com/.

About the Author

Orion Cassetto

Orion Cassetto joined Distil Networks as Director of Product Marketing in 2015, bringing with him nearly a decade of experience in the Cyber Security industry. His strengths include competitive strategy, positioning, and messaging for web application security and SaaS-based security solutions.

More Content by Orion Cassetto
Previous Article
451 Report Reviews the Web Behavior Analytics Landscape
451 Report Reviews the Web Behavior Analytics Landscape

Web behavior analytics (WBA): driving out bots, cutting out fraud - 451 Research Report. Learn how these c...

Next Article
Digital Ad Fraud Prevention Using The Reverse Proxy Approach
Digital Ad Fraud Prevention Using The Reverse Proxy Approach

Learn the key differences between Reverse Proxy Approach & single pixel with ad approach for Digital Ad Fra...