Five Ways to Optimize Your WAF to Protect Against Bad Bots

July 15, 2015 John Stauffacher

This is a guest post by John Stauffacher, a world renowned expert in web application security, and the author of Web Application Firewalls: A Practical Approach. John is a certified Network Security and Engineering specialist with over 17 years of experience in IT Security.

Only 41% of web traffic originates from humans, so it’s important to be able to identify and police malicious bots. While your WAF may not have been purpose built to thwart bots, you can certainly tune your settings to catch the majority of bad actors. The approach below is not rocket science. You simply need a basic understanding of your customers and how Web clients work. It focuses on five ways to optimize your WAF to block malicious bots:

1.       User Agent Knowledge

2.       Geolocation Enforcement

3.       JavaScript No-op Usage

4.       Session Limiting

          5.       Login Page Behavior Monitoring

 

1. User Agent Knowledge

Let’s face it. You know who your customers are. By simply transferring that knowledge to your WAF, you can restrict web application access to your current and prospective customers. One way to do this is by defining a list of valid client user agents. Any user agent using Curl or a text-based reader, for example, would be quickly rejected, preventing unwanted application access.

2. Geolocation Enforcement

Similar to understanding valid customer user agents, you know where you do business. If you’re ecommerce site doesn’t ship to China then you can probably block the entire country. Letting Chinese traffic hit your site would provide no upside benefit, only risks to your security. Most WAFs allow you to set places from which traffic cannot access your site and apps.

3. JavaScript No-op Usage

Most websites and applications live and die by their ability to run and process JavaScript (JS). As such, your customers rely on JS to consume your valuable applications and content. Force site visitors’ web clients to prove they can process JavaScript by sending them a bit of JS no-op to process. Since some bots can handle a small amount of JS (enough to make the subterfuge last long enough to gain acceptance), make sure to send them something to process which you know a full browser would have.

4. Session Limiting

Many WAFs do not automatically kill lengthy user sessions. Set your WAF to do that. Your session limit settings should be based on the ways in which your customers use your applications. Set appropriate user limits for time, number of visits and traffic volume.

Additionally, make sure your application’s “Logout” button actually logs out users. Too many people assume users are logged out when the button sends them away to the home page. But often, users can return and maintain their session cookie to continue using your application.

5. Login Page Behavior Monitoring

Many people believe that establishing username and password controls is enough to protect login page intrusion. But you need to monitor your login page to know if false users like bad bots are hitting it and timing out or putting in multiple passwords to circumvent security. Simple monitoring will quickly identify brute force attacks and illegitimate users, such as those with a single user name that attempts to log in from multiple networks and geographies. By enforcing session revocation and “normal” page behavior, you will block a wide swath of threats that circumvented your browser checks.

 

To learn more about optimizing your WAF against bad bots, view the Ultimate WAF Torture Test Webinar.

About the Author

John Stauffacher

John Stauffacher is a world renowned expert in web application security, and the author of Web Application Firewalls: A Practical Approach. John is a certified Network Security and Engineering specialist with over 17 years of experience in IT Security.

More Content by John Stauffacher
Previous Article
Web Scraping : Everything You Wanted to Know (but were afraid to ask)
Web Scraping : Everything You Wanted to Know (but were afraid to ask)

Everything you wanted to know about web scraping. Advanced scraping technologies, industry targets, legali...

Next Article
A Testament to a Great Team
A Testament to a Great Team

A Testament to a Great Team - Distil Networks delivers results with great employees, supportive investors, ...