Vulnerability scanners are tools used to find weaknesses or exploitable vulnerabilities in the infrastructure or code of a website.
Best practice for organizations is for penetration testers, sometimes known as white-hat hackers, to run vulnerability scanners against the new web pages with new functionality before they are ever deployed. Any vulnerabilities found by the penetration testers are quickly corrected before launching the new website to the public.
But there’s a flip side. Vulnerability scanners can be used just as effectively (or even more so) by the “bad guys”, otherwise known as black-hat hackers. The problem for websites comes from the fact that many websites are built using a variety of technologies so every update of those individual technologies has the potential to introduce new vulnerabilities. So every IT department is always playing catch-up to update software and, in an ideal world, they would be running a new vulnerability scan after each patch. Because of this, increasingly, hackers are using vulnerability scanners to find holes due to lapses in patching.
As a result of this weakness in the process of patching, recent industry research shows large spikes in unauthorized vulnerability scans during major zero-day events such as Heartbleed, Bash and Poodle—which is to be expected.
And the truth is that vulnerability scanners are automated tools also known as bots that are being used all over the internet looking for weaknesses to exploit.
Based on our annual bot report, 20% of traffic, on average, is bad bots which includes vulnerability scanners. It’s this 20% that causes the majority of website problems. These problems are quite diverse in nature and vary from website to website depending on what industry you’re in and what type of data may be valuable to hackers or competitors.
While we see large spikes in bot traffic, presumably from vulnerability scanners, when new vulnerabilities are disclosed, the reality is that these unauthorized scans are being run all the time and for the most part, unbeknownst to the website owner. With today’s dependence on e-commerce sites and online marketplaces, hackers can use relatively unsophisticated methods to commit fraud, hijack accounts, and even pluck valuable credit card information.
Opportunistic Vulnerability Scans
Opportunistic attacks involve a known or newly discovered vulnerability in a high-profile or highly-adopted system, program, or application. For example, let’s say there is a known vulnerability in a certain version of WordPress. Of course, WordPress will fix the gap and release a new version without the vulnerability. However, any websites that fail to update to the new, secure version of WordPress will continue to have the vulnerability. Now, malicious users can send a scanner to probe the internet looking for sites running that unsecured version of Wordpress. All they do is scan websites around the world looking for that particular vulnerability. Once they have a list of websites that are running the un-patched version, they now have a known target with a known vulnerability they can exploit. This is not a targeted attack. The hacker is not looking specifically to attack that particular website. The scan just added those websites onto a list of companies that have the known vulnerability and are worth further exploration. By preventing these types of vulnerability scans from hitting your website businesses can stop their website being on that list, and thwart the downstream effects of the continuation of the attack.
Zero Day Vulnerabilities
On the other hand, zero day vulnerabilities involve gaps in a system, program, or application, but are only known by the hackers themselves. These malicious users exploit the opportunity until the system’s owners and developers discover the vulnerability, fix the problem, and release a new, more secure and stable version. But as with the opportunistic attack described above, once the problem is announced, the publicity is both helpful and a curse.
On the one hand, you must quickly patch the vulnerability to reduce the attack surface and improve your security. But on the other hand, malicious attackers have been given a roadmap to a vulnerability at companies all over the world.
From the period of time between when it was announced and when the patch is available and widely deployed around the globe, there is an open window of potential exploitation when the vulnerability is there for any attacker to go after. And they do.
Characteristics of a Vulnerability Scanner
Hackers don’t have a roadmap to their victims. They use vulnerability scanners to help create the roadmap of areas to exploit. Many vulnerability scanners are tools that are free to download, while others are expensive to purchase. Some perform tasks with more efficiency than others. But overall, the high-level profile of a vulnerability scanning tool is the same:
- Automated tool - They are effectively an automated bad bot on your website.
- They are very noisy - Typically hit every page or directory, and can’t discriminate between pages.
- Systematically process pages - They go through a site unlike a typical human user so their path through the website is not normal.
- Browser automation tool - Can run in browser automation tool like Selenium, PhantomJS
- Access all code - Indiscriminately hit everything on page even deceptive links/code/forms
- Too Targeted - A random single request to one particular area of the site might be a probe for just one weakness.
Bot Defense for Vulnerability Scanners
Distil detects vulnerability scanners before they reap their damage on your website or APIs. Because vulnerability scanners are automated tools, they fail the checks and challenges that Distil puts before them that prove it is a real human using a genuine browser to access the website or API.
To prevent vulnerability scanners from accessing your website and API, businesses must use a tool built to distil real human traffic from automated threats.
About the Author
Edward Roberts leads Product Marketing and has over twenty years experience in technology marketing. Previously he worked for Juniper Networks, heading up Product Marketing for the Counter Security team. Before that he ran marketing for Mykonos Software, a web security company.More Content by Edward Roberts