Study: Web Scrapers May be More Prevalent than Many Thought

September 1, 2016

Web scraping, the act of extracting information from websites to create data which can then be used to analyze or repurpose content, may be more prevalent than many experts believed. That is one takeaway from The 2016 Economics of Web Scraping report which found that web scrapers can be used for less than $4/hour.

Web scrapers can be anything from competitors trying to get a leg up on a category, fraudsters and hackers, and can even mimic user behaviors. The data, from Distil Networks, shows that just over one-third (38%) of web scraping is done to get new content - research, price comparisons and other data, with real estate websites getting scraped the most.

"If your content can be viewed on the web, it can be scraped," said Rami Essaid, CEO and co-founder of Distil Networks. "Not only does web scraping pose a critical challenge to a website's brand, it can threaten sales and conversions, lower SEO rankings, or undermine the integrity of content that took considerable time and resources to produce. Understanding the pervasive nature of today's web scraping economy not only raises awareness about this growing challenge, it also allows website owners to take action in the protection of their proprietary information."

Read the Article

Previous Article
Portals Under Attack from Bad Bots, Reveals Report
Portals Under Attack from Bad Bots, Reveals Report

Real estate portals are the number one victim of malicious web scraping, with a 300 percent increase in bad...

Next Article
What's The Lost Revenue From Bots Scraping Original Content?
What's The Lost Revenue From Bots Scraping Original Content?

Distil Networks published a study this week titled "The 2016 Economics of Web Scraping."