Study finds web scrapers can be employed for as little as $3.33 per hour
San Francisco, CA – August 30, 2016 – Distil Networks, Inc., the global leader in bot detection and mitigation, today released a new study titled, “The 2016 Economics of Web Scraping” which illustrates web scraping’s prevalence, sophistication, and industry use cases. Through analysis of top web scraping platforms and services, the report also outlines how the democratization of web scraping allows users to effortlessly steal sensitive information on the web.
Web scraping is a computer software technique for extracting information from web sites, and often includes transforming unstructured web site data into a database for analysis or repurposing content into the web scraper’s own website and business operations. In most cases, bots, which make up 46 percent of web traffic, are implemented by individuals to perform web scraping at a much faster rate than humans alone. Diverse actors leverage web scraping bots, including nefarious competitors, internet upstarts, hedge funds, fraudsters, hackers, and spammers, to effortlessly steal whatever pieces of content they are programmed to find, and often mimic regular user behavior, making them hard to detect and even harder to block. Roughly two percent of online revenue is lost as a result of web scraping.
“If your content can be viewed on the web, it can be scraped,” said Rami Essaid, CEO and co-founder of Distil Networks. “Not only does web scraping pose a critical challenge to a website’s brand, it can threaten sales and conversions, lower SEO rankings, or undermine the integrity of content that took considerable time and resources to produce. Understanding the pervasive nature of today’s web scraping economy not only raises awareness about this growing challenge, it also allows website owners to take action in the protection of their proprietary information.”
Key findings in the Distil Networks report reveal that:
38 percent of companies engaged in web scraping do so to obtain content
Web scraping is also used for research, contact scraping, price comparison, weather data monitoring, and website change detection.
Real estate sites are the number one web scraping victim
Other industries plagued by web scraping include digital publishing, travel, online directories, e-commerce, marketplace, and classifieds.
Web scraping services cost as little as $3.33 per hour
The average web scraping project costs roughly $135.
The average web scraper earns $58,000 annually
When working for a large company specializing in web scraping, some individuals gross up to $128,000 per year for their efforts.
To download a copy of the The 2016 Economics of Web Scraping study, visit here.
About Distil Networks
Distil Networks, the global leader in bot detection and mitigation, is the first easy and accurate way to identify and police malicious website traffic, blocking 99.9% of bad bots without impacting legitimate users. Distil protects against web scraping, brute force attacks, competitive data mining, online fraud, account hijacking, unauthorized vulnerability scans, spam, man-in-the-middle attacks, digital ad fraud, and downtime. Slash the high tax that bots place on your internal teams and web infrastructure and make your online applications more secure with API security, real-time threat intelligence, a 24/7 security operations center, and complete visibility and control over human, good bot, and bad bot traffic. For more information on Distil Networks, visit us at www.distilnetworks.com or follow @DISTIL on Twitter.