Real estate portals are the number one victim of malicious web scraping, with a 300 percent increase in bad bot activity from 2014-2015 according to a new report from Distil Networks, a specialist in bot detection and mitigation.
The 2016 Economics of Web Scraping report revealed companies lose millions each year as a result of web scraping, defined as a computer software technique to extract information from the internet, usually transforming unstructured data on the web into structured data that can be stored and analyzed in a central database.
The practice has been around for as long as the internet and, according to the report, but is increasing exponentially, particularly in the real estate sector.
The main type is content scraping (38 percent) followed by research, where 26 percent of companies that hire web scrapers to monitor consumer opinions about products and companies. And 19 percent of companies use web scraping for contact scraping to gain access to customers’ emails.