Why StubHub Needs Distil to Proactively Defend its Marketplace Against Bots
One of the biggest challenges online retailers face today is the seemingly endless wave of bots designed to scrape prices, place fraudulent transactions, launch brute force attacks, commit click fraud, and perform other nefarious tasks. To help retailers combat the bot threat, Retail TouchPoints invited Distil Networks and StubHub to present a webinar on how the two companies have been collaborating to fight back as part of their Retail Strategy & Planning series.
Senior RTP Editor Alicia Fiorletta introduced Distil Networks’ Co-founder and CEO Rami Essaid and StubHub Senior Director of Technology Operations Marty Boos. Marty has over 25 years’ experience building and maintaining high-performance web infrastructure systems for leading retailers.
Why Dealing with Bots is critical to Online Retailers
Marty spent much of the webinar sharing his experiences dealing with bad bots in one of the world's largest and most dynamic online marketplaces. Much of this wisdom could be applied to any website looking to protect itself from automated threats, but it's especially pertinent to online retailers and ecommerce websites. Here are some of the top takeaways Marty left us with:
Bots perform competitive arbitrage. Constantly. Competitors use bots to scrape price lists, product descriptions, vendor lists and inventory information in real-time, adjusting their own prices to lure your customers away
Account takeovers and transaction fraud are bigger than ever. By combining stolen username and password lists from major security breaches like Ashley Madison with clever automation, bots can be created which takeover user accounts, make fraudulent purchases, and perform validity checks for stolen credit cards, all of which hurt online retail businesses. As these breaches grow in numbers, so do the password lists.
Low entry barriers contribute to growth in bot usage. Easy access to botnet rentals and turnkey scraping tools are prevalent in the public domain. Any kid in his basement can begin attacking your site with little to no formal training.
Outsourcing beats Homegrown. Marty spent three years and countless man hours trying to solve the bot problem with in-house solutions but in end decided that using a 3rd party tool was more cost effective due to the highly-dynamic and distributed nature of the attackers.
WAFs don’t solve the problem. Web application firewalls, a security solution commonly deployed in attempts to corral bad bos, provide application security value with their static rule sets but they can’t handle bots. You can use your WAF to block 10,000 IP addresses but a week later, these bots will have 10,000 new ones. The problem is too dynamic for WAFs to tackle.
The problem is big, and getting bigger
Distil’s 2015 Bad Bot Report found that up to 60% of ecommerce site traffic is bad bots, and bot traffic is growing as fast, if not faster, than ecommerce traffic:
The more opportunities for online buying and selling, the more opportunities to subvert the process. Dynamically changing pricing, availability, descriptions, and vendor reviews are valuable commodities to competitors.
Anyone can do it. There are plenty of free scraping tools (or scrapers-for-hire), and 1,000 compromised computers can be rented on the black market for less than $12 an hour.
Bots cycle through random IP addresses and hide behind anonymous proxies in an endless game of hide-and-seek. An attack can move from 10,000 hits from two IP addresses/hour to two hits from 10,000 IP addresses/hour in seconds.
StubHub’s report on CAPTCHAs served against CAPTCHAs solved is telling of how many visitors are actually automated clients, unable to solve, or even attempt these challenges:
It’s time to get ahead of the game
The problem is too big for reactive, home-grown solutions. Collaboration between solution providers and successful retail sites like StubHub provides the foundation for a proactive and effective pushback against the bad guys.
StubHub first became aware of the problem when brute force account takeovers surged – as a reseller of virtual goods, StubHub is particularly vulnerable to this type of attack, which the bad guys use to turn stolen credit cards into cash before the cards are cancelled.They began to see more attempts by competitors scraping prices and monitoring inventory and customer behavior. Unaffiliated groups were stealing data and openly selling it, damaging legitimate partner relationships. Unpredictable spikes in pageviews were skewing analytics and impacting site resource usage. Marty’s wishlist for a solution was growing fast:
Must block scrapers without impacting human visitors – a much more difficult task with today’s browser-based bots
Must accurately separate good bots from bad bots so that partners and their own customer service agents, as well as search bots, can access the site unhindered
Must include automated learning that adjusts protection as threats morph, ending the endless whack-a-mole cycles
Must have a way to “crowdsource” information about emerging threats while protecting individual site identities
Must seamlessly co-exist with other web security tools (even better, replace some of them)
Learn more about StubHub and Distil’s partnership for a universally safer ecommerce environment in this case study.
Do you know what’s running on your site?
Distil Networks is offering two months’ free service, no strings attached, and a deep dive with an analyst. To take advantage of this, go to http://www.distilnetworks.com/trial
About the AuthorMore Content by Orion Cassetto