Ecommerce – Your Conversion Rate is Off by 7.8%

April 26, 2017 Peter Zavlaris

If you’re struggling with low conversion rates and above average cart abandonment, your site may have an advanced persistent bot (APB) problem. APBs are sophisticated bad bots that scrape from sites—without permission—so as to reuse the data (e.g., pricing, inventory levels) and gain a competitive edge.

APBs are used against payment processors, logins, and web forms. Sites that possess one or more of these attributes are almost certain to have skewed analytics. Meanwhile, the truly nefarious ones undertake criminal activities, such as fraud and outright theft.

In 2016, bad bots accounted for 15.6% of web traffic on Ecommerce sites and about half (7.8%) were APBs. Good bots, such as search engine crawlers, application performance tools, and scanners accounted for 9.3% of traffic. The remaining 75.2% was human.

Unfortunately for Ecommerce, bad bots prefer sites with pricing and proprietary data, such as those offering product descriptions, logins, web forms, and payment processors. Data from our Bad Bot Report 2017 shows 97% of websites having pricing and proprietary data get scraped.

96% of logins are hit with bad bots looking to takeover accounts or test stolen login credentials collected from other sites. Customer review sections aren’t spared, either; 31% of web forms are hit by spammer bots that post competitors’ ads and negative reviews.

Why Analytics Tools are Fooled by Bad Bots

Analytics and tracking tools used by conversion tracking rely on code snippets for user identification. These amount to snippets of JavaScript code injected into the visitor’s browser, their primary function being to ping back information to analytics programs in real-time. The data is used to compute interaction, intent tracking, buying patterns, etc.

Over half of all bad bots (55.7%) can load JavaScript and are tagged by such trackers. Say your Ecommerce site has 100k visitors per day. If 15.6% of those are bad bots, and half of those (7.8%) are bad bots loading JavaScript, your tracking tools count 7,800 bad bots per day as potential customers—that’s 234,000 bad bots per month.

Whether your site is above or below average, there is a 94% chance it was visited by APBs in the last year, according to our Bad Bot Report 2017.

How Bad Bots Skew Conversion Rates

Bad bots go to great lengths to avoid detection and penetrate deeper into applications. Our report found that they log in as customers and carry out internal attacks in 9 out of 10 applications. Once behind the login page, bad bots can access shopping carts to scrape more exclusive product pricing, resulting in skewed cart abandonment metrics.

APBs are used to arbitrage deals on competitor sites. They reserve inventory on one site and post the same items for sale elsewhere. They only complete the transaction on the first site when an item sells on another, thereby skewing conversion rates because carts containing items that don’t sell are abandoned.

Bad bots get away with such activity because they can pass as humans—thanks to browser automation technology. Using such tools (e.g., Selenium), APBs can do everything from executing JavaScript and accepting and storing cookies to emulating mouse clicks and varying click patterns. They appear to be valid human customers and are tracked accordingly.

How Can You Tell If You Have a Bad Bot Problem?

Unexpected and volumetric traffic spikes are the most obvious way to tell if bad bots are attacking your site. This is particularly true if the source of the traffic spike is unusual, such as from a foreign country you never do business with or from proxy networks.

A bad bot penetration can spike traffic numbers beyond what your infrastructure can handle, leading to an application denial of service. But that is generally a side effect of an aggressive bad bot campaign, not its real goal.

Some bad bots are becoming increasingly more efficient as their operators utilized ‘low and slow’ techniques. That is, attacks where numerous bots make only small requests or few requests, generating less noise thereby making them much harder to detect.

The only way to be confident you have accurate conversion tracking is to eliminate all interference caused by bad bots. To this end, Distil Networks is the only easy and accurate bot mitigation solution. By eliminating bad bots, we clean up your analytics so you can finally determine your true conversion rates.

Case Study: Learn how Hayneedle, a leading online provider of home furnishings and décor, cleaned up its conversion tracking and A/B testing using Distil Networks.

Want to know how bad bots attack? You can find comprehensive definitions and examples of the dozens of automated threats posed by bad bots in this blog post on the Open Web Application Security Project’s (OWASP) in depth study bad bot threats.

What is the risk from bad bots on your business? Quantify the annualize risk to your ecommerce website with our risk calculator.


About the Author

Peter Zavlaris

Peter Zavlaris weighs in on various topics around bot mitigation, bot defense sharing white papers, videos and other resources on the topic.

More Content by Peter Zavlaris
Previous Article
The 4 Reasons You Have Bad Bots
The 4 Reasons You Have Bad Bots

If you have just 1 of these features, your site is vulnerable to attacks from malicious bots. Learn more.

Next Article
The 4 Things You Need to Know About Application Denial of Service (DOS attacks)
The 4 Things You Need to Know About Application Denial of Service (DOS attacks)

Protect your website from application denial of service attacks (app dos). Learn about dos protection and h...