In the first Internet revolution, affectionately referred to as Internet Bubble, it was the parlance to disintermediate, e-enable, and aggregate… and blurt out other nonsensical combinations of nouns, verbs, and adjectives, etc… in a stream of unexpurgated grammatical cluster fun. In the first round of revolution, travel agents were indeed real casualties and the distribution of airline tickets, hotel rooms, and travel as a whole was transformed.
Now, more than a decade after the Internet Bubble, some would lead you to believe that the Internet has democratized access to information and leveled the playing field, “the world is flat”. We would also like to believe that we are getting the best price when we buy online (because it is easy to compare prices online), but in reality that is not the case either. Bots, crawlers, web scrapers, screen scrapers, spiders, etc are not working for travelers in most cases; bots are not transparent, democratizing agents for good – bots are malicious, drive up costs, and reduce options for consumers.
Today, the ‘user agent’ is still driving up the cost of travel, it is a robotic automated ‘user agent’, scraping pricing info, making reservations, etc.
Have no doubt, bots are driving up travel costs and limiting choices for consumers. Let’s plan a trip to see how this works – fly to another city to catch a concert, and I will explain how bots affect our travel plans all along the way – Airfare, dinner, and concert tickets.
First, we need to buy our airline ticket. If you go to a “meta search site” you will get a view of options from a number of airlines. Seems like a great idea, but in fact, the bots are deceiving you. Take the case of one of Europe’s largest airlines, easyJet, which concludes this: “final fares quoted by eDreams were up to 60% more expensive than a ticketwould have been if booked directly with easyJet.” Ouch. As a follow-on to this, consumer transactions through these bot-powered metasearch travel sites create customer care headaches between travelers and airlines.
Time to eat. Of course we are from out of town so we search for a great restaurant and we are on a schedule, so lets use Open Table. Well, guess who has reserved our table already, a bot. Yes, one restaurateur comments “In under a minute, all the reservations were being taken.” What? We cannot get a table? One could argue that bidding for a table reservation is a “market efficiency”, but I just want to compete against humans on OpenTable. “Perhaps it’s that [bots were] supposed to make life easier is being capitalized on by a small group of technically competent people, shutting out the average user,” comments a restaurateur. It is hard to see how consumers and foodies win here… Anyways, lets get to the concert now.
Buying tickets for the concert… that is secure from bots, right? No, that “CAPTCHA” (the distorted text you have to type in) is in fact pretty much useless for stopping bots (see our blog by Andrew) Our friends at Ticketmaster are not immune from bots. In fact, bots snap up the best seats and sell them at a premium on secondary market. So we are paying well above face value and we are competing, again with a bot.
There is also a real cost for operators in the travel industry – about 30% of web traffic to travel sites, on average, is bots scraping price and availability data. On our network, we can see that most travel websites receive from 12% to 40% of their page requests from bots. Based on our little research project, we found 150,000 bots were trolling travel websites. Check out our full Travel Infographic to get the whole picture.
The problem of bots in the travel industry is not just a few bad actors, it is a pervasive large scale problem affecting the airlines, hotel chains, cruises, restaurants, and event/concert promoters… and all of us consumers making vacation plans. It is time for a new Internet Revolution – time to distermediate bots, level the playing field (again) and give the Internet back to the humans.
About the AuthorFollow on Twitter More Content by Charlie Minesinger