Boxever Implements DevOps, Continuous Delivery On Amazon

8 Quiet Firsts In Tech In 2014

When Alan Giles co-founded Boxever, a travel experience personalization company, he knew from a prior role that he needed to build a culture around continuous software delivery. Boxever was going to supply the information with which major airlines would put their next product in front of the customer. Frequent changes would be needed to its core systems.

That's one reason he established Boxever on Amazon Web Services EC2. Not only did his firm avoid the capital expense of a data center, it also started to use a development environment that was a match for its production environment. That means many of the steps needed to move newly developed code into production can be automated, said Giles, CTO, in a telephone interview from Boxever's headquarters in Dublin, Ireland.

Boxever collects data on traveler details from client airlines -- where travelers are going, when they will arrive, who they are traveling with, what they will do upon arrival -- and deduces from that whether it's a business trip, a family trip, or a personal vacation. That guidance can be used by an airline to decide on which product to present to the traveler next.

To do that, Boxever must collect data on travelers in the US and in Europe, making sure to keep data collected in Europe either inside the country of origin or inside the European Union. "There's a lot more scrutiny than there used to be after incidents like (Edward) Snowden," he noted.

[ Want to learn more about DevOps in the Amazon cloud? See Amazon Cloud: 10 New Insights. ]

Nevertheless, Amazon, with its presence in Dublin and Frankfurt, Germany, as well as several US locations, meets all of his needs. He can spread his data across three separate locations in Ireland alone due to Amazon's zones. "Amazon was a strategic move for us. Its global presence is very important," he said.

Boxever uses three Cassandra open source NoSQL systems to collect travel data and store copies. "Our application sees the three data centers as one big distributed database," Giles said. An entire data center can go down and the Boxever traveler analytics service will stay up and running. Each version of Cassandra is in sync with the other two.

Boxever has adopted Amazon's CloudTrail Service to collect all the API calls that its virtual servers make from EC2 and other Amazon services, such as Elastic Block Store, Virtual Private Cloud, Relational Database Service, and Identity and Access Management service. By capturing a record of the API calls, Giles can see which Boxever applications are drawing the most on which services. The data is saved in Amazon S3 storage, or in the case of older data, in Amazon Glacier storage.

To see operations below the API calls -- CloudTrail stops at the host server and its operating system -- Boxever uses a complementary product, Logentries, to collect data on the virtual servers it's running and on their operating systems. Amazon can't inspect that data below the host system. It's up to the customer to construct a way to do so, said Giles. Through a Logentries API, Boxever can feed all log events related to its virtual servers into a preprocessing, tagging, and analysis engine capable of spotting a pattern that spells trouble and sounding an alert. The analysis works for a single server or groups of servers simultaneously, which is good for Giles' three-pronged system.

"When we started Boxever three years ago, we started using log files on day one. Our developers look at the log file reports every day on our virtual servers," he said. Those reports help tell Boxever how well its software is running.

Boxever is a small company of 37 people where the software development staff is also the operations staff. One engineer out of the 25-person development team is designated each week to serve as the on-call engineer -- and is the first line of defense if alerts go off. He's backed up by a more experienced escalation engineer. The experience gained in operations tends to be experience used in the next round of software development.

The airlines are constantly trying to figure how to appeal more directly to existing customers, with an upgrade or an offer that engages the customer with a special service. Is the customer taking an "open jaw" trip, where an airline flight will be combined with a train segment? That offers several opportunities to connect to the customer again.

Boxever is constantly changing its software to take advantage of new data or new ways of looking at the data. It makes changes to production systems daily through an automated unit test system, automated regression testing, and then "one-click deployment" to production by an administrator.

In previous jobs, Giles has been part of companies that prided themselves on agile development and frequent production system updates, with frequent meaning quarterly or at best, monthly. Now, when he goes to developer meet-ups in Dublin, "three or four updates a day is regarded as brilliant."

One change that needed to be implemented rapidly was for a customer that wanted a passenger's frequent flyer number added to a dashboard display. Giles knew that Boxever collected the data, but its client hadn't previously considered it important. Four hours later, the change was completed and the number started showing up. More subtle additions may be next, like what things the customer searched on while visiting the airline site. Either way, customers are happy to see an ability to respond quickly to recommended changes, said Giles.

To conduct operations with frequent changes would be impossible without feedback from Amazon's CloudTrail and the Logentries log data capture, said Giles. With them, members of the development team "are able to get more hands-on experience with different areas of the product."

There's a "balance between becoming a jack of all trades and having a particular development expertise," Giles said. The firm's development team maintains the product they've produced, "a big culture change with DevOps," Giles said.

That change would have been much more difficult without Boxever's launch and operation in the Amazon cloud, he said.

Network Computing's new Must Reads is a compendium of our best recent coverage of storage. In this issue, you'll learn why storage arrays are shrinking for the better, discover the ways in which the storage industry is evolving towards 3D flash, find out how to choose a vendor wisely for cloud-based disaster recovery, and more. Get the Must Reads: Storage issue from Network Computing today.