IoT
IoT
DevOps
News
12/31/2015
08:06 AM
50%
50%

Algorithms: Turning Data Into Products

Everybody talks about algorithms, but do you know what they are? We take a closer look at the origins of the word algorithm, what it means for business today, and how companies such as Google and Airbnb use them.

9 Scripting Languages You Need To Know
9 Scripting Languages You Need To Know
(Click image for larger view and slideshow.)

Blame Persian mathematician Al-Khwarizmi. Around 825 AD, he wrote a book entitled "Al-Khwarizmi on the Hindu Art of Reckoning," basically a math book using the numbers we use today. It was translated into Latin as "Algorithmi de numero Indorum." From this, we get our current word "algorithm".

Every software developer knows what an algorithm is -- a set of rules that list a set of instructions. The rest of us rely on algorithms for much of our daily Web and mobile interactions, though we're not always conscious of the important role they play.

In fact, at the Gartner CIO Symposium in October, the research firm's CEO Gene Hall proclaimed that we are living in "The age of the algorithm." While your ability to build effective algorithms depends upon the quality of your data, the data itself is effectively useless without algorithms that can tease out meaning. "Data is inherently dumb," said senior vice president of Gartner Research Peter Sondergaard at the October event. "It doesn't do anything unless you know how to use it and act with it. Algorithm is where the real value lies. Algorithms define action."

[ Find out why algorithms matter so much. Read CIOs Need To Welcome Age Of The Algorithm, Gartner Says.]

We recently spoke with developers from Google and Airbnb, two businesses whose fates rise and fall on the quality of their algorithms, to find out more about what algorithms are, and how they function as the underpinning of entire business models.

As with any development process, though, the action starts with defining your goals and what you're trying to achieve, said assistant professor Ariel Procaccia, in an interview. Procaccia teaches computer science at Carnegie Mellon University. "You start with the problem, not the algorithm."

Once you've identified the problem you're trying to solve -- or the business result you're trying to achieve -- the algorithm sets forth the steps that will get you where you want to go.

"An algorithm is an ordered set of instructions," said Kourtni Marshall, a software engineer for Google. "Writing down what happens after something happens should terminate in a satisfied condition." 

Algorithms rely on data. If you can get results from clear data, you can spot patterns, such as why a query is returning the wrong results, according to Surabhi Gupta, engineering manager at Airbnb.

For companies such as Google and Airbnb, algorithms are their "product." In the simplest terms, a Google search algorithm has to produce an effective search result. That search result is the company's product. Airbnb's algorithms have to connect travelers with those offering accomodations. Practical expediency governs how algorithms are used and referenced.

In the business world, algorithms are built for speed. "You are working with people with the expectation of immediate answers," said Google's Marshall. "Your algorithm has another constraint on it."

In the end, a developer is taking a sub-optimal solution that runs fast, and comparing it to the optimal solution to see how close the two compare, he added. If a problem is computationally simple, it can be solved quickly. If the problem is difficult, then the developer may have to compromise, preferring "good enough" instead of "perfect" in order to get the job done quickly enough, said Carnegie Mellon's Procaccia.

At Airbnb, the process of developing an algorithm involved a repeated build-test cycle, as the algorithm is improved to obtain a good result. Then the solution undergoes online A/B testing to see if there is any significant change between two possible solutions. Once all is cleared, the "product" is ready to go online, Gupta said.

Gupta and Procaccia both told us that if you know what you want to do, coding is not the hardest part.

(Image: DianaHirsch/iStockphoto)

(Image: DianaHirsch/iStockphoto)

Implicit in crafting the algorithm is "worst case" thinking -- how long will an algorithm take to run if given a sizable input? Procaccia illustrated the problem with the following example: What is the shortest route between New York and San Francisco? Drawing a straight line between the two points is easiest. Using a road map can be a little more difficult, but the Interstate Highway System provides a ready-made route that is pretty direct. Writing an algorithm that uses state and county roads only would be more complicated.

"Labeling the input here, that part is the easy part. The hard part is understanding what to do," Procaccia said. "The more elegant the idea, the more intuitive it can be."

Once you have a satisfactory algorithm, said Google's Marshall, it gets filed in a library for software engineers to reference. Should another project arise that is trying to solve a similar problem, that filed algorithm can become the basis for another solution, Marshall said.

It's also a good idea to revisit algorithms and "re-factor" them for improvement, Marshall said. While performing that re-factor is an ideal best practice, in reality, it's not done often. "If it is not broke, don't fix it" is an operative phrase for many developers.

Search queries propel a lot of Airbnb's business, as they do for Google. At Airbnb, refining the algorithm means looking at search results and looking for the conditions that produced lousy results, explained Gupta.

Gartner's Sondergaard in October described the algorithm economy as being about the "density of contact" with "your business at the center of a network." Whether you are talking a web of customers or a Web of millions of devices or "things," your business is a network of algorithms interacting together. Are you ready for the algorithm economy?

**Elite 100 2016: DEADLINE EXTENDED TO JAN. 15, 2016** There's still time to be a part of the prestigious InformationWeek Elite 100! Submit your company's application by Jan. 15, 2016. You'll find instructions and a submission form here: InformationWeek's Elite 100 2016.

William Terdoslavich is an experienced writer with a working understanding of business, information technology, airlines, politics, government, and history, having worked at Mobile Computing & Communications, Computer Reseller News, Tour and Travel News, and Computer Systems ... View Full Bio

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
Gary_EL
100%
0%
Gary_EL,
User Rank: Ninja
1/3/2016 | 3:46:18 PM
Re: Algorithm and data
And let's not forget good documentation. If conditions change, especially if the original algorithm writer is gone, good documentation can make the differerence between a smooth transition or the need for starting over from scratch.
Ariella
50%
50%
Ariella,
User Rank: Author
1/3/2016 | 11:20:11 AM
Re: Algorithm and data
< At Airbnb, refining the algorithm means looking at search results and looking for the conditions that produced lousy results, explained Gupta.>

That's a very important step in the process. Based on what I see in searches on many sites, they don't bother with that. It can be very frustrating to put in clear, simple terms and get back irrelevant results. 
Li Tan
50%
50%
Li Tan,
User Rank: Ninja
1/2/2016 | 1:44:22 AM
Algorithm and data
Alogorithm and data are both very important for a good software. More precisely, data model and algorithm should be designed/chosen together so that they fit well with each other. You cannot simply say algorithm A is always better than algorithm B - it depends on what data set you want them to operate upon.
Register for InformationWeek Newsletters
White Papers
Current Issue
Top IT Trends to Watch in Financial Services
IT pros at banks, investment houses, insurance companies, and other financial services organizations are focused on a range of issues, from peer-to-peer lending to cybersecurity to performance, agility, and compliance. It all matters.
Video
Slideshows
Twitter Feed
InformationWeek Radio
Archived InformationWeek Radio
Join us for a roundup of the top stories on InformationWeek.com for the week of August 14, 2016. We'll be talking with the InformationWeek.com editors and correspondents who brought you the top stories of the week to get the "story behind the story."
Sponsored Live Streaming Video
Everything You've Been Told About Mobility Is Wrong
Attend this video symposium with Sean Wisdom, Global Director of Mobility Solutions, and learn about how you can harness powerful new products to mobilize your business potential.