Algorithms: Turning Data Into Products

9 Scripting Languages You Need To Know

Blame Persian mathematician Al-Khwarizmi. Around 825 AD, he wrote a book entitled "Al-Khwarizmi on the Hindu Art of Reckoning," basically a math book using the numbers we use today. It was translated into Latin as "Algorithmi de numero Indorum." From this, we get our current word "algorithm".

Every software developer knows what an algorithm is -- a set of rules that list a set of instructions. The rest of us rely on algorithms for much of our daily Web and mobile interactions, though we're not always conscious of the important role they play.

In fact, at the Gartner CIO Symposium in October, the research firm's CEO Gene Hall proclaimed that we are living in "The age of the algorithm." While your ability to build effective algorithms depends upon the quality of your data, the data itself is effectively useless without algorithms that can tease out meaning. "Data is inherently dumb," said senior vice president of Gartner Research Peter Sondergaard at the October event. "It doesn't do anything unless you know how to use it and act with it. Algorithm is where the real value lies. Algorithms define action."

[ Find out why algorithms matter so much. Read CIOs Need To Welcome Age Of The Algorithm, Gartner Says.]

We recently spoke with developers from Google and Airbnb, two businesses whose fates rise and fall on the quality of their algorithms, to find out more about what algorithms are, and how they function as the underpinning of entire business models.

As with any development process, though, the action starts with defining your goals and what you're trying to achieve, said assistant professor Ariel Procaccia, in an interview. Procaccia teaches computer science at Carnegie Mellon University. "You start with the problem, not the algorithm."

Once you've identified the problem you're trying to solve -- or the business result you're trying to achieve -- the algorithm sets forth the steps that will get you where you want to go.

"An algorithm is an ordered set of instructions," said Kourtni Marshall, a software engineer for Google. "Writing down what happens after something happens should terminate in a satisfied condition." 

Algorithms rely on data. If you can get results from clear data, you can spot patterns, such as why a query is returning the wrong results, according to Surabhi Gupta, engineering manager at Airbnb.

For companies such as Google and Airbnb, algorithms are their "product." In the simplest terms, a Google search algorithm has to produce an effective search result. That search result is the company's product. Airbnb's algorithms have to connect travelers with those offering accomodations. Practical expediency governs how algorithms are used and referenced.

In the business world, algorithms are built for speed. "You are working with people with the expectation of immediate answers," said Google's Marshall. "Your algorithm has another constraint on it."

In the end, a developer is taking a sub-optimal solution that runs fast, and comparing it to the optimal solution to see how close the two compare, he added. If a problem is computationally simple, it can be solved quickly. If the problem is difficult, then the developer may have to compromise, preferring "good enough" instead of "perfect" in order to get the job done quickly enough, said Carnegie Mellon's Procaccia.

At Airbnb, the process of developing an algorithm involved a repeated build-test cycle, as the algorithm is improved to obtain a good result. Then the solution undergoes online A/B testing to see if there is any significant change between two possible solutions. Once all is cleared, the "product" is ready to go online, Gupta said.

Gupta and Procaccia both told us that if you know what you want to do, coding is not the hardest part.

Implicit in crafting the algorithm is "worst case" thinking -- how long will an algorithm take to run if given a sizable input? Procaccia illustrated the problem with the following example: What is the shortest route between New York and San Francisco? Drawing a straight line between the two points is easiest. Using a road map can be a little more difficult, but the Interstate Highway System provides a ready-made route that is pretty direct. Writing an algorithm that uses state and county roads only would be more complicated.

"Labeling the input here, that part is the easy part. The hard part is understanding what to do," Procaccia said. "The more elegant the idea, the more intuitive it can be."

Once you have a satisfactory algorithm, said Google's Marshall, it gets filed in a library for software engineers to reference. Should another project arise that is trying to solve a similar problem, that filed algorithm can become the basis for another solution, Marshall said.

It's also a good idea to revisit algorithms and "re-factor" them for improvement, Marshall said. While performing that re-factor is an ideal best practice, in reality, it's not done often. "If it is not broke, don't fix it" is an operative phrase for many developers.

Search queries propel a lot of Airbnb's business, as they do for Google. At Airbnb, refining the algorithm means looking at search results and looking for the conditions that produced lousy results, explained Gupta.

Gartner's Sondergaard in October described the algorithm economy as being about the "density of contact" with "your business at the center of a network." Whether you are talking a web of customers or a Web of millions of devices or "things," your business is a network of algorithms interacting together. Are you ready for the algorithm economy?

**Elite 100 2016: DEADLINE EXTENDED TO JAN. 15, 2016** There's still time to be a part of the prestigious InformationWeek Elite 100! Submit your company's application by Jan. 15, 2016. You'll find instructions and a submission form here: InformationWeek's Elite 100 2016.