Most IT leaders have heard of deep learning, but few really understand how this new technology works.
Deep learning burst onto the public consciousness in 2016 when Google’s AlphaGo software, which was based on deep learning, beat the human world champion at the board game Go. Since then, deep learning has begun appearing in news reports and product literature with more frequency, but few organizations are actually using it today.
The 2018 O’Reilly survey report How Companies Are Putting AI to Work Through Deep Learning found that only 28% of the more than 3,300 respondents were currently using deep learning. However, 92% believed that deep learning would play a role in their future projects, with 54% saying it would play a large or essential role in those initiatives.
As companies plan to move ahead with those projects, the number one challenge holding them back is a lack of skilled people. The report noted, “Leaders in AI have long talked about the need to make deep learning accessible to developers without a Ph.D. That’s essential to progress; AI must become accessible to domain experts in other disciplines.”
The following slideshow doesn’t even scratch the surface of what it takes to become an expert in deep learning. But it does provide a high-level overview of the topic and covers the basics that CIOs, IT managers, and business leaders need to understand about this emerging technology.
[You can learn more about deep learning at Interop ITX, opening April 30 at the Mirage in Las Vegas. Check out the related sessions in the Interop AI Summit and the Data and Analytics Track.]
1. Deep learning is a subset of machine learning.
Sometimes people use the terms artificial intelligence (AI), machine learning, deep learning, and cognitive computing as if they meant the same thing. In actuality, consensus suggests that only two of those terms — AI and cognitive computing — are synonyms.
AI (and cognitive computing) is a broad category that involves teaching computers to think the way people think.
Machine learning is the subcategory of AI that involves teaching computers to get better at various tasks without being explicitly programmed; essentially, it enables computers to improve their performance without humans telling them what to do.
Deep learning is a specialized type of machine learning that uses a hierarchical approach. It’s a set of algorithms — math — that has proven particularly good at solving certain types of computing problems that are difficult to define with explicit programming.
2. Deep learning is fairly new.
In his paper Deep Learning: A Critical Appraisal, New York University professor Gary Marcus explained, “Although deep learning has historical roots going back decades, neither the term ‘deep learning’ nor the approach was popular just over five years ago.” While scientists have been working on ways to make machines learn since the invention of the first computers, deep learning’s unique approach only became popular around 2012 when several researchers published papers on the topic.
In the years since, the technique has been applied to a lot of different problems, and it has proven remarkably adept at training computers to do a variety of things that have always been easy for humans but fairly difficult to teach to machines. Many AI researchers believe that deep learning will continue to be very influential in the coming years, but others, including Marcus, think that deep learning is of limited usefulness and will need to be supplemented by a lot of other techniques as AI research progresses.
3. Deep learning is “deep” because it has a lot of layers.
The name “deep learning” refers to the way in which these algorithms work, which is to process data through many layers. They take an input, create an output, and then use that output as the input for the next layer of learning. It starts by making very small abstractions or generalizations and then moving up to broader generalizations.
For example, imagine that you wanted to use deep learning techniques to teach a computer to recognize pictures of housecats. In the early stages, the system might try to classify all animals as cats. Then, after analyzing many images, it might begin classifying all small, fuzzy creatures as cats. Over time — and with lots of different layers of learning — the system might eventually become as good as people at recognizing pictures of cats.
4. Deep learning makes computer vision and other popular forms of AI possible.
The example on the previous slide is particularly appropriate because computer vision is one application where deep learning really excels. In fact, when the O’Reilly survey asked people which applications of deep learning most interested them, computer vision was the number one response. Other popular uses for deep learning include natural speech processing, game playing, translation, recommendation engines, text mining, forecasting, and analytics.
The thing that these use cases have in common is that it’s hard for people to write logical rules that will tell the computer what to look for. In the cat image example, someone might be able to tell the system that cats have four legs (usually) or that they have fur (usually), but that kind of logic can’t give the system all the information it needs to do the kind of intuitive analysis that people do all the time when identifying cats. The same problem occurs in natural language processing. Teaching the system the rules of grammar can only take you so far because there are so many exceptions to every rule.
Deep learning overcomes those limitations by having the computer figure out for itself which features in the training data are important for the task at hand, whether that task is identifying images, interpreting audio, or figuring out which movie you might like to watch next.
5. Deep learning can be supervised or unsupervised.
Within the field of machine learning, people frequently talk about supervised and unsupervised learning. It’s tempting to assume that supervised means that people are involved, and unsupervised means the computer learns on its own.
But that’s not really what supervised and unsupervised means. In any machine learning task, you’ll need to train your model by feeding it a lot of data. In the image analysis example, you would need to give your system a whole bunch of pictures, including some pictures of cats. In supervised deep learning, all the pictures would be labelled, either as “cat” and “non-cat” or as “cat,” “dog,” “tree,” “cars,” etc. Over time, it would learn to tell cats apart from the other images.
In unsupervised deep learning, you would show the computer a whole lot of images without labels. The machine wouldn’t be able to figure out the label “cat” on its own, but it would be able to group together all the similar images. This unsupervised learning can be very helpful in cases where all your test data isn’t labeled (as in the cat image example) or when you don’t know what you are looking for (as in many data mining applications).
6. Deep learning usually runs on an artificial neural network.
Another term that pops up frequently in discussions of deep learning is “artificial neural network,” or ANN. As the name suggests, these are computer systems that are roughly patterned after the human brain. The brain has lots of interconnected neurons that transmit signals to one another, and artificial neural networks have lots of interconnected nodes, also called neurons, that transmit data to one another. In the case of deep learning, those nodes are arranged in layers, with neurons passing information from one layer to another.
You might also hear talk of two specialized kinds of neural networks: recurrent neural networks (RNNs) and generative adversarial networks (GANs).
RNNs rely on memory to provide context for the current input that they are processing. So, for example, if you were teaching a computer to read cursive handwriting, an RNN would be better than other types of neural networks because it would take into account the previous letters when determining what label to give to a current letter.
A GAN involves two different neural networks, where one comes up with potential answers to a given question and the other evaluates those solutions. Over time, both neural networks in the GAN become better at generating and evaluating responses.
7. Deep learning requires massive amounts of resources.
Part of the reason why deep learning only became popular recently is that it requires huge amounts of computing power. Until the last few years, systems powerful enough to do the multiple simultaneous calculations involved either didn’t exist or were prohibitively expensive. Now, however, systems with highly advanced graphics processing units (GPUs), which are very good processing deep learning workloads, are much more affordable. And cloud computing vendors are making those systems available on a pay-as-you-go basis, which puts them within reach for almost any organization.
Deep learning also requires huge volumes of data for training, and enterprises’ ever-increasing stores of big data are perfect for this purpose.
8. Deep learning is already better than humans at some tasks.
Deep learning has already proven to be better than humans at playing Go, as well as playing some video games. A system called LipNet is better at reading lips than the average person, and deep learning is also better than humans at altering website design to improve e-commerce conversion rates.
In addition, many analytics systems are far better than humans at making predictions and finding correlations in seemingly disparate data. For example, one group of researchers trained a deep learning system on Google Street View and voter data to determine that “if the number of sedans encountered during a 15-minute drive through a city is higher than the number of pickup trucks, the city is likely to vote for a Democrat during the next presidential election (88% chance); otherwise, it is likely to vote Republican (82%).” That paper, published in 2017, used 2008 election results.
That doesn’t mean that deep learning systems are good at everything, however. For example, deep learning isn’t very good at drawing open-ended inferences or at distinguishing causation from correlation. For at least the foreseeable future, these types of reasoning will require human intervention or other types of artificial intelligence.
9. Google’s TensorFlow is the most popular deep learning framework.
When the O’Reilly survey asked people about which deep learning tools they were using, Google’s TensorFlow was the clear favorite, used by 61% of respondents. TensorFlow has also been popular with cloud computing providers, and Amazon and Google both offer TensorFlow-based services. The survey also found that 70% of respondents considered cloud services to be important to their deep learning applications.
The next most popular deep learning framework was Keras, which was used by less than half as many people (25%) as TensorFlow. Other options included PyTorch (20%), Caffe (6%), MXNet (4%) and CNTK (3%).
10. Several organizations offer classes in deep learning.
So far, only 11% of the respondents in the O’Reilly survey said that their organizations had hired people specifically for deep learning projects. Part of the problem may be that the number of deep learning experts is far smaller than the number of available jobs.
Developers and other IT professionals interested in learning more about deep learning have plenty of training options available. Many universities now offer courses in deep learning, both online and in the classroom. There are also online classes available through websites like Coursera (which was co-founded by Andrew Ng, one of the leading experts in machine learning), and companies that offer in-person training.Cynthia Harvey is a freelance writer and editor based in the Detroit area. She has been covering the technology industry for more than fifteen years. View Full Bio