Published on: January, 2017
Topics:
- Development •
- Methodologies
Published on: January, 2017
Deep learning and its representation of the human brain based on applied math and statistics has drawn a new way to solve machine learning problems. When we talk about deep learning, we are implicitly referring to neural networks (NN) which are a representation of our neurons and how they work. Neural networks have been used to solve different problems ranging from natural language processing, computer vision, sentiment analysis, voice recognition, and autonomous vehicles.
There are some good reasons to implement deep learning:
- Deep neural networks spend a lot of time in training, but recent advances in technology have improved and as a result are able to reduce the amount of time they require in training, from one month to one week using GPU features.
- It makes pattern recognition more accurate when we have more than 10 features compared with traditional machine learning methods.
- Deep nets are useful because they split complex patterns in simple ones.
However, using deep learning can be complicated. Common problems that we can face when using deep learning include:
- The “black box” of artificial intelligence (AI). This refers to the difficulty in explaining how algorithms got to their results and why those results are correct to, for example 99% accuracy. Just as it is hard to explain how our brain works and comes to conclusions, so it is difficult to explain how neural networks come to their conclusions. Scientists have been working to improve this. The X-TREPAN research, by scientists Awudu Karim and Shangbo Zhou, explains that “Rule extraction attempts to translate this numerically stored knowledge into a symbolic form that can be readily comprehended”. This is an approach to extract rules, so we can understand how neural networks arrive at their results.
- High computation power. Although we can achieve great results when working with deep learning methods, we need high computational power. This is due to the complexity and need for massive amounts of data.
- Limited data. We can face problems when we have limited data with which to train a neural network, and in turn it can affect the results. Good quality data is key for neural networks, but it’s not always easy to get hold of. Here we can use data augmentation to boost performance. Data augmentation can simply involved applying transformations to your data set, or adding noise, to make your final model more rigorous. There are also more advanced techniques for augmenting your data.
- Neural networks can be fooled. In 2014 Jeff Clune, Assistant Professor at Wyoming University, and his team discovered that it was possible for unrecognisable images to be labelled by a neural network as an identifiable object (for example some lines might be identified as a table). The team found it was relatively easy to create such images. Such “weaknesses” could be used by hackers to change the expected behavior of an artificial intelligence implementation.
How can we use deep learning?
There is a long list of possible platforms or libraries that you can use, but here I want to focus on some of the most common.
Platforms provide one option to get started
Machine learning platforms are useful because they help reduce the complexity needed to get started. The downside is the lack of flexibility because you need to adapt to their features and options, therefore making it difficult to fit with specific problems in your project.
Some examples of machine learning platforms include:
- Turi GraphLab. This platform uses different algorithms, and depending on the input data it will switch between convolutional nets when using images and multilayer perceptron with text data. It also has an API that can implement algorithms as UDFs in Python. Turi was acquired by Apple in 2016.
- H2O. H20 claims to be the world’s leading open source machine learning platform. It makes things easy to use with an understandable user interface. Although it only has multi layer perceptron for deep learning, it has other machine learning models such as GLM, Distributed Random Forest, K-Means Clustering, Gradient Boosting Machine, and Naive Bayes Classifier, which can be applied to business problems.
- BisQue. This open source platform was originally designed for scientific researchers. It can help us when conducting research that involves anything related to visual information, image data, and metadata. It is useful if you are working in a team, and want to share results.
- Google Cloud Machine Learning Platform. Google’s platform was launched in early 2016, and provides a range of pre-trained models. However Google won’t be launching GPU-centric machines until 2017.
Machine learning software libraries will help you avoid complexity
Just as developers use software libraries in their usual work, with deep learning it is no different. A library is simply a set of functions that are pre-made by a development team. The purpose is that they can be easily used, avoid complexity, and make it reusable and extensible. It is common that a library is maintained by the community, and that can help you learn and get involved. Some common libraries for deep learning include:
- Torch. Torch is a library made in the Lua programming language. It has a large community and a lot of extensions to add new functionalities. Torch allows you to parallelize CPUs and GPUs in an efficient manner and can be flexible using hyper parameters. This powerful library is compatible with CUDA implementation.
- DeepLearning4j. This is an open source project from the company Skymind. It is written in Java with support in vectorization, can work with CPUs and GPUs, parallel training via iterative reduce, and it is scalable on Hadoop.
- Caffe. Caffe is a library written in C++ with CUDA compatibility. It has multi layer and hyper parameter support, and can switch from CPUs to GPUs easily only setting a single flag. Their community has been using caffe in research projects, startups, and large scale industrial applications.
- Theano. Created by Yoshua Bengio, Theano is written in Python, and it allows you to deal with mathematical operations involving multi-dimensional arrays. Benefits include transparent use of a GPU, speed and stability, and a huge community.
Which Neural Net should I use?
There is no single answer because every case is different. This graphic explains the most common neural nets for different cases.
Use cases for Deep Neural Networks
Here we provide some real-life examples of where deep neural networks are already being used.
FinTech
Financial companies are implement new technologies to improve revenues or security, are using neural networks and other machine learning techniques to achieve their goals.
- Fraud detection. Abnormal transactions can be detected and accounts blocked to prevent fraudulent use.
- Customer service. Enterprises are using chatbots to answer questions from customers in a natural way using text or voice. You may not be able to easily recognize that you are talking with a machine.
- Marketing. Financial firms can better use the data they collect to create a great marketing strategy and ensure they are meeting customer desires, even in niche markets.
Autonomous-driving cars
Neural networks can be used to recognize objects and then process their output on a decision based on the environment. This is what is underlying the development of self-driving cars. Companies involved include:
- Nvidia. Nvidia is the company that provides automotive manufacturers an AI computing platform. They offer the Drive PX 2 which has HD mapping and highway automated driving. This is the technology behind Tesla’s self-driving cars, including their Model S and Model X.
- MobilEye. This company provides a vision-based driver assistance system which alerts the driver when a collision is a high possibility.
Memory-Augmented Networks
Recurrent networks can’t retain information for a long time, which was the motivation to implement a separate memory module to ensure that information is accessible and work as a “soft” RAM circuit in our neural net.
Physics
Deep learning is closely related to classical problems found in the world of physics. Indeed tools used by scientists and physicists underpin much of machine learning. We will be able to create and visualize new scenarios, such as creating different images of falling objects.
Adversarial Training
Prediction under uncertainty is one of the biggest challenges that scientists have been working on. Learning predictive forward models of the world, where machines are asked to predict the future like our brain does, continues to be a significant challenge. However there have been some limited success using adversarial training. Some examples include:
- DCGAN. Generates images that don’t exist by a neural network.
- EBGAN. Energy-based Generative Adversarial Networks (GANs).
- Video Prediction. Based on 4 to 8 frames, multi-scale convolutional neural networks (ConvNet) are able to predict 1 to 8 frames as outputs, but with a blur.