What is AlphaGo, AlphaGo Zero, and AlphaZero

AlphaGo – developed by the DeepMind team of Google – is an AI program which plays the board game Go.

A Go board (By Donarreiskoffer - Self-photographed, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=43383)

The Go board game is an abstract strategy game, which has been invented in China over 2500 years ago. Despite its simple set of rules, Go is considered to be much more complex than Chess, and is one of the most studied strategy game of all time.

The AlphaGo Logo

The AlphaGo uses a Monte Carlo tree search algorithm to find moves using the trained deep neural network which works as its knowledge core. AlphaGo was initially trained on a training set of over 30 million moves data from human Go matches. It was then further trained by letting it compete against copies of itself using reinforcement learning.

TensorFlow Lite Developer Preview Announced

TensorFlow yesterday (14th Nov) announced the developer preview of TensorFlow Lite, a lightweight solution of TensorFlow for mobile and embedded devices, targeted for low-latency inference of on-device machine learning models.

TensorFlow Lite Logo

TensorFlow Lite is an evolution of TensorFlow Mobile, and designed to be lightweight, cross-platform (Android and iOS for a start), and fast.

Through the Android Neural Networks API, TensorFlow Lite would be capable of utilizing purpose-built machine learning hardware in the devices as they become available.

A trained TensorFlow model can be converted to the TensorFlow Lite format (.tflite) using the provided converter, and deployed to the mobile app (Android or iOS), where the converted model gets executed using the TF Lite Interpreter.

TensorFlow Lite contains a C++ API with a Java API wrapper on Android.

It has out-of-the-box support for MobileNet, Inception V3, and Smart Reply Models.

Read more about TensorFlow Lite on the following links,

• Announcing TensorFlow Lite - Google Developers
• Introduction to TensorFlow Lite 

Related Links,

https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/lite

https://developer.android.com/ndk/guides/neuralnetworks/index.html

Build Deeper: The Path to Deep Learning

Learn the bleeding edge of AI in the most practical way: By getting hands-on with Python, TensorFlow, Keras, and OpenCV. Go a little deeper...

Get your copy now!

Codes of Interest becomes 1 Year Old!

It's been one year!

I began the Codes of Interest Blog a year ago with a simple thought - to share my enthusiasm of Machine Learning and Computer Vision with everyone.

It's been a fulfilling journey.

Now, after a year, Codes of Interest has become a thriving community of like-minded and engaging Machine Learning and Computer Vision enthusiasts. We've covered over 40 tutorials on Deep Learning on libraries like Keras, TensorFlow, and Theano, and playing around Computer Vision libraries such as OpenCV and Dlib.

The constant feedback and suggestions from you all has allowed me to launch the Codes of Interest Facebook Community.
And, I was able to release my first book on Deep Learning: Build Deeper: Deep Learning Beginners' Guide, which is now available on Amazon.

Build Deeper: Deep Learning Beginners' Guide

I'm getting more and more emails and comments daily. So much that I'm having trouble responding to all.

So, I'm taking this opportunity to thank all the readers and subscribers of Codes of Interest for your input and encouragement. I can assure you that more exciting content is coming.

Migrating a Model to Keras 2.0

Keras v2.0 has been released for a couple of months now - v2.0.0 released on 5th May, 2017, while the latest version is 2.0.8 at the time of this writing. It brought in a lot of new features and improvements, but also made some syntax changes. Trying to run a code with the old syntax may result in anything from a flood of deprecation warnings, to not being able to run the code at all. Since there are many code examples online which uses the older syntax - including some older posts in Codes of Interest - it's better to know how to get such older syntax model to work on the 2.0 API.

The complete list of changes in Keras v2.0 was extensive, but the following list would help you to narrow down majority of the changes.

The most prominent change is the changing of image_dim_ordering parameter to image_data_format, and its associated values from "tf", and "th" to "channels_last" and "channels_first". We talked about this change in detail in our earlier post "What is the image_data_format parameter in Keras, and why is it important".

Likewise, in all the places where "dim_ordering" argument/parameter was used, it has been changed to "data_format".

All of the Convolution* layers have now need renamed to Conv*.

E.g. Convolution2D is renamed to Conv2D

What is the image_data_format parameter in Keras, and why is it important

We've talked about the image_dim_ordering parameter in Keras and why is it important. But since from Keras v2 changed the name of the parameter, I thought of bringing this up again.

As you know, Keras  is a higher-level neural networks library for Python, which is capable of running on top of TensorFlow, CNTK (Microsoft Cognitive Toolkit), or Theano, (and with limited support for MXNet and Deeplearning4j), which Keras refers to as 'Backends'.

The 'image_data_format' parameter in the keras.json file

Which backend Keras should use is defined in the keras.json file, which is located at ~/.keras/keras.json in Linux and Mac OS, and at %USERPROFILE%\.keras\keras.json on Windows.

The default keras.json file (default set to TensorFlow) would look like this,

 {  
   "epsilon": 1e-07,  
   "floatx": "float32",  
   "image_data_format": "channels_last",  
   "backend": "tensorflow"  
 }  

The "backend" parameter should either be "tensorflow", "cntk", or "theano". When switching the backend, make sure to switch the "image_data_format" parameter too. For "tensorflow "or "cntk" backends, it should be “channels_last”. For “theano”, it should be “channels_first”.

Build Deeper: Deep Learning Beginners' Guide

I've been away from writing a post for about three weeks. That's because I've been preparing something exciting.

Today, I'm happy to announce the first book release from Codes of Interest - Build Deeper: Deep Learning Beginners' Guide.

Build Deeper: Deep Learning Beginners' Guide

Deep Learning has become a household name. It’s the bleeding edge in AI, and already achieving some phenomenal feats. Breakthroughs are happening daily, and the tech giants are not only pursuing it, they’re leading it.

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning. Learn what Deep Learning is, and how it came to be. See what it's capable of, and its milestones. And get hands-on with building your first Deep Learning model.

All you need to get started is a bit of enthusiasm, and some basic programming skills.

Build Deeper: Deep Learning Beginners' Guide is now available from Amazon.

Get your copy now!

Using Bottleneck Features for Multi-Class Classification in Keras and TensorFlow

Training an Image Classification model - even with Deep Learning - is not an easy task. In order to get sufficient accuracy, without overfitting requires a lot of training data. If you try to train a deep learning model from scratch, and hope build a classification system with similar level of capability of an ImageNet-level model, then you'll need a dataset of about a million training examples (plus, validation examples also). Needless to say, it's not easy to acquire, or build such a dataset practically.

So, is there any hope for us to build a good image classification system ourselves?

Yes, there is!

Luckily, Deep Learning supports an immensely useful feature called 'Transfer Learning'. Basically, you are able to take a pre-trained deep learning model - which is trained on a large-scale dataset such as ImageNet - and re-purpose it to handle an entirely different problem. The idea is that since the model has already learned certain features from a large dataset, it may be able to use those features as a base to learn the particular classification problem we present it with.

This task is further simplified since popular deep learning models such as VGG16 and their pre-trained ImageNet weights are readily available. The Keras framework even has them built-in in the keras.applications package.

An image classification system built with transfer learning

The basic technique to get transfer learning working is to get a pre-trained model (with the weights loaded) and remove final fully-connected layers from that model. We then use the remaining portion of the model as a feature extractor for our smaller dataset. These extracted features are called "Bottleneck Features" (i.e. the last activation maps before the fully-connected layers in the original model). We then train a small fully-connected network on those extracted bottleneck features in order to get the classes we need as outputs for our problem.

Need More Fonts on OpenCV?

OpenCV has a built-in simple function to add text on your images - the cv2.putText() function. With just one line of code, you can add text anywhere on the image. You just need to specify the position, colour, scale (font size), and which the font to use as the minimum parameters.

 cv2.putText(image,  
           text_to_show,  
           (20, 40),  
           fontFace=cv2.FONT_HERSHEY_SIMPLEX,  
           fontScale=1,  
           color=(255, 255, 255))  

OpenCV also gives you a choice from a handful of fonts - all variants of the "Hershey" font.

But, there may come a point where you want more fonts. Have you wished that you could use a specific True Type or Open Type font on OpenCV?

The good news is, it's possible.

True Type Fonts working on OpenCV

Snapchat like Image Overlays with Dlib, OpenCV, and Python

You're probably familiar with Snapchat, and it's filters feature where you can put some cool and funny image overlays on your face images. As computer vision enthusiasts, we typically look at applications like these, and try to understand how it's done, and whether we can build something similar.

It turns out, we can make our own application with Snapchat like image overlays using Python, OpenCV, and Dlib.

Snapchat like Image Overlays with Dlib, OpenCV, and Python

So, how do we build it?

1. We'll first load the Webcam feed using OpenCV.
2. We'll load an image (in our example, and image for the 'eye') to be used as the overlay.
3. Use Dlib's face detection to localize the faces, and then use facial landmarks to find where the eyes are.
4. Calculate the size and the position of the overlay for each eye.
5. Finally, place the overlay image over each eye, resized to the correct size.

Let's start.

Codes of Interest Facebook Community is now Live!

We are on Facebook!

The Codes of Interest Page is now live on Facebook. I created the page so that our community can come together to share ideas, discuss about questions, quickly address issues you face with your Deep Learning / Machine Learning and Computer Vision experiments, and talk about what you would like to see from the Codes of Interest site.

Join Now

... and start discussing.

Build Deeper: The Path to Deep Learning

Learn the bleeding edge of AI in the most practical way: By getting hands-on with Python, TensorFlow, Keras, and OpenCV. Go a little deeper...

Get your copy now!

Encog-Node: Simple Machine Learning on Node.js

Before I got into serious Machine Learning and Computer Vision coding (which I mostly use Python for), I did a lot of my development on Node.js. Few years back (around 2012), I was trying to add a simple neural network to one of my Node.js applications. I looked around, but couldn't find a satisfactory node module which was lightweight and flexible. Around that time, I came across the Encog Machine Learning framework, which was created by Jeff Heaton, and was one of the most popular Machine Learning libraries for Java at the time. I noticed that there was a Javascript version of the Encog library, which worked surprisingly well, and set myself on to porting that to Node.js.

I released the first version of Encog-Node in early 2012, and the latest version v0.3.0 is now available from NPM - https://www.npmjs.com/package/encog-node, and is recommended for anyone who wants to add lightweight, simple machine learning capabilities to their Node.js applications.

GitHub user Rui Cardoso contributed a lot for the latest release, with restructuring and cleaning up the codebase, and adding more examples.

You can install it by simply running,

 npm install encog-node  

in your node application.

Milestones of Deep Learning

Deep Learning has been around for about a decade now. We talked about how Deep Learning evolved through Artificial Intelligence, and Machine Learning (See "What is Deep Learning?"). Since its inception, Deep Learning has taken the world by storm due to its success. Here are some of the more significant achievements of Deep Learning throughout the years,

AlexNet - 2012

The AlexNet Architecture (Image from the research paper)

• Proved that Convolutional Neural Networks actually works. AlexNet - and its research paper "ImageNet Classification with Deep Convolutional Neural Networks" by Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton - is commonly considered as what brought Deep Learning in to the mainstream.
• Won 2012 ILSVRC (ImageNet Large-Scale Visual Recognition Challenge) with 15.4% error rate. (For reference, the 2nd best entry at ILSVRC had 26.2% error rate).
• 8 layers: 5 convolutional, 3 fully connected.
• Used ReLU for the non-linearity function rather than the conventional tanh function used until then.
• Introduced the use of Dropout Layers, and Data Augmentation to overcome overfitting.

Research Paper: ImageNet Classification with Deep Convolutional Neural Networks - Alex Krizhevsky, Ilya Sutskever, Geoffrey E. Hinton

Machine UI : An IDE for Machine Learning, currently in Alpha

Machine UI, or just "Machine" as it's commonly referred, is an IDE for Machine Learning, which is currently in its Alpha stage. It has been designed to work with TensorFlow, and aims at simplifying setting up machine Learning experiments so that you spend more time experimenting, and less time configuring.

The interface of Machine UI (Note: This is a screenshot from their announcement video)

As per their announcement video, the machine learning experiments are set up visually. The input data, convolutions, and the outputs are placed as nodes on a graph. You can think of it as a more interactive version of the Tensor Board which comes with TensorFlow.

Wink Detection using Dlib and OpenCV

A couple of weeks ago, I was going through a tutorial for eye blink detection by Adrian at PyImageSearch. It was an excellent tutorial, which explained the use of Eye Aspect Ratio (EAR) in order to detect when an eye gets closed. Then, few weeks back, I was having a chat with Shirish Ranade, a reader of this blog and a fellow computer vision and machine learning enthusiast, on whether we can perform an action by winking at the computer. So, I decided to try out a code to detect winking.

Wink Detection Running with Dlib and OpenCV

It's an interesting idea to perform an action or a task just by winking at your computer. It can be thought as a form of gesture detection or facial expression detection as well. So, here's how you can build your own 'wink' detector for it.

We start by importing all the necessary packages,

 import numpy as np  
 import cv2  
 import dlib  
 from scipy.spatial import distance as dist  

OpenCV 3.2 and Dlib 19.4 Packages Now Available from Conda-Forge

Anaconda is an asset for us Machine Learning enthusiasts. Not only does it have the ability to create fully isolated Python environments, its pre-built packages for many operating systems and architectures helps you to spend less time setting up, and more time doing actual machine learning stuff.

In some scenarios, to get some Python packages to work in certain environments, getting them from Anaconda was the only way that worked. If you tried installing Dlib or OpenCV 3 on Windows 64-Bit, then you know the effort it takes to get them setup, if it wasn't for Anaconda.

OpenCV and Dlib working perfectly together, thanks to Conda

If you check my posts on Installing Dlib on Anaconda Python on Windows, and Installing OpenCV 3 on Anaconda Python 3.5 on Windows, you know how easy it is to use conda to install them on Python 3.5 64-Bit on Windows.

But there was a catch: The Anaconda registry only had OpenCV 3.1 and Dlib 19.0 - not the latest versions.

What is Deep Learning? - Updated

What is Deep Learning? And, how does it relates to Machine Learning, and Artificial Intelligence?

I did an article to answer these questions some time back.

Now, thanks to the feedback I got from you all, I was able to updated it, with more clarifications, improved examples, and answers to more questions in Deep Learning.

Check out the updated article here,

What is Deep Learning?

Your feedback are always welcome.

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning.

Get your copy now!

image_data_format vs. image_dim_ordering in Keras v2

If you have been using Keras for some time, then you would probably know the image_dim_ordering parameter of Keras. Specially, if you switch between TensorFlow and Theano backends frequently when using Keras.

When I first started using Keras for image classification, most of my experiments failed because I have set the image_dim_ordering incorrectly. Learning from my mistakes, last year I did a post on what image_dim_ordering is and why is it important.

The keras.json file houses the configuration options for Keras

In short, image_dim_ordering instructed Keras to properly rearrange the image data structure when passing to the backend:
Both TensorFlow and Theano expects 4D tensors of image data as input. But, while TensorFlow expects its structure/shape to be (samples, rows, cols, channels), Theano expects it to be (samples, channels, rows, cols). So, setting the image_dim_ordering to 'tf' made Keras use the TensorFlow ordering, while setting it to 'th' made it Theano ordering.

At least, that's how it used to work.

But recently, if you have updated to the latest version of Keras, you might have run into issues with the dimension ordering, even if you're sure that you set the image_dim_ordering correctly.

You may have gotten errors like,

 ValueError: The shape of the input to "Flatten" is not fully defined (got (0, 7,  
  50). Make sure to pass a complete "input_shape" or "batch_input_shape" argument  
  to the first layer in your model.  

It may seem to you that Keras has started to ignore your image_dim_ordering setting.

And you're right.

Visualizing Keras Models - Updated

About 2 months back, I did a post on how you can visualize the structure of a Keras model. As I mentioned, when the machine learning (or deep learning) model you're building is complex, then it may be easier to understand it if you can see a visual representation of it.

I showed you how to use the Visualization utility in Keras in order to draw the structure of a model in Keras, such as this visualization of the LeNet model,

Visualizing the LeNet model

But a few days back, several people had got some errors when following the steps I explained. I digged a bit to find why the errors are happening, and found that with the latest version of Keras (v 2.*) they have changed the API of the visualization utility.

The following are the main changes,

• The module has been renamed, from visualize_util to vis_utils.
• The function name for plotting has been renamed, from plot to plot_model.

So, here's the updated guide on how to visualize a Keras model.

Energy Threshold Calibration in Speech Recognition

In my last post on Speech Recognition, I showed how to setup the Python SpeechRecognition package with PyAudio, and pocketsphinx to recognize speech with just a few lines of code. And, as you can remember, we ran into issues where the speech recognition just hangs there unable to recognize our speaking.

Speech Recognition just hanging there, not recognizing that you're speaking

We found out that this was happening due to ambient noise.

Although we humans are able to distinguish speech from noise naturally, for a computer program they are just audio levels. It needs to know which levels should be considered speech (which it needs to process in order to recognize what's being said), and which levels should be considered silence or background noise. So, libraries like the SpeechRecognition has an energy threshold set which defines what audio level and above should be considered speech.

Now, this default energy threshold works most of the time. If your environment is sufficiently quiet, it will be able to recognize you talking without problems. But, if your environment is noisy - e.g. an office environment with many people talking, or there's machinery around - then the program will have issues distinguishing speech from noise, which will cause the issue we observed.

So, in a situation like that, we should adjust the energy threshold to properly distinguish the speech from noise. The SpeechRecognition package has a couple of parameters that helps you with this.

How deep should it be to be called Deep Learning?

If you remember, some time back, I made an article on What is Deep Learning?, in which I explored the confusion that many have on terms Artificial Intelligence, Machine Learning, and Deep Learning. We talked about how those terms relate to each other: how the drive to build an intelligent machine started the field of Artificial Intelligence, when building an intelligence from scratch proved too ambitious, how the field evolved into Machine Learning, and with the expansion of both the capabilities of computer hardware and our understanding of the natural brain, dawned the field of Deep Learning

We learned that the deeper and more complex models (compared to traditional models) of Deep Learning are able to consume massive amounts of data, and able to learn complex features by Hierarchical Feature Learning through multiple layers of abstraction. We saw that Deep Learning algorithms don’t have a "plateau in performance" compared to traditional machine learning algorithms: that they don’t have a limit on the amount of data they can ingest. Simply, the more data they are given, the better they would perform.

The Plateau in Performance in Traditional vs. Deep Learning

With the capabilities of Deep Learning grasped, there’s one question that usually comes up when one first learns about Deep Learning:

If we say that deeper and more complex models gives Deep Learning models the capabilities to surpass even human capabilities, then how deep a machine learning model should be to be considered a Deep Learning model?

I’ve had the same question when I was first getting started with Deep Learning, and I had few other Deep Learning enthusiasts asking me the same question.

It turns out, we were asking the wrong question. We need to look at Deep Learning from a different angle to understand it.

Let’s take a step back and see how a Deep Learning model works.

Extracting individual Facial Features from Dlib Face Landmarks

If you remember, in my last post on Dlib, I showed how to get the Face Landmark Detection feature of Dlib working with OpenCV. We saw how to use the pre-trained 68 facial landmark model that comes with Dlib with the shape predictor functionality of Dlib, and then to convert the output of into a numpy array to use it in an OpenCV context. We were able to get all 68 feature points on to our face image.

Dlib detecting the 68 Face Landmarks

The 68 feature points which the Dlib model detects include the Jawline of the face, left and right eyes, left and right eyebrows, the nose, and the mouth. So, what if you only want to detect few of those features on a face? E.g. you may only want to detect the positions of the eyes and the nose. Is there a way to extract only few of the features from the Dlib shape predictor?

There is actually a very simple way to do that. Here’s how.

Easy Speech Recognition in Python with PyAudio and Pocketsphinx

If you remember, I was getting started with Audio Processing in Python (thinking of implementing an audio classification system) a couple of weeks back (see my earlier post). I got the PyAudio package setup and was having some success with it. As you know, one of the more interesting areas in audio processing in machine learning is Speech Recognition. So, although it wasn't my original intention of the project, I thought of trying out some speech recognition code as well.

I searched around to see what Python packages are available for the task and found the SpeechRecognition package.

Python Speech Recognition running with Sphinx

SpeechRecognition is a library for Speech Recognition (as the name suggests), which can work with many Speech Engines and APIs. The current version supports the following engines and APIs,

• CMU Sphinx
• Google Speech Recognition
• Google Cloud Speech API
• Wit.ai
• Microsoft Bing Voice Recognition
• Houndify API
• IBM Speech to Text

I decided to start with the Sphinx engine since it was the only one that worked offline. But keep in mind that Sphinx is not as accurate as something like Google Speech Recognition.

First, let's set up the SpeechRecognition package.

Setting up TensorFlow with CUDA on Windows

I did the post about How to setup TensorFlow on Windows about a month back. I only covered setting up the CPU version of TensorFlow there, and promised that I'll do the guide for the GPU version soon. But I haven't had the change to come round to write the guide until now.

I guess better late than never. So, here's how to setup the GPU version of TensorFlow on Windows.

So, what do you need to get TensorFlow working on GPU?
You need to setup the following prerequisites, in that order.

1. Microsoft Visual Studio 2015 (The free community edition of VS 2015 will work)
2. CUDA 8.0
3. cuDNN 5.1 for CUDA 8.0

Start by installing Visual Studio 2015. You can get the free community edition from here. When you install, make sure to select ‘Custom Installation’, and select ‘Visual C++’ in the programming language selection (by default, C++ is not selected). Once installed, check whether you have C++ capability by checking the ‘New Project’ options.

Visual Studio 2015 installed with C++

How to Graph Model Training History in Keras

When we are training a machine learning model in Keras, we usually keep track of how well the training is going (the accuracy and the loss of the model) using the values printed out in the console. Wouldn't it be great if we can visualize the training progress? Not only would it be easier to see how well the model trained, but it would also allow us to compare models.

Something like this?

Training accuracy and loss for 100 epochs

Well, you can actually do it quite easily, by using the History objects of Keras along with Matplotlib.

How to solve CNMEM_STATUS_OUT_OF_MEMORY error with Theano on CUDA

Have yo come across the CNMEM_STATUS_OUT_OF_MEMORY error when using Theano with CUDA, with Keras? You might have been trying to train a slightly larger model, and just when the training starts it throws this error and fails.

The CNMEM_STATUS_OUT_OF_MEMORY thrown in Theano with CUDA

The full error stack looks something like this,

Visualizing Model Structures in Keras

Update 3/May/2017: The steps mentioned in this post need to be slightly changed with the updates in Keras v2.*. Please check the updated guide here: Visualizing Keras Models - Updated.

Have you ever wanted to visualize the structure of a Keras model? When you have a complex model, sometimes it's easy to wrap your head around it if you can see a visual representation of it. What if there's a way to automatically build such a visual representation of a model?

Well, there is a way. Keras has a model visualization function, that can plot out the structure of a model. It would look something like this,

The visualization of the LeNet model

Above is the visualization of the LeNet model, which is defined in code as follows,

 # initialize the model  
 model = Sequential()  
   
 # first set of CONV => RELU => POOL  
 model.add(Convolution2D(20, 5, 5, border_mode="same",  
     input_shape=(height, width, depth)))  
 model.add(Activation("relu"))  
 model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))  
   
 # second set of CONV => RELU => POOL  
 model.add(Convolution2D(50, 5, 5, border_mode="same"))  
 model.add(Activation("relu"))  
 model.add(MaxPooling2D(pool_size=(2, 2), strides=(2, 2)))  
   
 # set of FC => RELU layers  
 model.add(Flatten())  
 model.add(Dense(500))  
 model.add(Activation("relu"))  
   
 # softmax classifier  
 model.add(Dense(classes))  
 model.add(Activation("softmax"))  

TensorFlow 1.0 Released!

TensorFlow 1.0 has been released!

Just a week back I posted about the announcement of TensorFlow 1.0 and the new features coming. Now, it's finally here. It was released at the first TensorFlow Dev Summit held yesterday (15th Feb) at Mountain View, California.

The official TensorFlow website is now updated for the 1.0 release, which also now has a section for TensorFlow Fold which allows to work with input data of different shapes and sizes, and which has now been made open-source. The installation instructions have also been updated for the latest version, and they have added a guide on how to upgrade your application to TensorFlow 1.0.

The release notes for 1.0 suggests that all the anticipated features are here: The XLA compiler, Debugger, The Java API, Mobile support improvements, and much more. There's also news reports coming in which states that more machine learning models, such as SVMs, will come to TensorFlow. Better support for Keras is also hinted. I'm eagerly waiting for Keras to upgrade to support new features in TensorFlow 1.0.

Related posts:

• http://www.codesofinterest.com/2017/02/tensorflow-1-mobile-and-java-support.html

Related links:

• https://www.tensorflow.org/
• https://research.googleblog.com/2017/02/announcing-tensorflow-fold-deep.html
• http://venturebeat.com/2017/02/15/google-releases-tensorflow-1-0-with-new-machine-learning-tools/
• https://github.com/tensorflow/tensorflow/releases/tag/v1.0.0

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning.

Get your copy now!

How to solve Scikit-learn Deprecation Warning on cross_validation

When using the Scikit-learn library and trying out various examples found over the web, have you come across a DeprecationWarning for the cross_validation module?

The DeprecationWarning on cross_validation

This most commonly happens when the code you're trying to run utilizes the train_test_split() function - a handy function used to quickly split the training and test datasets from a main dataset. The full warning message is something like this,

 C:\Users\Thimira\Anaconda3\envs\tensorflow12\lib\site-packages\sklearn\cross_val  
 idation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in  
  favor of the model_selection module into which all the refactored classes and f  
 unctions are moved. Also note that the interface of the new CV iterators are dif  
 ferent from that of this module. This module will be removed in 0.20.  
  "This module will be removed in 0.20.", DeprecationWarning)  

So, how to solve this?

TensorFlow 1.0 announced with better Mobile support and Java API

TensorFlow, with its r0.12 version gave the (much awaited) compatibility to Windows (check How to setup TensorFlow on Windows), and an Experimental API for the GO language. Now, they have announced the first major version - version r1.0 - of TensorFlow, with several exciting features.

While Python would still be the primary API - and would be the most complete API for TensorFlow - version r1.0 will introduce a new, experimental Java API. While it could be far from complete, and may take a few releases to stabilize, having the Java API may open TensorFlow to new possibilities. Also note that in the current release candidates of r1.0, you will need to compile TensorFlow from source in order to get the Java interface (instructions from TensorFlow GitHub page), and it's still only available for Linux and Mac OS.

Installing OpenCV 3 on Anaconda Python 3.5 on Windows

If you checked my last post, you know that TensorFlow now officially works on Windows. This was really exciting to me, since I was using a Linux machine for my TensorFlow experiments, I had to switch between the Linux and Windows for my other tasks. Now, I should be able to perform some of those experiments on Windows as well. (I would still switch to Linux for more processor intensive deep learning models). So, I decided to set everything up on Windows again.

I've been using OpenCV 3 on Anaconda Python 3.5 on Linux. (Check out my guide on Installing OpenCV from source on Anaconda Python on Ubuntu 16.10 if you are trying to install on Linux). But on Windows, I've previously only tried OpenCV on Python 2.7. Since the OpenCV Downloads page has pre-built binaries for OpenCV 3 (version 3.2 being latest), I thought the installation would be straightforward.

I downloaded and ran the OpenCV 3.2 installer (which just extracts OpenCV to a directory of your choosing), set the OPENCV_DIR environment variable, add the bin dir in OpenCV to the PATH, and finally copy the cv2.pyd file to the site-packages directory in my Anaconda environment. I fired up the Anaconda environment, and tried loading cv2 in Python. But I got the following error,

 (tensorflow) C:\>python  
 Python 3.5.2 |Continuum Analytics, Inc.| (default, Jul 5 2016, 11:41:13) [MSC v  
 .1900 64 bit (AMD64)] on win32  
 Type "help", "copyright", "credits" or "license" for more information.  
 >>> import cv2  
 Traceback (most recent call last):  
  File "<stdin>", line 1, in <module>  
 ImportError: DLL load failed: The specified module could not be found.  

Setting up TensorFlow on Windows

TensorFlow, since it has been released just over an year ago, has gained a huge popularity with its capabilities. But the platforms it worked on was limited - there were only pre-packaged Python packages for Linux initially, while it gained a Mac OS compatible version along the way. Windows users were left wondering when (or if) there will be a Windows version of TensorFlow. There has been few unofficial builds for Windows, and some have attempted to build from source for Windows with various degrees of success. But, there was nothing official.

Now, I'm happy to learn that TensorFlow now officially supports Windows. The release notes suggests that Windows compatibility was present from TensorFlow v0.12.0 RC0, and it has been tested on Windows 7, Windows 10, and Windows Server 2016. It only supports 64-Bit Python 3.5. Both the standard Python distribution (from Python.org) and Anaconda Python are supported.

This was great news for me, since I was switching between Linux and Windows for my machine learning experiments. So, I gave a try to setup TensorFlow on Windows.

I've been using Anaconda Python for most of my experiments, so I created an Anaconda environment - named 'tensorflow' - with all the basic packages I need.

 conda create --name tensorflow numpy scipy scikit-learn pillow h5py mingw libpython  

Then I activated the environment I just created,

 activate tensorflow  

Now for the big step, installing TensorFlow from pip.

New Page - What is Deep Learning?

Deep Learning is the latest trend in Machine Learning. We hear about new innovations in Deep Learning every day, to a point that it has started to become a household name. Big companies such as Google, Apple, Amazon, Microsoft, IBM, and many others seems to be pushing for Deep Learning.

But, do you know what Deep Learning really is?

Since it's one of the most frequent questions that comes up, I created a new page "What is Deep Learning?" which talks about what Deep Learning is, how it relates to Machine Learning and Artificial Intelligence, and their history. Have a look at it from the below link,

What is Deep Learning?

Let me know what you think of Deep Learning.

Build Deeper: Deep Learning Beginners' Guide is the ultimate guide for anyone taking their first step into Deep Learning.

Get your copy now!

Stepping into audio classification - Getting started with PyAudio

I've had an idea to attempt to train a deep learning model that can classify different audio noises. Having a system that can accurately identify different sounds would have implications in many fields, from medical diagnostics to echo locations systems, among others. I wouldn't expect building such a system would be an easy task. But, let's give it a try.

I first had the idea about 8 years back. But back then, libraries that enable audio processing were scarce, and what was available required a massive effort to get installed. Now it seems that things have improved a lot. I myself was searching online for audio processing libraries for Python, and I came across this excellent post - Realtime Audio Visualization in Python - by Scott Harden of University of Florida.

Realtime Audio Visualization with PyAudio

The tutorial uses the PyAudio Python package (PyAudio homepage), which is the Python bindings for PortAudio (PortAudio homepage) - a cross-platform audio I/O library - allowing PyAudio to give a consistent interface to process audio across platforms. So, I decided to give PyAudio a try. First of all, I needed to install PyAudio.

Installing PyAudio

At the time of this writing, the latest version of PyAudio is v0.2.9, and the PyAudio team has made the installation as simple as possible.

Installing OpenCV from source on Anaconda Python on Ubuntu 16.10

I recently switched to Linux for my Machine Learning experiments, and I did a post on How to install Keras and Anaconda Python on Ubuntu 16.10.

Now, I wanted to install OpenCV on Ubuntu also. Since OpenCV does not have a pre-built package for Linux, it meant I had to compile OpenCV from source.

OpenCV 3.1 running on Lubuntu 16.10

Adrian of PyImageSearch has recently done a post about how to compile OpenCV on Ubuntu 16.04 using virtualenv. I followed his steps as a base, but had to make numerous adjustments to some of the packages which gets installed (e.g. libpng-dev, libhdf5-serial-dev) and the build commands due to the changes from Ubuntu 16.04 to 16.10, and because I'm using Anaconda environments rather than virtualenv.

I'll be installing OpenCV 3.1, and will be using the Lubuntu 16.10 virtual machine which I used in my earlier post. But the steps and commands will be exactly the same for any flavor of Ubuntu 16.10.

First, as a habit, get and install the latest updates for Ubuntu,

 sudo apt-get update   
 sudo apt-get upgrade   

Then (if you have not done already) install the necessary build tools,

 sudo apt-get install build-essential cmake git unzip pkg-config  

Then, we install the following packages which allows OpenCV to interact with various image and video formats,

 sudo apt-get install libjpeg8-dev libtiff5-dev libjasper-dev libpng-dev  
 sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev  
 sudo apt-get install libxvidcore-dev libx264-dev  

Note: on Ubuntu 16.04, the package name for libpng was libpng12-dev. But on 16.10, it should be libpng-dev.