# The first step towards deep learning

Recently, as planning to do some project related to deep learning (specifically image classification, detection etc.) in the near future, I spent some time dig into this field.

Based on the previous understanding of linear algebra, calculus, statistics and a few knowledge about machine learning, I had a tiny bit of conceptual understanding of deep learning, though I was completely unable to transfer any of my knowledge into code.

This is what I wanted to change.

### Why Keras?

The framework I started with is Keras. Keras is an actual deep learning framework: a well-designed API that allows you use to build deep learning models by clipping together high-level building blocks. And since Keras runs on top of TensorFlow or Theano , there is no performance cost to using Keras compared to using the one of these lower-level frameworks.

If you are familiar with Numpy and Scikit-Learn, then a fair comparison would be to say that Theano and TensorFlow are closer to Numpy, while Keras is closer to Scikit-Learn.

However the comparison isn’t perfect, since Keras is more flexible than Scikit-Learn: it allows you to define your own machine learning models, rather than just use pre-defined models. As for Torch, Keras has a significantly larger community than Torch, we can benefit from the extensive Python ecosystem in our workflow.

### Build a neural network with Keras

import numpy as np
import matplotlib.pyplot as plt
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.utils import np_utils

Using TensorFlow backend.

nb_classes = 10

# the data, shuffled and split between tran and test sets
(X_train, y_train), (X_test, y_test) = mnist.load_data() #
print("X_train original shape", X_train.shape)
print("y_train original shape", y_train.shape)

X_train original shape (60000, 28, 28)
y_train original shape (60000,)


Take a look at some examples of the training data.

for i in range(9):
plt.subplot(3,3,i+1)
plt.imshow(X_train[i], cmap='gray', interpolation='none')
plt.title("Class {}".format(y_train[i]))
plt.show()


Format the data for training.

X_train = X_train.reshape(60000, 784)
X_test = X_test.reshape(10000, 784)
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255
print("Training matrix shape", X_train.shape)
print("Testing matrix shape", X_test.shape)

Training matrix shape (60000, 784)
Testing matrix shape (10000, 784)

Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)


Build the neural-network. Here we’ll do a simple 3 layer fully connected network.

model = Sequential()

model.add(Dropout(0.2))   # Dropout helps protect the model from memorizing or "overfitting" the training data



When compiing a model, Keras asks you to specify your loss function and your optimizer. The loss function we’ll use here is called categorical crossentropy, and is a loss function well-suited to comparing two probability distributions.

model.compile(loss='categorical_crossentropy',metrics=["accuracy"], optimizer='adam')


Train the model.

model.fit(X_train, Y_train,
batch_size=128, nb_epoch=4,
verbose=1,
validation_data=(X_test, Y_test))

Train on 60000 samples, validate on 10000 samples
Epoch 1/4
60000/60000 [==============================] - 15s - loss: 0.2492 - acc: 0.9254 - val_loss: 0.1199 - val_acc: 0.9615
Epoch 2/4
60000/60000 [==============================] - 15s - loss: 0.0991 - acc: 0.9691 - val_loss: 0.0807 - val_acc: 0.9736
Epoch 3/4
60000/60000 [==============================] - 15s - loss: 0.0711 - acc: 0.9773 - val_loss: 0.0922 - val_acc: 0.9712
Epoch 4/4
60000/60000 [==============================] - 15s - loss: 0.0541 - acc: 0.9826 - val_loss: 0.0670 - val_acc: 0.9787

<keras.callbacks.History at 0x119233b70>


Finally, evaluate its performance

score = model.evaluate(X_test, Y_test,verbose=0)
print('Test score:', score[0])
print('Test accuracy:', score[1])

Test score: 0.0669518932859
Test accuracy: 0.9787


Inspecting the output


predicted_classes = model.predict_classes(X_test)

# Check which items we got right / wrong
correct_indices = np.nonzero(predicted_classes == y_test)[0]
incorrect_indices = np.nonzero(predicted_classes != y_test)[0]

 9952/10000 [============================>.] - ETA: 0s

plt.figure()
for i, correct in enumerate(correct_indices[:9]):
plt.subplot(3,3,i+1)
plt.imshow(X_test[correct].reshape(28,28), cmap='gray', interpolation='none')
plt.title("Predicted {}, Class {}".format(predicted_classes[correct], y_test[correct]))

plt.figure()
for i, incorrect in enumerate(incorrect_indices[:9]):
plt.subplot(3,3,i+1)
plt.imshow(X_test[incorrect].reshape(28,28), cmap='gray', interpolation='none')
plt.title("Predicted {}, Class {}".format(predicted_classes[incorrect], y_test[incorrect]))
plt.show()


#### References:

CS231n

Keras Documentation