How to generate Faces using VAE with Keras? (Video Tutorial)

How to Generate Faces Using VAE with Keras?

Variational Autoencoder(VAE) can do many amazing things if we increase the latent space dimensionality from 2D to multi-dimensional space for generating faces. In the previous tutorial, we have learned about building the VAE and trained with MNIST handwritten digits dataset and also done analysis with testing data. Go through it once. Welcome to aiRobott, I am Kishor Kumar Vajja. In this tutorial we will learn how to generate celebrity faces using VAE with Keras, and we will  Generate new faces from latent space,   Multi variate standard normal distribution latent space points , Latent space arithmetic functions and morphing between faces.

How to build Variational Autoencoder (VAE) using Keras?

How to improve Deep Neural Network using Keras?

How to build a simple Deep Neural Network using Keras?

It is very easy to build a simple Deep Neural Network using Keras, it requires three things to build it, they are: 

1. A Dataset for loading and scaling it.
2. Layers to build the Model.
3. Activation functions and Model class.

after building the Model, we need to compile it with an optimizer and loss function. Now the Model is ready for training with dataset. Next, we will test the model to evaluate it.

we will see in detail of all these steps.

1. Dataset - ( CIFAR-10 )

For this model we are using CIFAR-10 dataset,it is used for training the Model. Our Deep Neural Network has a input layer with hidden dense layers and output layer, with this Neural Network we can make predictions on a new dataset, this is a supervised learning method.

Loading the Dataset for scaling:

Actually images are numpy arrays, so it is required to import numpy package at the beginning. And import the keras dataset package to import CIFAR-10 dataset.

Now we will load CIFAR-10 dataset:

here, x_train and x_test are input datasets for training and testing, they are numpy arrays of shape [50000, 32, 32, 3] and [10000, 32, 32, 3] respectively. 

It’s worth noting the shape of the image data in x_train: [50000, 32, 32, 3]. The first dimension of this array references the index of the image in the dataset, the second and third relate to the size of the image, and the last is the channel (i.e., red, green, or blue, since these are RGB images). 

  

y_train and y_test are numpy arrays with shape [50000, 1] and [10000, 1] respectively, containing the integer labels in the range 0 to 9 for the class of each image.

now verify the values for : x_train

By default, image data consists of integers between 0 and 255 for each pixel channel. Similarly, x_test image data pixel values are also between 0 and 255. see below,

y_train and y_test are :

for classifying the output in 10 classes :

NUM_CLASSES is a variable for number of classes.

Neural Networks work best when each input is inside the range -1 to 1, so we need to divide by 255 to x_train and x_test pixel values.

Now check the values for x_train and x_test :

notice x_train and x_test pixel values are converted to floating point values in the range from -1 to 1, see the difference between previous values and current values.

We also need to change the integer labelling of the images to one-hot-encoded vectors of length 10. Using the following code , the new shape of y_train and y_test are therefore [50000, 10] and [10000, 10] respectively.

There are no columns or rows in this dataset; instead, this is a tensor with four dimensions. For example, if we want to know the green channel i.e, 1, and the value of the pixel in the (12, 13) position of an image index of 54, just type like this..

Like this CIFAR-10 dataset downloaded and scaled to build the model.

CIFAR-10 dataset

CIFAR-10 is an established computer-vision dataset used for object recognition. It is a subset of the 80 million tiny images dataset and consists of 60,000 32x32 color images containing one of 10 object classes, with 6000 images per class. There are 50,000 training images and 10,000 test images. It was collected by Alex Krizhevsky, Vinod Nair and Geoffrey Hinton.

The dataset is divided into five training batches and one test batch, each with 10,000 images. The test batch contains exactly 1000 randomly selected images from each class. The training batches contain the remaining images in random order, but some training batches may contain more images from one class than another. Between them, the training batches contain exactly 5000 images from each class.

Here are the classes in the dataset, as well as 10 random images from each: Airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck.

The classes are completely mutually exclusive. There is no overlap between automobiles and trucks. “Automobiles” includes sedans, SUVs things of that sort. “Truck” includes only big trucks Neither includes pickup trucks.

bisect module in Python3

The bisect module implements an algorithm for inserting elements into a list while maintaining the list in sorted order.

(1). Inserting in Sorted Order:

Here is a simple example, in which insort( ) is used to insert items into a list in sorted order.

It's output :

The first column of the output shows the new random number. The second column shows the position where the number will be inserted into the list. The remainder of each line is the current sorted list.

Like this, we can manipulate the given data, it might be faster to simply build the list and then sort it once. For long lists, significant time and memory savings can be achieved using this insertion sort algorithm [ i.e, insort( ) ], especially when the operation to compare two members of the list requires expensive computation.

(2). Handling Duplicates:

In the above example the result shown a repeated value, 77. The bisect module provides two ways to handle repeats. New values can be inserted either to the left of existing values, or to the right.
The insort( ) function is actually an alias for insort_right( ), which inserts an item after the existing value. The corresponding function insort_left( ) inserts an item before the existing value.

Let's see an example :

Here is the output :

When the same data is manipulated using bisect_left( ) and insort_left( ), the results are the same sorted list but the insert positions are different for the duplicate values.

=============================================================================

How to create a dataframe from a list of dictionaries?

The Lambda Function

The lambda function is a dynamic way of compacting functions inside the code.
For example, the function:
>>> def area(b, h):
...          return 0.5*b*h
...
>>> area(5, 4)
10.0

this function can be compacted by using lambda function, like this:
>>> area = lambda b, h: 0.5*b*h
>>> area(5, 4)
10.0

The zip( ) method

The zip function takes two or more sequences and creates a new sequence of tuples where each tuple contains one element from each list.

let's see an example :
>>> a = [1, 2, 3, 4, 5]
>>> b = ['a', 'b', 'c', 'd', 'e']

now we will use zip( ) function, like this
>>> zip(a, b)
<zip at 0x5fdb7d8>

now we will see the list of resultant list:

>>> list(zip(a, b))

[(1, 'a'), (2, 'b'), (3, 'c'), (4, 'd'), (5, 'e')]
Like this we can use the zip() method.

NDArray Basics using MXNet

NDArray module is a primary tool of MXNet, it is used for storing and transforming the data. It is just like NumPy's multi-dimensional array. It has some advantages like, NDArrays support asynchronous computation on CPU, GPU and distributed cloud architectures. NDArrays provide support for automatic differentiation. So these advantages make the NDArray indispensable for deep learning.

NDArrays are multi-dimensional arrays of numerical values. NDArrays with one axis corresponds to vectors, two axes to matrices, more than two axes it corresponds to tensors.

to use mxnet in python, you need to install in your PC by typing at command prompt as shown below:
C:\Users\xxxx> pip install mxnet

to get started, let's import mxnet and import ndarray from mxnet.

>>> import mxnet as mx
>>> from mxnet import nd

(1). we can create a simple 1-dimensional array using mxnet from a python list, like this :
>>> x = nd.array([1, 2, 3])
>>> print(x)
[1. 2. 3.]
<NDArray 3 @cpu(0)>

<NDArray 3 @cpu(0)> indicates that x is a one-dimensional array of length 3 and it resides in CPU main memory. The 0 in @cpu(0) has no special meaning and does not represent a specific core.

(2). we can create a 2-dimensional array using mxnet from a python list, like this :
>>> y = nd.array([[1, 2, 3, 4], [1, 2, 3, 4], [1, 2, 3, 4]])
>>> print(y)
[[1. 2. 3. 4.]
 [1. 2. 3. 4.]
 [1. 2. 3. 4.]]
<NDArray 3x4 @cpu(0)>

(3). we can create an empty 2D array ( also called matrix) with 3 rows and 3 columns like this :
>>> x = nd.empty((3, 3))
>>> print(x)
[[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
<NDArray 3x3 @cpu(0)>
if empty function is used, it grabs some memory and gives us back a matrix without setting the values of any of its entries. This means that the entries can have any form of values.

(4). if we want our matrices to be initialized with zeros, then we have to use .zeros function like this:
>>> x = nd.zeros((3, 3))
>>> print(x)
[[0. 0. 0.]
 [0. 0. 0.]
 [0. 0. 0.]]
<NDArray 3x3 @cpu(0)>

(5). similarly, ndarray has a function to create a matrix of all ones, use .ones function like this:
>>> x = nd.ones((3, 4))
>>> print(x)
[[1. 1. 1. 1.]
 [1. 1. 1. 1.]
 [1. 1. 1. 1.]]
<NDArray 3x4 @cpu(0)>


(6). we can fill with a value ( for Example 7)in a 2D array with 3 rows and 3 columns like this:
>>> x = nd.full((3, 3), 7)
>>> print(x)
[[7. 7. 7.]
 [7. 7. 7.]
 [7. 7. 7.]]
<NDArray 3x3 @cpu(0)>

(7). sometimes, we need to create an array of random values ( this is very common in neural networks ) to use the array as a parameter. For that we can use random_normal function with a zero mean and unit variance form standard normal distribution like this :
>>> y = nd.random_normal(0, 1, shape=(3,4))
>>> print(y)
[[ 1.1630785   0.4838046   0.29956347  0.15302546]
 [-1.1688148   1.558071   -0.5459446  -2.3556297 ]
 [ 0.54144025  2.6785064   1.2546344  -0.54877406]]
<NDArray 3x4 @cpu(0)>

(8). sometimes, you need to copy an array by its shape but not its contents, then use .zeros_like( ) function like this:
>>> z = nd.zeros_like(y)
>>> print(z)
[[0. 0. 0. 0.]
 [0. 0. 0. 0.]
 [0. 0. 0. 0.]]
<NDArray 3x4 @cpu(0)>

(9). you can access the dimensions of array using .shape attribute, like this:
>>> y.shape
(3, 4)

(10). you can access the size of array using .size attribute, like this:
>>> y.size
12

(11). you can query the data type using .dtype, like this:
>>> y.dtype
numpy.float32

float32 is the default data type.

(12). Operations and memory storage of your device can be revealed by using .context attribute, like this:
>>> y.context
cpu(0)

Linear Algebra - Basics for Deep Learning

Scalars :

using MXNet, we can work with scalars by creating NDArray with just one element.We will see some addition,multiplication,division and exponentiation in this session.

if you have not installed MXNet package, first install in your PC using the following command at cmd prompt, goto command prompt :
C:\Users\ABC> pip install mxnet

let us take two scalars, x and y.

C:\Users\ABC> python

>>> from mxnet import nd

>>> x = nd.array([3.0])
>>> y = nd.array([2.0])

>>> print('x + y = ', x+y)
x + y = [5.]

>>> print('x * y = ', x*y)
x * y = [6.]

>>> print('x / y = ', x/y)
x / y = [1.5]

>>> print('x ** y = ', nd.power(x,y))
x ** y = [9.]

Vectors :

a vector is a (array) list of numbers, for example [1.0, 3.0, 5.0, 2.0]. These numbers are called as scalars, each of the numbers in the vector consists of a single scalar value. We call these values the entries or components of the vector. 

in MXNet, we work with vectors via 1D NDArrays.

>>> x = nd.arange(4)

>>> print('x = ', x)

x = [0. 1. 2. 3.]

for Example if we want the 3rd element in a vector,use

>>> x[3]

[3.]

Length, Dimensionality and Shape :

The length of a Vector is called as its dimension. in NDArray we can access a vector's length using .shape attribute. like this
>>> x.shape
(4,)
The shape is a tuple that lists the dimensionality of the NDArray along each of its axes. Because a vector can only be indexed along one axis, its shape has just one element.

Note that a scalar would have 0 dimensions and a vector would have 1 dimension.
so you can think of 2D array as 2 axes and 3D array as 3 axes, and so on.

let's see some examples,

>>> a = 2
>>> x = nd.array([1,2,3])
>>> y = nd.array([10,20,30])
>>> print(a * x)
[2. 4. 6.]

>>> print(a * x + y)
[12. 24. 36.]

Matrices :

Matrices are 2D arrays, it can be denoted with capital letter like, A, B, C etc.

>>> A = nd.arange(20).reshape((5,4))
>>> print(A)
[[0. 1. 2. 3.]
 [4. 5. 6. 7.]
 [8. 9. 10. 11.]
 [12. 13. 14. 15.]
 [16. 17. 18. 19.]]

we can transpose the matrix through T.

>>> print(A.T)
[[0. 4. 8. 12. 16.]
[1. 5. 9. 13. 17.]
[2. 6. 10. 14. 18.]
[3. 7. 11. 15. 19.]]

Tensors :

Tensors give us a generic way of discussing arrays with an arbitrary number of axes.

for example, Vectors are first-order tensors, and matrices are second-order tensors.

Using tensors, images ( 3D data structures) its axes corresponding to height, width and three (RGB) color channels we can work with it.

>>> X = nd.arange(24).reshape((2, 3, 4))

>>> print('X.shape =', X.shape)

X.shape = (2, 3, 4)

>>> print('X =', X)

X =

[[[ 0.  1.  2.  3.]

  [ 4.  5.  6.  7.]

  [ 8.  9. 10. 11.]]

 [[12. 13. 14. 15.]

  [16. 17. 18. 19.]

  [20. 21. 22. 23.]]]

Basic properties of tensor arithmetic :

for all tensors, multiplication by a scalar produces a tensor of the same shape.

>>> a = 2
>>> x = nd.ones(3)
>>> y = nd.zeros(3)

>>> print(x.shape)
(3,)
>>> print(y.shape)
(3,)
>>> print((a * x).shape)
(3,)
>>> print((a * x + y).shape)
(3,)

Sums and means :

>>> print(x)

[1. 1. 1.]

>>> print(nd.sum(x))

[3.]

>>> print(A)

[[ 0. 1. 2. 3.]

[ 4. 5. 6. 7.]

[ 8. 9. 10. 11.]

[12. 13. 14. 15.]

[16. 17. 18. 19.]]

>>> print(nd.sum(A))

[190.]

Mean : it is an average. 

Mean = sum / total number of elements.

>>> print(nd.mean(A))

[9.5]

>>> print(nd.sum(A) / A.size)

[9.5]

Dot product :

>>> x = nd.arange(4)
>>> y = nd.ones(4)
>>> print(x, y, nd.dot(x, y))
[0. 1. 2. 3.]
[1. 1. 1. 1.]
[6.]

where, nd.dot(x, y) is equivalently to nd.sum(x * y) this gives same result.

Dot products are useful in a wide range of contexts. For Example, given a set of weights, the weighted sum of some values could be expressed as the dot product. 
when the weights are non-negative and sum to one, the dot product expresses a weighted average.
when two vectors each have length one. dot products can also capture the cosine of the angle between them.

Matrix-vector product :

>>> nd.dot(A, x)
[14. 38. 62. 86. 110.]
 Note that the column dimension of A must be the same as the dimension of x.

Matrix-matrix multiplication :

>>> B = nd.ones(shape=(4,3))
>>> nd.dot(A, B)
[[ 6. 6. 6.]
[22. 22. 22.]
[38. 38. 38.]
[54. 54. 54.]
[70. 70. 70.]]

Norms :

Norms are operators in linear algebra, they tell us how big a vector or matrix is. 

we represent norms with a notation ||.||, where the . is just a placeholder.

for example , a vector X is ||X|| and matrix A is ||A||.

l1 norm is simply the sum of the absolute values.
the Euclidean distance sqrt (x1**2+ x2**2+....) is l2-norm.

>>> nd.norm(x)
[3.7416573]

to calculate L1-norm we can simply perform the absolute value and then sum over the elements.

>>> nd.sum(nd.abs(x))
[6.]

Norms and objectives :

In machine learning we are often trying to solve optimization problems: like (a). Maximize the probability assigned to observed data. (b). Minimize the distance between predictions and the ground-truth observations. Assign vector representations to items ( like words, products, or news articles) such that the distance between similar items is minimized, and the distance between dissimilar items is maximized. oftentimes, these objectives, perhaps the most important component of a machine learning algorithm are expressed as norms.

Python's - The Complete argparse module Tutorial Part - 3

Image Thresholding

1. Simple Thresholding

This is a technique for images processing, Using OpenCV we can get the desired results,

Let us understand the theory and practice about thresholding techniques.

in OpenCV we have to use cv2.threshold( ) function.

the usage is : cv2.threshold(img, 127, 255, cv2.THRESH_BINARY)

where, first argument : img is a source image, which should be a grayscale image.

2nd argument : 127 is the threshold value which is used to classify the pixel values. this is a global value also.

3rd argument : 255 is the maxVal which represents the value to be given if pixel value is more than (sometimes less than) the threshold value.

4th argument : different styles of thresholding.

as in 2nd argument, threshold value, i.e, 127, ( it is a global value) If pixel value is greater than a threshold value, it assigned one value ( may be white), else it is assigned another value (may be black).

4th argument : thresholding techniques - Different types :

1). cv2.THRESH_BINARY

2). cv2.THRESH_BINARY_INV

3). cv2.THRESH_TRUNC

4). cv2.THRESH_TOZERO

5). cv2.THRESH_TOZERO_INV

we will see an example on these thresholding techniques. see the code below,

in this code , we have used an input image , 'gradient.png', it is given below(pls download) and save in your working directory.

see the code , in line number 9:
 1st argument, 'img', - it is a input image which we have read from my current directory using cv2.imread() function as grayscale image. 2nd argument: 127 is the global threshold value, it means the pixel values above this value will get the value of 3rd argument, i.e, 255 (here) that means the pixel values will get white pixels.
4th argument : cv2.THRESH_BINARY, it converts the pixel values in black and white as per the 2nd and 3rd arguments specifies. it's output is shown in "BINARY" name.
it returns two values , ret & thresh1, here we need thresh value which is the required image. in the next section we will see what is ret value?

in line number 10: cv2.THRESH_BINARY_INV , it is the opposite to the code in line number 9.

in the line number 11: cv2.THRESH_TRUNC, it truncates and separates black and white pixels.

in the line number 12: cv2.THRESH_TOZERO, it sets the pixel vales either black or white.

in the line number 13: cv2.THRESH_TOZERO_INV, it is opposite to the code in line number 12.
.
now run the code, then you will see the output (below shown) like this :

(2). Adaptive Thresholding:

in Simple thresholding, we have used a global value as threshold value, i.e, 127. But it may not be sufficient in all the conditions where image has different lighting conditions in different areas. In that case, we go for Adaptive Thresholding. 

Adaptive Method - it decides how thresholding value is calculated

In this method, the algorithm calculate the threshold for a small regions of the image. So we get the different thresholds for different regions of the same image and it gives us better results for images with varying illumination.

It has three 'special' input parameters and only one output argument..

Here is the two algorithms in adaptive method:

(a). cv2.ADAPTIVE_THRESH_MEAN_C : this algorithm finds a threshold value from the mean value of neighbourhood area.

(b). cv2.ADAPTIVE_THRESH_GAUSSIAN_C : this algorithm finds a threshold value from the weighted sum of neighbourhood values where weights are a gaussian window.

we will see the differences of these two algorithms using an example.

frequently used terms  are :

Block Size - It decides the size of neighbourhood area.

C - It is just a constant which is subtracted from the mean or weighted mean calculated.

let's see the code : 

the input image - dave.jpg, (pls download from here):

in this code, we have read an input image "dave.jpg" in grayscale mode.
in line number 6 : This image is blurred with cv2.medianBlur( ) function it takes median of all the pixels under kernel area and central element is replaced with this median value. This is highly effective against salt-and-pepper noise in the images. central element is always replaced by some pixel value in the image. It reduces the noise effectively. Its kernel size should be a positive odd integer only.

in line number 8 : cv2.threshold(....., cv2.THRESH_BINARY) , using this line, our median blurred image is (from line 6) thresholded binary (see in the output "Global Thresholding(v=127))

in line number 9: cv2.adaptiveThreshold( ) function uses cv2.ADAPTIVE_THRESH_MEAN_C algorithm,
see the code :
cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_THRESH_MEAN_C, cv2.THRESH_BINARY, 11, 2)
where (i). img is the median blurred image,
(ii). 255 is the maxValue, it is a non-zero value assigned to the pixels for which the condition satisfied only.
(iii). threshold type : cv2.THRESH_BINARY or cv2.THRESH_BINARY_INV
(iv). blockSize – 11: it is a size of a pixel neighborhood that is used to calculate a threshold value for the pixel: 3, 5, 7, and so on.
(v). C –2 : Constant subtracted from the mean or weighted mean. Normally, it is positive but may be zero or negative as well.

in line number 10: cv2.adaptiveThreshold( ) function uses cv2.ADAPTIVE_GAUSSIAN_C algorithm,

see the code :
cv2.adaptiveThreshold(img, 255, cv2.ADAPTIVE_GAUSSIAN_MEAN_C, cv2.THRESH_BINARY, 11, 2)


lets' run the code, here is the output :

(3). Otsu's Binarization :

In global thresholding, we used an arbitrary value for threshold value, it is purely a trail and error method.

But consider a bimodal image ( it is an image whose histogram has two peaks), for this image, we can approxmately take a value in the middle of those peaks as threshold value, that thing is what Otsu binarization does. it automatically calculates a threshold value from image histogram for a bimodal image. For the other modals ( except bimodal), binarization won't be accurate.

For this cv2.threshold( ) function is used, but we have to pass an extra flag, cv2.THRESH_OTSU. For threshold value simply pass zero. Then the algorithm finds the optimal threshold value and returns as the second output, i.e, retVal. If Otsu thresholding is not used, retVal is same as the threshold value you used.

in the following code, Input image is a noisy image. In first case,global thresholding value used 127, In second case Otsu's thresholding applied directly. In third case the image is filtered with a 5x5 gaussian kernal to remove the noise, then applied Otsu thresholding. See how noise filtering improves the result.

see the code :

the input image "noisy2.png" download from here.

now run the code : here is the output-

Python's - The Complete argparse module Tutorial Part - 2

What is foo , bar , baz really mean?

foo, bar , baz, foonley are metasyntactic variables.

metasyntactic variables are used in formal logic, and used in spoken languages

A metasyntactic variable is a specific word or set of words identified as a placeholder in computer science and specifically computer programming. These words are commonly found in source code and are intended to be modified or substituted to be applicable to the specific usage before compilation (translation to an executable).

The words foo and bar are good examples as they are used in over 330 Internet Engineering Task Force Requests for Comments, which are documents explaining foundational internet technologies like HTTP (websites), TCP/IP, and email protocols.

By mathematical analogy, a metasyntactic variable is a word that is a variable for other words, just as in algebra letters are used as variables for numbers.

Metasyntactic variables are used to name entities such as variables, functions, and commands whose exact identity is unimportant and serve only to demonstrate a concept, which is useful for teaching programming.

Due to English being the foundation-language, or lingua franca, of most computer programming languages these variables are commonly seen even in programs and examples of programs written for other spoken-language audiences.

The typical names may depend however on the subculture that has developed around a given programming language.

General usage

Metasyntactic variables used commonly across all programming languages include foobar, foo, bar, baz, qux, quux, quuz, corge, grault, garply, waldo, fred, plugh, xyzzy, thud, Wibble, wobble, wubble, and flob are also used in the UK.

A complete reference can be found in a MIT Press book titled The Hacker's Dictionary.

Usage

In C & C++ programming languages foo  and bar are used as function names and variables.

In Python programming language : Spam, ham, and eggs are the principal metasyntactic variables used in the Python programming language. This is a reference to the famous comedy sketch, "Spam", by Monty Python, the eponym of the language. 

Python's - The Complete argparse module Tutorial

aiRobott Promo

OpenCV - Handling Files and Images tutorial

OpenCV Drawing Functions from aiRobott