Solving Some Image Processing Problems with Python libraries - Part 1

In this article a few popular image processing problems along with their solutions are going to be discussed. Python image processing libraries are going to be used to solve these problems.

Image Transformations and Warping

0. Display RGB image color channels in 3D

A gray-scale image can be thought of a 2-D function f(x,y) of the pixel locations (x,y), that maps each pixel into its corresponding gray level (an integer in [0,255], e.g.,).
For an RGB image there are 3 such functions, f_R(x,y), f_G(x.y), f_B(x.y).
matplotlib’s 3-D plot functions can be used to plot each function.

The following python code shows how to plot the RGB channels separately in 3D:

def plot_3d(X, Y, Z, title, cmap):
    import matplotlib.pylab as plt
    from mpl_toolkits.mplot3d import Axes3D
    # implement this function to plot the channel pixel values in 3D
    plt.show()
 
im = imread('../new images/parrot.jpg')
Y = np.arange(im.shape[0])
X = np.arange(im.shape[1])
Z1 = im[...,0]
Z2 = im[...,1]
Z3 = im[...,2]
plot_3d(X, Y, Z1, cmap='Reds', title='3D plot for the Red Channel')
plot_3d(X, Y, Z2, cmap='Greens', title='3D plot for the Green Channel')
plot_3d(X, Y, Z3, cmap='Blues', title='3D plot for the Blue Channel')

The RGB image
https://sandipanweb.files.wordpress.com/2018/07/parrot.jpg?w=150&am... 150w, https://sandipanweb.files.wordpress.com/2018/07/parrot.jpg?w=300&am... 300w" sizes="(max-width: 570px) 100vw, 570px" />

1. Wave Transform

Use scikit-image’s warp() function to implement the wave transform.
Note that wave transform can be expressed with the following equations:

We shall use the madrill image to implement the wave transform. The next python code fragment shows how to do it:

def wave(xy):
    xy[:, 1] += 20*np.sin(2*np.pi*xy[:, 0]/64)
    return xy
 
from skimage.io import imread
from skimage.transform import warp
import matplotlib.pylab as plt
im = imread('images/mandrill.jpg')
im = warp(im, wave)
plt.imshow(im)
plt.show()

The next figure shows the original mandrill input image and the output image obtained after applying the wave transform.

https://sandipanweb.files.wordpress.com/2018/07/mandrill.jpg?w=150&... 150w" sizes="(max-width: 320px) 100vw, 320px" /> https://sandipanweb.files.wordpress.com/2018/07/mandrill_w1.png?w=1... 150w" sizes="(max-width: 317px) 100vw, 317px" />

2. Swirl Transform

Use scikit-image’s warp() function to implement the swirl transform.
Note that swirl transform can be expressed with the following equations

We shall use the madrill image to implement the wave transform. The next python code fragment shows how to do it:

def swirl(xy, x0, y0, R):
    r = np.sqrt((xy[:,1]-x0)**2 + (xy[:,0]-y0)**2)
    a = np.pi * r / R
    xy[:, 1] = (xy[:, 1]-x0)*np.cos(a) + (xy[:, 0]-y0)*np.sin(a) + x0
    xy[:, 0] = -(xy[:, 1]-x0)*np.sin(a) + (xy[:, 0]-y0)*np.cos(a) + y0
    return xy
 
im = imread('../images/mandrill.jpg')
im = warp(im, swirl, map_args={'x0':112, 'y0':112, 'R':512})
plt.imshow(im)
plt.axis('off')
plt.show()

The next figure shows the original mandrill input image and the output image obtained after applying the swirl transform.

https://sandipanweb.files.wordpress.com/2018/07/mandrill.jpg?w=150&... 150w" sizes="(max-width: 320px) 100vw, 320px" />

Compare this with the output of the scikit-image swirl() function.

3. Very simple Face morphing with α-blending

Start from one face image (e.g., let image₁ be the face of Messi) and end into another image (let image₂ be the face of Ronaldo) iteratively, creating some intermediate images in between.
At each iteration create an image by using a linear combination of the two image numpy ndarrays given by

3. Iteratively increase α from 0 to 1.

The following code block shows how to implement it using matplotlib’s image and pylab modules.

im1 = mpimg.imread("../images/messi.jpg") / 255 # scale RGB values in [0,1]
im2 = mpimg.imread("../images/ronaldo.jpg") / 255
i = 1
plt.figure(figsize=(18,15))
for alpha in np.linspace(0,1,20):
 plt.subplot(4,5,i)
 plt.imshow((1-alpha)*im1 + alpha*im2)
 plt.axis('off')
 i += 1
plt.subplots_adjust(wspace=0.05, hspace=0.05)
plt.show()

The next animation shows the simple face morphing:

There are more sophisticated techniques to improve the quality of morphing, but this is the simplest one.

4. Creating Instagram-like Gotham Filter

The Gotham filter

The Gotham filter is computed as follows (the steps taken from here), applying the following operations on an image, the corresponding python code, input and output images are shown along with the operations (with the following input image):

https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/city2.jpg?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" />

A mid-tone red contrast boost

from PIL import Image
import numpy as np
import matplotlib.pylab as plt
im = Image.open('../images/city.jpg') # pixel values in [0,255]
r, g, b = im.split()
red_levels = [0., 12.75, 25.5, 51., 76.5, 127.5, 178.5, 204., 229.5, 242.25, 255.]
r1 = Image.fromarray((np.reshape(np.interp(np.array(r).ravel(), np.linspace(0,255,len(red_levels)), red_levels), (im.height, im.width))).astype(np.uint8), mode='L')
plt.figure(figsize=(20,15))
plt.subplot(221)
plt.imshow(im)
plt.title('original', size=20)
plt.axis('off')
plt.subplot(222)
im1 = Image.merge('RGB', (r1, g, b))
plt.imshow(im1)
plt.axis('off')
plt.title('with red channel interpolation', size=20)
plt.subplot(223)
plt.hist(np.array(r).ravel(), normed=True)
plt.subplot(224)
plt.hist(np.array(r1).ravel(), normed=True)
plt.show()

https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png?w=1024 1024w, https://sandipanweb.files.wordpress.com/2018/07/gotham1.png 1172w" sizes="(max-width: 676px) 100vw, 676px" />

Make the blacks a little bluer

plt.figure(figsize=(20,10))
plt.subplot(121)
plt.imshow(im1)
plt.title('last image', size=20)
plt.axis('off')
b1 = Image.fromarray(np.clip(np.array(b) + 7.65, 0, 255).astype(np.uint8))
im1 = Image.merge('RGB', (r1, g, b1))
plt.subplot(122)
plt.imshow(im1)
plt.axis('off')
plt.title('with transformation', size=20)
plt.tight_layout()
plt.show()

https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham2.png?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" />

A small sharpening

from PIL.ImageEnhance import Sharpness
plt.figure(figsize=(20,10))
plt.subplot(121)
plt.imshow(im1)
plt.title('last image', size=20)
plt.axis('off')
im2 = Sharpness(im1).enhance(3.0)
plt.subplot(122)
plt.imshow(im2)
plt.axis('off')
plt.title('with transformation', size=20)
plt.tight_layout()
plt.show()

https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham3.png?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" />

A boost in blue channel for lower mid-tones

A decrease in blue channel for upper mid-tones

blue_levels = [0., 11.985, 30.09, 64.005, 81.09, 99.96, 107.1, 111.945, 121.125, 143.055, 147.9, 159.885, 171.105, 186.915, 215.985, 235.875, 255.]
b2 = Image.fromarray((np.reshape(np.interp(np.array(b1).ravel(), np.linspace(0,255,len(blue_levels)), blue_levels), (im.height, im.width))).astype(np.uint8), mode='L')
plt.figure(figsize=(20,15))
plt.subplot(221)
plt.imshow(im2)
plt.title('last image', size=20)
plt.axis('off')
plt.subplot(222)
im3 = Image.merge('RGB', (r1, g, b2))
plt.imshow(im3)
plt.axis('off')
plt.title('with blue channel interpolation', size=20)
plt.subplot(223)
plt.hist(np.array(b1).ravel(), normed=True)
plt.subplot(224)
plt.hist(np.array(b2).ravel(), normed=True)
plt.show()

https://sandipanweb.files.wordpress.com/2018/07/gotham41.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham41.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham41.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham41.png?w=1024 1024w, https://sandipanweb.files.wordpress.com/2018/07/gotham41.png 1172w" sizes="(max-width: 676px) 100vw, 676px" />

The output image obtained after applying the Gotham filter is shown below:

https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=1352 1352w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=150 150w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=300 300w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=768 768w, https://sandipanweb.files.wordpress.com/2018/07/gotham_out.png?w=1024 1024w" sizes="(max-width: 676px) 100vw, 676px" />

Down-sampling with anti-aliasing using Gaussian Filter

Start with a large gray-scale image and reduce the image size 16 times, by reducing both height and width by 4 times.
Select every 4th pixel in the x and the y direction from the original image to compute the values of the pixels in the smaller image.
Before down-sampling apply a Gaussian filter (to smooth the image) for anti-aliasing.
Compare the quality of the output image obtained by down-sampling without a Gaussian filter (with aliasing).

The next code block performs the above steps. Since the Gaussian blur is a low-pass filter, it removes the high frequencies from the original input image, hence it’s possible to achieve sampling rate above the Nyquist rate (by sampling theorem) to avoid aliasing.

from scipy.ndimage import gaussian_filter
im = rgb2gray(imread('images/umbc.png'))
print(im.shape)
plt.figure(figsize=(20,20))
plt.imshow(im)
plt.show()
plt.figure(figsize=(20,20))
im_blurred = gaussian_filter(im, sigma=2.5) #(5,5,1)
plt.imshow(im_blurred)
plt.show()
n = 4 # create and image 16 times smaller in size
w, h = im.shape[0] // n, im.shape[1] // n
im_small = np.zeros((w,h))
for i in range(w):
   for j in range(h):
      im_small[i,j] = im[n*i, n*j]
plt.figure(figsize=(20,20))
plt.imshow(im_small)
plt.show()
im_small = np.zeros((w,h))
for i in range(w):
   for j in range(h):
      im_small[i,j] = im_blurred[n*i, n*j]
plt.figure(figsize=(20,20))
plt.imshow(im_small)
plt.show()