In this article an implementation of the LucasKanade optical flow algorithm is going to be described. This problem appeared as an assignment in a computer vision course from UCSD. The inputs will be sequences of images (subsequent frames from a video) and the algorithm will output an optical flow field (u, v) and trace the motion of the moving objects. The problem description is taken from the assignment itself.
def optical_flow(I1, I2, window_size, tau) # returns (u, v)
has rank 2, which is what the threshold is checking. A typical value for τ is 0.01.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

import numpy as np from scipy import signal def optical_flow(I1g, I2g, window_size, tau = 1e  2 ): kernel_x = np.array([[  1. , 1. ], [  1. , 1. ]]) kernel_y = np.array([[  1. ,  1. ], [ 1. , 1. ]]) kernel_t = np.array([[ 1. , 1. ], [ 1. , 1. ]]) #*.25 w = window_size / 2 # window_size is odd, all the pixels with offset in between [w, w] are inside the window I1g = I1g / 255. # normalize pixels I2g = I2g / 255. # normalize pixels # Implement Lucas Kanade # for each point, calculate I_x, I_y, I_t mode = 'same' fx = signal.convolve2d(I1g, kernel_x, boundary = 'symm' , mode = mode) fy = signal.convolve2d(I1g, kernel_y, boundary = 'symm' , mode = mode) ft = signal.convolve2d(I2g, kernel_t, boundary = 'symm' , mode = mode) + signal.convolve2d(I1g,  kernel_t, boundary = 'symm' , mode = mode) u = np.zeros(I1g.shape) v = np.zeros(I1g.shape) # within window window_size * window_size for i in range (w, I1g.shape[ 0 ]  w): for j in range (w, I1g.shape[ 1 ]  w): Ix = fx[i  w:i + w + 1 , j  w:j + w + 1 ].flatten() Iy = fy[i  w:i + w + 1 , j  w:j + w + 1 ].flatten() It = ft[i  w:i + w + 1 , j  w:j + w + 1 ].flatten() #b = ... # get b here #A = ... # get A here # if threshold τ is larger than the smallest eigenvalue of A'A: nu = ... # get velocity here u[i,j] = nu[ 0 ] v[i,j] = nu[ 1 ] return (u,v) 
Input Sequences
Output Optical Flow with different window sizes
window size = 15
window size = 21
Input Sequences
Output Optical Flow
Input Sequences (hamburg taxi)
Output Optical Flow
Input Sequences
Output Optical Flow
Input Sequences
Output Optical Flow
Input Sequences
Output Optical Flow
Input Sequences
Output Optical Flow
Input Sequences
Output Optical Flow
Input Sequences
Output Optical Flow
Input Sequences
Output Optical Flow
Output Optical Flow
Input Sequences
Output Optical Flow with window size 45
Output Optical Flow with window size 10
Output Optical Flow with window size 25
Output Optical Flow with window size 45
© 2020 Data Science Central ® Powered by
Badges  Report an Issue  Privacy Policy  Terms of Service
Most Popular Content on DSC
To not miss this type of content in the future, subscribe to our newsletter.
Other popular resources
Archives: 20082014  20152016  20172019  Book 1  Book 2  More
Most popular articles
You need to be a member of Data Science Central to add comments!
Join Data Science Central