In this article, interactive image segmentation with graph-cut is going to be discussed. and it will be used to segment the source object from the background in an image. This segmentation technique was proposed by Boycov and Jolli in this paper.
Let’s implement “intelligent paint” interactive segmentation tool using graph cuts algorithm on a weighted image grid. Our task will be to separate the foreground object from the background in an image.
Since it can be difficult sometimes to automatically define what’s foreground and what’s background for an image, the user is going to help us with a few interactive scribble lines using which our algorithm is going to identify the foreground and the background, after that it will be the algorithms job to obtain a complete segmentation of the foreground from the background image.
The following figures show how an input image gets scribbling from a user with two different colors (which is also going to be input to our algorithm) and the ideal segmented image output.
Scribbled Input Image Expected Segmented Output Image
The following describes how the segmentation problem is transformed into a graph-cut problem:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
|
import numpy as np from collections import defaultdict def compute_pdfs(imfile, imfile_scrib): ''' # Compute foreground and background pdfs # input image and the image with user scribbles ''' rgb = mpimg.imread(imfile)[:,:,: 3 ] yuv = rgb2yiq(rgb) rgb_s = mpimg.imread(imfile_scrib)[:,:,: 3 ] yuv_s = rgb2yiq(rgb_s) # find the scribble pixels scribbles = find_marked_locations(rgb, rgb_s) imageo = np.zeros(yuv.shape) # separately store background and foreground scribble pixels in the dictionary comps comps = defaultdict( lambda :np.array([]).reshape( 0 , 3 )) for (i, j) in scribbles: imageo[i,j,:] = rgbs[i,j,:] # scribble color as key of comps comps[ tuple (imageo[i,j,:])] = np.vstack([comps[ tuple (imageo[i,j,:])], yuv[i,j,:]]) mu, Sigma = {}, {} # compute MLE parameters for Gaussians for c in comps: mu[c] = np.mean(comps[c], axis = 0 ) Sigma[c] = np.cov(comps[c].T) return (mu, Sigma) |
3. In order to compute the non-terminal edge weights, we need to compute the similarities in between a pixel node and the nodes corresponding to its neighborhood pixels, e.g., with the formula shown in the next figures (e.g., how similar neighborhood pixels are in RGB / YIQ space).
The following figures / animations show the interactive-segmentation results (computed probability densities, subset of the flow-graph & min-cut, final segmented image) on a few images, some of them taken from the above-mentioned courses / videos, some of them taken from Berkeley Vision dataset.
Input Image
Input Image with Scribbles
Fitted Densities from Color Scribbles
A Tiny Sub-graph with Min-Cut
Input Image
Input Image with Scribbles
Fitted Densities from Color Scribbles
A Tiny Sub-graph with Min-Cut
Input Image (liver)
Input Image with Scribbles
Input Image
Input Image with Scribbles
Input Imagehttps://sandipanweb.files.wordpress.com/2018/02/bunny.png?w=150&... 150w, https://sandipanweb.files.wordpress.com/2018/02/bunny.png?w=300&... 300w" sizes="(max-width: 681px) 100vw, 681px" />
Input Image with Scribbles
Fitted Densities from Color Scribbles
A Tiny Sub-graph of the flow-graph with Min-Cut
Input Image
Input Image with Scribbles
A Tiny Sub-graph of the flow-graph with Min-Cut
Input Image Input Image with Scribbles
Input Image
Input Image with Scribbles
Fitted Densities from Color Scribbles
Input Image
Input Image with Scribbles
Fitted Densities from Color Scribbles
A Tiny Sub-graph of the flow-graph with Min-Cut
Input Image (UMBC Campus Map)
Input Image with Scribbles
Input Image
Input Image with Scribbles
A Tiny Sub-graph of the flow-graph with Min-Cut (with blue foreground nodes)
The following figures / animation show how the background of a given image can be replaced by a new image using cut & paste (by replacing the corresponding pixels in the new image corresponding to foreground), once the foreground in the original image gets identified after segmentation.
Original Input Image
New Background
Posted 1 March 2021
© 2021 TechTarget, Inc.
Powered by
Badges | Report an Issue | Privacy Policy | Terms of Service
Most Popular Content on DSC
To not miss this type of content in the future, subscribe to our newsletter.
Other popular resources
Archives: 2008-2014 | 2015-2016 | 2017-2019 | Book 1 | Book 2 | More
Most popular articles
You need to be a member of Data Science Central to add comments!
Join Data Science Central