James Montantes's Posts - Data Science Central2021-06-12T19:45:49ZJames Montanteshttps://www.datasciencecentral.com/profile/JamesMontanteshttps://storage.ning.com/topology/rest/1.0/file/get/3839400361?profile=RESIZE_48X48&width=48&height=48&crop=1%3A1https://www.datasciencecentral.com/profiles/blog/feed?user=0j9ezn74j8tpj&xn_auth=noThe Most Important Fundamentals of PyTorch you Should Knowtag:www.datasciencecentral.com,2020-05-22:6448529:BlogPost:9537252020-05-22T20:00:00.000ZJames Montanteshttps://www.datasciencecentral.com/profile/JamesMontantes
<div class="blog-slider blog-slider-style3 5579"><div class="blog-slider-list slick-initialized"><div class="blog-slider-item"><div class="blog-slider-container"><div class="blog-slider-content"><div class="blog-slider-content-details"></div>
</div>
</div>
</div>
</div>
</div>
<div class="layout-default" id="wrapper"><div class="content-container sh-page-layout-full"><div class="container entry-content"><div id="content-wrapper"><div class="content-layout-standard-mini" id="content"></div>
</div>
</div>
</div>
</div>
<div class="blog-slider blog-slider-style3 5579"><div class="blog-slider-list slick-initialized"><div class="blog-slider-item"><div class="blog-slider-container"><div class="blog-slider-content"><div class="blog-slider-content-details"></div>
</div>
</div>
</div>
</div>
</div>
<div id="wrapper" class="layout-default"><div class="content-container sh-page-layout-full"><div class="container entry-content"><div id="content-wrapper"><div id="content" class="content-layout-standard-mini"><div class="blog-single blog-style-single blog-style-single-share blog-blockquote-style1 blog-style-post-slider"><div class="post-item-single-container"><div class="post-content"><h2><strong>Fundamentals of PyTorch – Introduction</strong></h2>
<p><span>Since it was introduced by the Facebook AI Research (FAIR) team, back in early 2017, PyTorch has become a highly popular and widely used Deep Learning (DL) framework. Since the humble beginning, it has caught the attention of serious AI researchers and practitioners around the world, both in industry and academia, and has matured significantly over the years.</span></p>
<p><span>Scores of DL enthusiasts and professionals started their journey with the Google TensorFlow (TF), but the learning curve with base TensorFlow has always been steep. On the other hand, PyTorch has approached DL programming in an intuitive fashion since the beginning, focusing on fundamental linear algebra and data flow operations in a manner that is easily understood and amenable to step-by-step learning.</span></p>
<p><span>Due to this modular approach, building and experimenting with complex DL architectures has been much easier with PyTorch than following the somewhat rigid framework of TF and TF-based tools. Moreover, PyTorch was built to integrate seamlessly with the numerical computing infrastructure of the Python ecosystem and Python being the lingua franca of data science and machine learning, it has ridden over that wave of increasing popularity.</span></p>
<p><span>PyTorch is a constantly developing DL framework with many exciting additions and features. In this article, we will go over some of the basic elements and show an example of building a simple Deep Neural Network (DNN) step-by-step.</span></p>
<h2><strong>Tensor Operations with PyTorch</strong></h2>
<p><span>Tensors are at the heart of any DL framework. PyTorch provides tremendous flexibility to a programmer about how to create, combine, and process tensors as they flow through a network (called computational graph) paired with a relatively high-level, object-oriented API.</span></p>
<h2><strong>What are Tensors?</strong></h2>
<p><span>Representing data (e.g. about the physical world or some business process) for Machine Learning (ML), in particular for DNN, is accomplished via a data/mathematical structure known as the </span><i><span>tensor</span></i><span>. A tensor is a container which can house data in </span><i><span>N</span></i><span> dimensions. A tensor is often used interchangeably with another more familiar mathematical object </span><i><span>matrix</span></i><span> (which is specifically a 2-dimensional tensor). In fact, tensors are generalizations of 2-dimensional matrices to </span><i><span>N</span></i><span>-dimensional space.</span></p>
<p><span>In simplistic terms, one can think of scalar-vectors-matrices- tensors as a flow.</span></p>
<ul>
<li><span>Scalar are 0-dimensional tensors.</span></li>
<li><span>Vectors are 1-dimensional tensors.</span></li>
<li><span>Matrices are 2-dimensional tensors</span></li>
<li><span>Tensors are generalized N-dimensional </span><i><span>tensors</span></i><span>. N can be anything from 3 to infinity…</span></li>
</ul>
<p> </p>
<p><span>Often, these dimensions are also called </span><i><span>ranks</span></i><span>.</span></p>
<p><img class="size-full wp-image-5580 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/pasted-image-0-1.png" alt="Tensors of various dimensions | PyTorch | Exxact Corp" width="891" height="441"/></p>
<p><span>Fig 1: Tensors of various dimensions (ranks) </span></p>
<p> </p>
<h2><strong>Why Are Tensors Important for ML and DL?</strong></h2>
<p><span>Think of a supervised ML problem. You are given a table of data with some labels (could be numerical entities or binary classification such as Yes/No answers). For ML algorithms to process it, the data must be fed as a mathematical object. A table is naturally equivalent to a 2-D matrix where an individual row (or instance), or individual column (or feature) can be treated as 1-D vector.</span></p>
<p><span>Similarly, a black-and-white image can be treated as a 2-D matrix containing numbers 0 or 1. This can be fed into a neural network for image classification or segmentation tasks. </span></p>
<p><span>A time-series or sequence data (e.g. ECG data from a monitoring machine or a stock market price tracking data stream) is another example of 2-D data where one dimension (time) is fixed.</span></p>
<p><span>These are examples of using 2-D tensors in classical ML (e.g. linear regression, support vector machines, decision trees, etc.) and DL algorithms.</span></p>
<p><span>Going beyond 2-D, a color or grayscale image can be treated as a 3-D tensor where each pixel is associated with a so-called ‘color-channel’ – a vector of 3 numbers representing intensities in the Red-Green-Blue (RGB) spectrum. This is an example of a 3-D tensor.</span></p>
<p><span>Similarly, videos can be thought of as sequences of color images (or frames) in time and can be thought of as 4-D tensors.</span></p>
<p><span>In short, all kinds of data from the physical word, sensors and instruments, business and finance, scientific or social experiments, can be easily represented by multi-dimensional tensors to make them amenable for processing by ML/DL algorithms inside a computing machine.</span></p>
<p><span>Let’s see how PyTorch defines and handles tensors.</span></p>
<h2><strong>Creating and Converting Tensors in PyTorch</strong></h2>
<p><span>Tensors can be defined from a Python list as follows,</span></p>
<p><img class="aligncenter size-full wp-image-5581" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-1.png" alt="Converting Tensors in PyTorch | PyTorch | Exxact Corp" width="415" height="487"/></p>
<p><span>Actual elements can be accessed and indexed as follows,</span></p>
<p><img class="aligncenter wp-image-5582 size-full" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-2.png" alt="Accessing Element and Data Type | PyTorch | Exxact Corp" width="666" height="501"/></p>
<p><span>Tensors with specific data types can be created easily (e.g. floating points),</span></p>
<p><img class="aligncenter size-full wp-image-5583" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-3.png" alt="Tensors with Specific Data Types | PyTorch | Exxact Corp" width="828" height="342"/></p>
<p><span>Size and dimensions can be read easily,</span></p>
<p><span><img class="aligncenter size-full wp-image-5584" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-4.png" alt="Size and Dimensions Reading | PyTorch | Exxact Corp" width="333" height="344"/></span></p>
<p><span>We can change the view of a tensor. Let us start with a 1-dimensional tensor as follows,</span></p>
<p><img class="aligncenter size-full wp-image-5585" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/unnamed-10.png" alt="Changing the view in the tensor | PyTorch | Exxact Corp" width="443" height="382"/></p>
<p><span>Then change the view to a 2-D tensor,</span></p>
<p><img class="aligncenter size-full wp-image-5586" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-6.png" alt="2-D Tensor | PyTorch | Exxact Corp" width="279" height="521"/><br/> <span>Changing back and forth between a PyTorch tensor and a NumPy array is easy and efficient.</span></p>
<p><img class="aligncenter wp-image-5587 size-full" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-7.png" alt="Changing tensor and NumPy | PyTorch | Exxact Corp" width="751" height="602"/></p>
<p><span>Converting from a Pandas series object is also easy,</span></p>
<p><img class="aligncenter size-full wp-image-5588" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-8.png" alt="Conversion from a Panda Series | PyTorch | Exxact Corp" width="513" height="457"/><br/> <span>Finally, converting back to a Python list can be accomplished,</span></p>
<p><img class="aligncenter size-full wp-image-5589" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Tensors-9.png" alt="Converting back to a list | PyTorch | Exxact Corp" width="538" height="207"/></p>
<h2><span>Vector and matrix mathematics with PyTorch tensors</span></h2>
<p><span>PyTorch provides an easy-to-understand API and programmatic toolbox to manipulate tensors mathematically. We show basic operations with 1-D and 2-D tensors here.</span></p>
<p><span>Simple vector addition,</span></p>
<p><img class="aligncenter size-full wp-image-5590" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-1.png" alt="Vector addition | PyTorch | Exxact Corp" width="326" height="458"/><br/> <span>Vector multiplication with a scalar,</span></p>
<p><img class="aligncenter size-full wp-image-5591" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-2.png" alt="Vector multiplication | PyTorch | Exxact Corp" width="429" height="383"/></p>
<p><span>Linear combination,</span></p>
<p><img class="aligncenter size-full wp-image-5592" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-3.png" alt="Linear Combination | PyTorch | Exxact Corp" width="651" height="400"/></p>
<p><span>Element-wise product,</span></p>
<p><img class="aligncenter size-full wp-image-5593" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-4.png" alt="Product of two tensors | PyTorch | Exxact Corp" width="637" height="665"/><br/> <span>Dot product,</span></p>
<p><img class="aligncenter size-full wp-image-5594" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-5.png" alt="PyTorch | Exxact Corp" width="430" height="418"/></p>
<p><span>Adding a scalar to every element of a tensor, i.e. broadcasting</span></p>
<p><img class="aligncenter size-full wp-image-5595" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-6.png" alt="PyTorch | Exxact Corp" width="722" height="341"/><br/> <span>Creating 2-D tensor from list of lists,</span></p>
<p><img class="aligncenter size-full wp-image-5596" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-7.png" alt="PyTorch | Exxact Corp" width="454" height="525"/></p>
<p><span>Slicing and indexing of matrix elements,</span></p>
<p><img class="aligncenter size-full wp-image-5597" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-8.png" alt="PyTorch | Exxact Corp" width="493" height="621"/></p>
<p><span>Matrix multiplication,</span></p>
<p><img class="aligncenter size-full wp-image-5598" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-9.png" alt="PyTorch | Exxact Corp" width="998" height="597"/><br/> <span>Matrix transpose,</span></p>
<p><img class="aligncenter size-full wp-image-5599" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-10.png" alt="PyTorch | Exxact Corp" width="637" height="384"/><br/> <span>Matrix inverse and determinant,</span></p>
<p><img class="aligncenter size-full wp-image-5600" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Math-11.png" alt="PyTorch | Exxact Corp" width="426" height="485"/></p>
<h1><strong>Autograd: Automatic differentiation</strong></h1>
<p><span>Neural network training and prediction involves taking derivatives of various functions (tensor-valued) over and over. The Tensor object supports the magical Autograd feature i.e. automatic differentiation which is achieved by tracking and storing all the operations performed on the Tensor while it flows through a network.</span></p>
<p><span>We show simple examples to illustrate the autograd feature of PyTorch.</span></p>
<p><img class="aligncenter size-full wp-image-5601" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-1.png" alt="PyTorch | Exxact Corp" width="874" height="356"/></p>
<p><span>We define a generic function and a tensor variable </span><b><i>x</i></b><span>, then define another variable </span><b><i>y</i></b><span> assigning it to the function of </span><b><i>x</i></b><span>.</span></p>
<p><img class="aligncenter size-full wp-image-5603" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-2.png" alt="PyTorch | Exxact Corp" width="921" height="541"/></p>
<p><span>Then, we use a special </span><b>backward()</b><span> method on </span><b><i>y</i></b><span> to take the derivative and calculate the derivative value at the given value of </span><b><i>x</i></b><span>.</span></p>
<p><img class="aligncenter size-full wp-image-5602" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-3.png" alt="PyTorch | Exxact Corp" width="1003" height="315"/></p>
<p><span>We can also deal with partial derivatives!</span></p>
<p><img class="aligncenter wp-image-5604 size-full" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-4.png" alt="PyTorch | Exxact Corp" width="797" height="368"/></p>
<p><span>We can define </span><b><i>u</i></b><span> and </span><b><i>v</i></b><span> as tensor variables, define a function combining them, apply the </span><b>backward()</b><span> method and calculate the partial derivatives. See below,</span></p>
<p><img class="aligncenter size-full wp-image-5606" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-5.png" alt="PyTorch | Exxact Corp" width="765" height="693"/></p>
<p><span>PyTorch computes derivatives of scalar functions only but if we pass a vector then essentially it computes derivatives element wise and stores them in an array of the same dimension.</span></p>
<p><img class="aligncenter size-full wp-image-5607" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-6.png" alt="PyTorch | Exxact Corp" width="1000" height="455"/></p>
<p><span>The following code will calculate the derivative with respect to the three constituent vectors.</span></p>
<p><img class="aligncenter size-full wp-image-5608" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-7.png" alt="PyTorch | Exxact Corp" width="553" height="306"/></p>
<p><span>We can show the plot of the derivative. Note, a derivative of a quadratic function is a straight-line, tangent to the parabolic curve.</span></p>
<p><img class="aligncenter size-full wp-image-5609" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/Autograd-8.png" alt="Quadratic Function | PyTorch | Exxact Corp" width="680" height="418"/></p>
<h1><span>Building a Full-Fledged Neural Network</span></h1>
<p><span>Apart from the tensors and automatic differentiation ability, there are few more core components/features of PyTorch that come together for a deep neural network definition.</span></p>
<p><span>The core components of PyTorch that will be used for building the neural classifier are,</span></p>
<ul>
<li><span>The </span><b>Tensor</b><span> (the central data structure in PyTorch)</span></li>
<li><span>The </span><b>Autograd</b><span> feature of the Tensor (automatic differentiation formula baked into the</span></li>
<li><span>The </span><b>nn.Module</b><span> class, that is used to build any other neural classifier class</span></li>
<li><span>The </span><b>Optimizer</b><span> (of course, there are many of them to choose from)</span></li>
<li><span>The </span><b>Loss</b><span> function (a big selection is available for your choice)</span></li>
</ul>
<p><span>We have already described in detail the Tensor and the Autograd. Let us quickly discuss the other components,</span></p>
<h2><strong>The nn.Module Class</strong></h2>
<p><span>In PyTorch, we construct a neural network by defining it as a custom class. However, instead of deriving from the native Python </span><span>object</span><span> this class inherits from the nnModule class</span><span>. This imbues the neural net class with useful properties and powerful methods. This way, full power of the Object-Oriented-Programming (OOP) can be maintained while working with neural net models. We will see a full example of such a class definition in our article.</span></p>
<h2><strong>The Loss Function</strong></h2>
<p><span>In a neural network architecture and operation, the loss functions define how far the final prediction of the neural net is from the ground truth (given labels/classes or data for supervised training). The quantitative measure of loss helps drive the network to move closer to the configuration (the optimal settings of the weights of the neurons) which classifies the given dataset best or predicts the numerical output with least total error.</span></p>
<p><span>PyTorch offers all the usual loss functions for classification and regression tasks —</span></p>
<ul>
<li><span>binary and multi-class cross-entropy,</span></li>
<li><span>mean squared and mean absolute errors,</span></li>
<li><span>smooth L1 loss,</span></li>
<li><span>neg log-likelihood loss, and even</span></li>
<li><span>Kullback-Leibler divergence.</span></li>
</ul>
<p></p>
<p><img class="aligncenter wp-image-5610 size-full" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/article-2.png" alt="Loss Functions PyTorch | PyTorch | Exxact Corp" width="911" height="281"/></p>
<h2><span>The Optimizer</span></h2>
<p><span>Optimization of the weights to achieve the lowest loss is at the heart of the backpropagation algorithm for training a neural network. PyTorch offers a plethora of optimizers to do the job, exposed through the torch.optim module —</span></p>
<ul>
<li><span>Stochastic gradient descent (SGD),</span></li>
<li><span>Adam, Adadelta, Adagrad, SpareAdam,</span></li>
<li><span>L-BFGS,</span></li>
<li><span>RMSprop, etc.</span></li>
</ul>
<h2><strong>The Five-Step-Process</strong></h2>
<p><span>Using these components, we will build the classifier in five simple steps,</span></p>
<ul>
<li><span>Construct our neural network as our custom class (inherited from the </span><b>nn.Module</b><span> class), complete with hidden layer tensors and forward method for propagating the input tensor through various layers and activation function</span></li>
<li><span>Propagate the feature (from a dataset) tensor through the network using this </span><b>forward()</b><span> method — say we get an output tensor as a result</span></li>
<li><span>Calculate the loss by comparing the output to the ground truth and using built-in loss functions</span></li>
<li><span>Propagate the gradient of the loss using the automatic differentiation ability (</span><b>Autograd</b><span>) with the backward method</span></li>
<li><span>Update the weights of the network using the gradient of the loss — this is accomplished by executing one step of the so-called optimizer — </span><b>optimizer.step()</b><span>.</span></li>
</ul>
<p><span>And that’s it. This five-step process constitutes </span><b>one complete epoch of training</b><span>. We just repeat it a bunch of times to drive down the loss and obtain high classification accuracy.</span></p>
<p><span>The idea looks like following,</span></p>
<p><img class="aligncenter size-full wp-image-5612" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/five-step-process.png" alt="Five Step Process | PyTorch | Exxact Corp" width="1400" height="847"/></p>
<h2><strong>Hands-on example</strong></h2>
<p><span>Let’s suppose we want to build and train the following 2-layer neural network.</span></p>
<p><img class="aligncenter size-full wp-image-5613" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/architecture.png" alt="2-Layer Neural Network | PyTorch | Exxact Corp" width="896" height="753"/></p>
<p><span>We start with the class definition,</span></p>
<p><img class="aligncenter size-full wp-image-5614" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/class-def.png" alt="Class Definition | PyTorch | Exxact Corp" width="1057" height="509"/></p>
<p><span>We can define a variable as an object belonging to this class and print the summary.</span></p>
<p><img class="aligncenter size-full wp-image-5615" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/model-1.png" alt="Print Model | PyTorch | Exxact Corp" width="641" height="328"/></p>
<p><span>We choose the Binary cross-entropy loss,</span></p>
<p><img class="aligncenter size-full wp-image-5616" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/model-2.png" alt="Binary Cross-Entropy | PyTorch | Exxact Corp" width="551" height="58"/></p>
<p><span>Let us run the input dataset through the neural net model we have defined i.e. </span><b>forward pass once and compute the output probabilities</b><span>. As the weights have been initialized as random, we will see random output probabilities (mostly close to 0.5). </span><b>This network has not been trained yet</b><span>.</span></p>
<p><img class="aligncenter size-full wp-image-5617" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/unnamed-11.png" alt="Logits | PyTorch | Exxact Corp" width="512" height="27"/></p>
<p><img class="aligncenter size-full wp-image-5618" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/model-3.png" alt="Output Probabilities | PyTorch | Exxact Corp" width="1030" height="256"/></p>
<p><span>We define the optimizer,</span></p>
<p><img class="aligncenter size-full wp-image-5619" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/model-4.png" alt="Optimizer | PyTorch | Exxact Corp" width="504" height="105"/></p>
<p><span>Next, we show how to do forward and backward passes with one step of optimizer. </span><b>This set of code can be found at the heart of any PyTorch neural net model</b><span>. We follow another five-step process,</span></p>
<ul>
<li><span>reset the gradients to zero (to prevent the accumulation of grads)</span></li>
<li><span>forward pass the tensors through the layers</span></li>
<li><span>calculate the loss tensor</span></li>
<li><span>calculate the gradients of the loss</span></li>
<li><span>update the weights by incrementing the optimizer by one step (in the direction of the negative gradient)</span></li>
</ul>
<p><span>The five steps above</span><b><span> </span>are exactly what you can observe and read about in all the theoretical discussion (and in the textbooks) on neural nets and deep learning</b><span>. And, with PyTorch, you are able to implement this process with deceptively simple code, step-by-step.</span></p>
<p><span>The code is shown below,</span></p>
<p><img class="aligncenter size-full wp-image-5620" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/model-5.png" alt="PyTorch Fundamentals | Exxact Corp" width="811" height="534"/></p>
<p><span>When </span><b>we run the same type of code over a loop (for multiple epochs)</b><span>, we can observe the familiar loss-curve going down i.e. the neural network getting trained gradually.</span></p>
<p><img class="aligncenter size-full wp-image-5621" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/model-6.png" alt="Loss over epochs | Exxact Corp" width="611" height="389"/></p>
<p><span>After training for 200 epochs, we can look at the probability distributions again directly to see how the neural network output probabilities are now different (trying to match with the true data distributions).</span></p>
<p><img class="aligncenter size-full wp-image-5622" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/unnamed-12.png" alt="Output Probabilities | Exxact Corp" width="512" height="124"/></p>
<h2><strong>Summary of PyTorch Fundamentals</strong></h2>
<p><span>PyTorch is a great package for reaching out to the heart of a neural net and customizing it for your application or trying out bold new ideas with the architecture, optimization, and mechanics of the network.</span></p>
<p><span>You can easily build complex interconnected networks, try out novel activation functions, mix and match custom loss functions, etc. The core ideas of computation graphs, easy auto-differentiation, and forward and backward flow of tensors will come in handy for any of your neural network definitions and optimization.</span></p>
<p><span>In this article, we summarized a few key steps which can be followed to quickly build a neural network for classification or regression tasks. We also showed how neat ideas can be easily tried out with this framework.</span></p>
<p><span>All the code for this article </span><a href="https://github.com/tirthajyoti/PyTorch_Machine_Learning"><b>can be found here in this Github repo</b></a><span>.</span></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>Which Face is Real? Applying StyleGAN to Create Fake Peopletag:www.datasciencecentral.com,2020-04-29:6448529:BlogPost:9484752020-04-29T20:00:00.000ZJames Montanteshttps://www.datasciencecentral.com/profile/JamesMontantes
<p><span>A Generative model aims to learn and understand a dataset’s true distribution and create new data from it using unsupervised learning. These models (such as StyleGAN) have had mixed success as it is quite difficult to understand the complexities of certain probability distributions. </span></p>
<p><span>In order to sidestep these roadblocks, The Adversarial Nets Framework</span><span> was created whereby the generative model is pitted against an adversary: a discriminative model that…</span></p>
<p><span>A Generative model aims to learn and understand a dataset’s true distribution and create new data from it using unsupervised learning. These models (such as StyleGAN) have had mixed success as it is quite difficult to understand the complexities of certain probability distributions. </span></p>
<p><span>In order to sidestep these roadblocks, The Adversarial Nets Framework</span><span> was created whereby the generative model is pitted against an adversary: a discriminative model that learns to determine whether a sample is from the model distribution or the data distribution.</span></p>
<p><span>The generative model generates samples by passing random noise through a multilayer perceptron, and the discriminative model is also a multilayer perceptron. We refer to this case as Adversarial Nets.</span></p>
<h2><b>Which Face is Real?</b></h2>
<p>Which Face is Real?<span> was developed by Jevin West and Carl Bergstrom </span><span>from the University of Washington</span><span> as part of the Calling Bullshit Project</span><span>. </span></p>
<p><i><span>“Computers are good, but your visual processing systems are even better. If you know what to look for, you can spot these fakes at a single glance — at least for the time being. The hardware and software used to generate them will continue to improve, and it may be only a few years until humans fall behind in the arms race between forgery and detection.” – Jevin West and Carl Bergstrom </span></i></p>
<h2><b>How Do You Tell the Difference Between the Images?</b></h2>
<p><span>The differences are determined in 6 main areas:</span></p>
<h3><b>Water splotches</b></h3>
<ul>
<li><span>The algorithm produces shiny blobs that look somewhat like water splotches on old photographic prints. </span></li>
</ul>
<p><img class="size-full wp-image-5504 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/image1.jpg" alt="Which Face is Real? GAN | Water Splotches | Exxact" width="529" height="176"/></p>
<h3><b>Hair</b></h3>
<ul>
<li><span>Disconnected strands, hair that’s too straight, or too streaked will be common problems when generating hair. </span></li>
</ul>
<p><img class="size-full wp-image-5505 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/image2.jpg" alt="Which Face is Real? GAN | Hair | Exxact" width="532" height="179"/></p>
<h3><b>Asymmetry</b></h3>
<ul>
<li><span>A common problem is asymmetry. Often the frame will take one style at the left and another at the right, or there will be a wayfarer-style ornament on one side but not on the other. Other times the frame will just be crooked or jagged. In addition, asymmetries in facial hair, different earrings in the left and right ear, and different forms of collar or fabric on the left and right side can be present. </span></li>
</ul>
<p><img class="size-full wp-image-5506 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/image3.jpg" alt="Which Face is Real? GAN | Asymmetry | Exxact" width="528" height="175"/></p>
<h3><b>Background problems</b></h3>
<ul>
<li><span>The background of the image may manifest in weird states like blurriness or misshapen objects. This is due to the fact that the neural net is trained on the face and less emphasis is given to the background of the image. </span></li>
</ul>
<p><img class="size-full wp-image-5507 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/unnamed-4.jpg" alt="Which Face is Real? GAN | Background Problems | Exxact" width="512" height="172"/></p>
<h3><b>Fluorescent bleed</b></h3>
<ul>
<li><span>Fluorescent colors can sometimes bleed into the hair or face from the background. And observers can mistake this for colored hair. </span></li>
</ul>
<p><img class="size-full wp-image-5508 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/unnamed-5.jpg" alt="Which Face is Real? GAN | Fluorescent Bleed | Exxact" width="512" height="171"/></p>
<h3><b>Teeth</b></h3>
<ul>
<li><span>Teeth are also hard to render and can come out as odd-shaped, asymmetric, or for those that can identify teeth, sometimes three incisors can appear in the image. </span></li>
</ul>
<p><img class="size-full wp-image-5509 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/image5.jpg" alt="Which Face is Real? GAN | Teeth | Exxact" width="533" height="175"/></p>
<h2><b>Testing Out the StyleGAN Algorithm</b></h2>
<p><span>All the code for the StyleGAN has been open-sourced in the stylegan</span><span> repository. It gives details on how you can run the styleGAN algorithm yourself. So let’s get started with sharing some of the basic system requirements. </span></p>
<h3><b>System Requirements</b></h3>
<ul>
<li><span>Both Linux and Windows are supported, but we strongly recommend Linux for performance and compatibility reasons. </span></li>
<li><span>64-bit Python 3.6 installation. We recommend Anaconda3 with numpy 1.14.3 or newer. </span></li>
<li><span>TensorFlow 1.10.0 or newer with GPU support. </span></li>
<li><span>One or more high-end NVIDIA GPUs with at least 11GB of DRAM. We recommend NVIDIA DGX-1 with 8 Tesla V100 GPUs. </span></li>
<li><span>NVIDIA driver 391.35 or newer, CUDA toolkit 9.0 or newer, cuDNN 7.3.1 or newer. </span></li>
</ul>
<p> </p>
<p><span>A minimal example to try a pre-trained example of the styleGAN is given in pretrained_example.py</span><span>. It is executed as follows:</span></p>
<div class="EnlighterJSWrapper enlighterEnlighterJSWrapper"><ul class="hoverEnabled enlighterEnlighterJS EnlighterJS">
<li class=" odd"><span class="">> python pretrained_example.py</span></li>
<li class=" odd"><span class="">Downloading https</span><span class="co1">://drive.google.com/uc?id=1MEGjdvVpUsu1jB4zrXZN7Y4kBBOzizDQ .... done</span></li>
<li class=" odd"><span class="">Gs Params OutputShape WeightShape</span></li>
<li class=" odd"><span class="">--- --- --- ---</span></li>
<li class=" odd"><span class="">latents_in -</span> <span class="br0">(</span><span class="">?,</span> <span class="nu0">512</span><span class="br0">)</span> <span class="">-</span></li>
<li class=" odd"><span class="">...</span></li>
<li class=" odd"><span class="">images_out -</span> <span class="br0">(</span><span class="">?,</span> <span class="nu0">3</span><span class="">,</span> <span class="nu0">1024</span><span class="">,</span> <span class="nu0">1024</span><span class="br0">)</span> <span class="">-</span></li>
<li class=" odd"><span class="">--- --- --- ---</span></li>
<li class=" odd"><span class="">Total</span> <span class="nu0">26219627</span></li>
<li class=" even"><span class="">Once you execute ‘python pretrained_example.py’, type in ‘ls results’ to see the results.</span></li>
<li class=" even"><span class="">> ls results</span></li>
<li class=" even"><span class="">example.png</span> <span class="co1"># <a href="https://drive.google.com/uc?id=1UDLT_zb-rof9kKH0GwiJW_bS9MoZi8oP">https://drive.google.com/uc?id=1UDLT_zb-rof9kKH0GwiJW_bS9MoZi8oP</a></span></li>
</ul>
</div>
<h2><b>Prepare the Datasets for Training</b></h2>
<p><span>The training and evaluation scripts operate on datasets stored as multi-resolution TFRecords. Each dataset is represented by a directory containing the same image data in several resolutions to enable efficient streaming. There is a separate *.tfrecords file for each resolution, and if the dataset contains labels, they are stored in a separate file as well. By default, the scripts expect to find the datasets at</span><span> </span><span>datasets/<NAME>/<NAME>-<RESOLUTION>.tfrecords</span><i><span>. </span></i><span>The directory can be changed by editing config.py</span><span>:</span></p>
<div class="EnlighterJSWrapper enlighterEnlighterJSWrapper"><ol class="hoverEnabled enlighterEnlighterJS EnlighterJS">
<li class=" odd"><span class="">result_dir =</span> <span class="st0">'results'</span></li>
<li class=" odd"><span class="">data_dir =</span> <span class="st0">'datasets'</span></li>
<li class=" odd"><span class="">cache_dir =</span> <span class="st0">'cache'</span></li>
</ol>
</div>
<p><span>To obtain the FFHQ dataset (</span><span>datasets/ffhq</span><span>), please refer to the </span><a href="https://github.com/NVlabs/ffhq-dataset"><span>Flickr-Faces-HQ repository</span></a><span>.</span></p>
<p><span>To obtain the CelebA-HQ dataset (</span><span>datasets/celebahq</span><span>), please refer to the </span><a href="https://github.com/tkarras/progressive_growing_of_gans"><span>Progressive GAN repository</span></a><span>.</span></p>
<p><span>To obtain other datasets, including LSUN, please consult their corresponding project pages. The datasets can be converted to multi-resolution TFRecords using the provided </span><a href="https://github.com/NVlabs/stylegan/blob/master/dataset_tool.py"><span>dataset_tool.py</span></a><span>:</span></p>
<div class="EnlighterJSWrapper enlighterEnlighterJSWrapper"><ul class="hoverEnabled enlighterEnlighterJS EnlighterJS">
<li class=" odd"><span class="">> python dataset_tool.py create_lsun datasets/lsun-bedroom-full ~/lsun/bedroom_lmdb --resolution</span> <span class="nu0">256</span></li>
<li class=" odd"><span class="">> python dataset_tool.py create_lsun_wide datasets/lsun-car-512x384 ~/lsun/car_lmdb --width</span> <span class="nu0">512</span> <span class="">--height</span> <span class="nu0">384</span></li>
<li class=" odd"><span class="">> python dataset_tool.py create_lsun datasets/lsun-cat-full ~/lsun/cat_lmdb --resolution</span> <span class="nu0">256</span></li>
<li class=" odd"><span class="">> python dataset_tool.py create_cifar10 datasets/cifar10 ~/cifar10</span></li>
<li class=" odd"><span class="">> python dataset_tool.py create_from_images datasets/custom-dataset ~/custom-images</span></li>
</ul>
</div>
<p> </p>
<h2><b>Training the StyleGAN Networks</b></h2>
<p><span>Once the datasets are set up, you can train your own StyleGAN networks as follows:</span></p>
<ol>
<li><span>Edit </span><a href="https://github.com/NVlabs/stylegan/blob/master/train.py"><span>train.py</span></a><span> to specify the dataset and training configuration by uncommenting or editing specific lines. </span></li>
<li><span>Run the training script with </span><span>python train.py</span><span>. </span></li>
<li><span>The results are written to a newly created directory </span><span>results/<ID>-<DESCRIPTION></span><span>. </span></li>
<li><span>The training may take several days (or weeks) to complete, depending on the configuration. </span></li>
</ol>
<p><span>By default, </span><span>train.py</span><span> is configured to train the highest-quality StyleGAN (configuration F in Table 1) for the FFHQ dataset at 1024×1024 resolution using 8 GPUs</span><i><span>.</span></i></p>
<h2><b>Expected StyleGAN Training Time </b></h2>
<p><span>Below you will find NVIDIA’s reported expected training times for default configuration of the </span><span>train.py</span><span> script (available in the stylegan repository) on a Tesla V100 GPU for the </span><a href="https://github.com/NVlabs/ffhq-dataset"><span>FFHQ dataset</span></a><span> (available in the stylegan repository).</span></p>
<p><img class="size-full wp-image-5510 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/image7.jpg" alt="Which Face is Real? GAN | StyleGan Training Time | Exxact" width="510" height="175"/></p>
<h2><b>StyleGAN: Bringing it All Together</b></h2>
<p><span>The algorithm behind this amazing app was the brainchild of </span><b>Tero Karras, Samuli Laine and Timo Aila</b><span> at NVIDIA and called it </span><b><i>StyleGAN</i></b><span>. The algorithm is based on earlier work by Ian Goodfellow and colleagues on General Adversarial Networks (GAN’s). </span></p>
<p><b><i>Generative models have a limitation in which it’s hard to control the characteristics such as facial features from photographs. NVIDIA’s StyleGAN is a fix to this limitation. The model allows the user to tune hyper-parameters that can control for the differences in the photographs.</i></b></p>
<p><span>StyleGAN solves the variability of photos by adding </span><i><span>styles</span></i><span> to images at each convolution layer. These styles represent different features of photos of a human, such as facial features, background color, hair, wrinkles, etc. The model generates two images A and B and then combines them by taking low-level features from A and the rest from B. At each level, different features (styles) are used to generate an image:</span></p>
<ul>
<li><span>Coarse styles (resolution between 4 to 8) – </span><i><span>pose, hair, face, shape</span></i><span> </span></li>
<li><span>Middle styles (resolution between 16 to 32) – </span><i><span>facial features, eyes</span></i><span> </span></li>
<li><span>Fine styles (resolution between 64 to 1024)- </span><i><span>color scheme</span></i><span> </span></li>
</ul>
<p><img class="size-full wp-image-5511 aligncenter" src="https://blog.exxactcorp.com/wp-content/uploads/2020/04/unnamed-9.png" alt="Which Face is Real? GAN | Nvidia’s Neural Network | Exxact" width="512" height="288"/></p>
<p><span>Have you tested out StyleGAN before? Or is this your first time? Let us know in the comment section below. We are always looking for new and creative ways from the community for any technologies or frameworks. </span></p>