Our model for recognizing specific animals in images is a neural network consisting of multiple layers, and the initial layers are already good at understanding the world in general. So instead of “re-inventing the wheel,” we only need to train the final layers.

I was excited to work on a recent project with one of our partners, Wild Detect, because it aligns with one of our goals at Appsilon — to use data science consulting to aid in the…

A.I. based automated Anomaly detection system is gaining popularity nowadays due to the increase in data generated from various devices and the increase in ever evolving sophisticated threats from hackers etc. Anomaly detection systems can be applied across various business scenarios like monitoring financial transactions of a fintech company, highlighting fraudulent activities in a network, e-commerce price glitches among millions of products, and so on. Anomaly detection system can work…

Cost driver: One of the key drivers of cost for ecommerce businesses is the last mile delivery charges. Consumers have the option to switch between e-retailers depending on their willingness to pay delivery charges, which is generally significantly low. This notion puts the power into the hands of the customer. Delivery charges, when added just before payment, makes the customer rethink and is one of the reasons for…

Introduction to topic model:

In machine learning and natural language processing, a topic model is a type of statistical model for discovering the abstract "topics" that occur in a collection of documents. Topic modeling is a frequently used text-mining tool for discovery of hidden semantic structures in a text body. 

In topic modeling, a topic is defined by a cluster of words with each word in the cluster having a probability of occurrence for the given topic, …

As we make more cashless payments for retail purchases, restaurants, and transportation – not to mention the increase in online shopping – wallets loaded with legal tender may become a thing of the past. According to 2018 research by BigCommerce, software vendor and Square payment processing solution provider, 51 percent of Americans think that online shopping is the best option. Last year, 1.66 billion people worldwide bought goods online. And the…

Summary: There are some interesting use cases where combining CNNs and RNN/LSTMs seems to make sense and a number of researchers pursuing this.  However, the latest trends in CNNs may make this obsolete.

There are things that just don’t seem to go together.  Take oil and water for instance.  Both valuable, but try putting…

In a previous blog I wrote about 6 potential applications of time series data. To recap, they are the following:

1. Trend analysis
2. Outlier/anomaly detection
3. Examining shocks/unexpected variation
4. Association analysis
5. Forecasting
6. Predictive analytics

Here I am focusing on outlier…

The use of formal statistical methods to analyse quantitative data in data science has increased considerably over the last few years. One such approach, Bayesian Decision Theory (BDT), also known as Bayesian Hypothesis Testing and Bayesian inference, is a fundamental statistical approach that quantifies the tradeoffs between various decisions using…

In a previous blog post we have seen how to build Convolutional Neural Networks (CNN) in Tensorflow, by building various CNN architectures (like LeNet5, AlexNet, VGGNet-16) from scratch and training them on the MNIST, CIFAR-10 and Oxflower17 datasets.

It starts to get interesting when you start thinking about the practical applications of CNN and other Deep Learning methods.  If you have been following the latest technical developments you probably know that CNN’s are…

Big Data tools not only simplify lengthy analytical procedures in any industry, but they also provide a competitive advantage to banks. With new regulations, banks are looking at ways to make compliance procedures more effective and accurate. Big Data in banking is slowly gaining momentum and becoming an inevitable necessity across the banking industry. As traditional…

In recent years, the field of object detection has seen tremendous progress, aided by the advent of deep learning. Object detection is the task of identifying objects in an image and drawing bounding boxes around them, i.e. localizing them. It’s a very important problem in computer vision due its numerous applications from self-driving cars to security and tracking.

Prior approaches of object detection…

Below are some images generated by an algorithm that I originally created to study stocks. The stock in this case is Yahoo! in the early 2000s. I realize that none of these images resemble conventional stock market patterns. Keep in mind that algorithms by nature interact with the data. They don't simply restate the numbers in graphical form. An algorithm designed to deal with historical stock trading data can be modified to deal with the data in real-time. However, there is no inherent…

I have been writing about the Crosswave Differential Algorithm for a number of years. I described in previous blogs how the algorithm emerged almost by accident while I was attempting to write an application intended to support quality control. In this blog I will be discussing the event model that powers the algorithm. Events are the details and circumstances…

I find myself habitually using the term "metrics." When I first started blogging, I normally used this term only in reference to performance metrics. These are not ordinary "readings" but rather criteria-driven amounts - the criteria being performance. Over the years I have come to recognize that data-gathering is normally premised on some type of criteria. When compiling revenue data, it should be noted that analysts are seeking out data pertaining to revenues. The quest is predefined. The…

I will be using this blog to assemble a number of different concepts that I introduced over many years in previous blogs (indicated in bold); then I will explain where all of this will be going in the future.  I am turning 50 years old in a couple of weeks, and I find that I habitually take inventory of my belongings these days before beginning any lengthy mission or journey.  I recently acquired a fairly expensive device called a CPAP machine.  It resembles a small stereo with…

In recent blogs, I wrote about using codified narrative as a form of data. I also discussed using attribution models to systematically evaluate codified narrative for ontological constructs: e.g. "child abuse" "physical confinement" "cannibalism." I provide a brief overview of these topics a bit later in the blog. The third important piece to make use of narrative data involves "attribution profiling" in a process that I call "catching scent." Following the odour of data involves…

With the online holiday shopping season heavily underway and expected to reach $83 billion this year, online stores need to run smoothly, including having prices in order. It may sound basic, but simple computer glitches can cause pricing errors that may excite consumers, but cause major problems down the line. For example, for a few hours this summer, visitors…

Ashley Madison, IRS, Target, Sony…What do they have in common? Here we only name a few but of the most tremendous crisis of data breach in recent years - yes, it is happening and it is happening everywhere. The cost of data breach comes to a new high at $154 per record of stolen or leaked data, adding up to millions of data for each incident, including the law suits,…

The idea of environmental determinism once made a lot of sense. Hostile climates and habitats prevented the expansion of human populations. The conceptual opposite of determinism is called possibilism. These days, human populations can found living in many inhospitable habitats. This isn't because humans have physically evolved. But rather, we normally occupy built-environments. We exist through our technologies and advanced forms of social interaction: a person might not be able to build a…

Above is a distribution of price differentials for the Dow Jones Industrial Average from the 1930s. The image was generated by one of my programs called Storm. I posted a few images from the same application in other blogs. If I recall correctly, the more volatile differentials (closer to the action) are at top; the more stable differentials (further from the…