A.I. based automated Anomaly detection system is gaining popularity nowadays due to the increase in data generated from various devices and the increase in ever evolving sophisticated threats from hackers etc. Anomaly detection systems can be applied across various business scenarios like monitoring financial transactions of a fintech company, highlighting fraudulent activities in a network, e-commerce price glitches among millions of products, and so on. Anomaly detection system can work…

Introduction to topic model:

In machine learning and natural language processing, a topic model is a type of statistical model for discovering the abstract "topics" that occur in a collection of documents. Topic modeling is a frequently used text-mining tool for discovery of hidden semantic structures in a text body. 

In topic modeling, a topic is defined by a cluster of words with each word in the cluster having a probability of occurrence for the given topic, …

As we make more cashless payments for retail purchases, restaurants, and transportation – not to mention the increase in online shopping – wallets loaded with legal tender may become a thing of the past. According to 2018 research by BigCommerce, software vendor and Square payment processing solution provider, 51 percent of Americans think that online shopping is the best option. Last year, 1.66 billion people worldwide bought goods online. And the…

Summary: There are some interesting use cases where combining CNNs and RNN/LSTMs seems to make sense and a number of researchers pursuing this.  However, the latest trends in CNNs may make this obsolete.

There are things that just don’t seem to go together.  Take oil and water for instance.  Both valuable, but try putting…

In a previous blog I wrote about 6 potential applications of time series data. To recap, they are the following:

1. Trend analysis
2. Outlier/anomaly detection
3. Examining shocks/unexpected variation
4. Association analysis
5. Forecasting
6. Predictive analytics

Here I am focusing on outlier…

The use of formal statistical methods to analyse quantitative data in data science has increased considerably over the last few years. One such approach, Bayesian Decision Theory (BDT), also known as Bayesian Hypothesis Testing and Bayesian inference, is a fundamental statistical approach that quantifies the tradeoffs between various decisions using…

In a previous blog post we have seen how to build Convolutional Neural Networks (CNN) in Tensorflow, by building various CNN architectures (like LeNet5, AlexNet, VGGNet-16) from scratch and training them on the MNIST, CIFAR-10 and Oxflower17 datasets.

It starts to get interesting when you start thinking about the practical applications of CNN and other Deep Learning methods.  If you have been following the latest technical developments you probably know that CNN’s are…

Big Data tools not only simplify lengthy analytical procedures in any industry, but they also provide a competitive advantage to banks. With new regulations, banks are looking at ways to make compliance procedures more effective and accurate. Big Data in banking is slowly gaining momentum and becoming an inevitable necessity across the banking industry. As traditional…

In recent years, the field of object detection has seen tremendous progress, aided by the advent of deep learning. Object detection is the task of identifying objects in an image and drawing bounding boxes around them, i.e. localizing them. It’s a very important problem in computer vision due its numerous applications from self-driving cars to security and tracking.

Prior approaches of object detection…

Below are some images generated by an algorithm that I originally created to study stocks. The stock in this case is Yahoo! in the early 2000s. I realize that none of these images resemble conventional stock market patterns. Keep in mind that algorithms by nature interact with the data. They don't simply restate the numbers in graphical form. An algorithm designed to deal with historical stock trading data can be modified to deal with the data in real-time. However, there is no inherent…

I have been writing about the Crosswave Differential Algorithm for a number of years. I described in previous blogs how the algorithm emerged almost by accident while I was attempting to write an application intended to support quality control. In this blog I will be discussing the event model that powers the algorithm. Events are the details and circumstances…

I find myself habitually using the term "metrics." When I first started blogging, I normally used this term only in reference to performance metrics. These are not ordinary "readings" but rather criteria-driven amounts - the criteria being performance. Over the years I have come to recognize that data-gathering is normally premised on some type of criteria. When compiling revenue data, it should be noted that analysts are seeking out data pertaining to revenues. The quest is predefined. The…

I will be using this blog to assemble a number of different concepts that I introduced over many years in previous blogs (indicated in bold); then I will explain where all of this will be going in the future.  I am turning 50 years old in a couple of weeks, and I find that I habitually take inventory of my belongings these days before beginning any lengthy mission or journey.  I recently acquired a fairly expensive device called a CPAP machine.  It resembles a small stereo with…

In recent blogs, I wrote about using codified narrative as a form of data. I also discussed using attribution models to systematically evaluate codified narrative for ontological constructs: e.g. "child abuse" "physical confinement" "cannibalism." I provide a brief overview of these topics a bit later in the blog. The third important piece to make use of narrative data involves "attribution profiling" in a process that I call "catching scent." Following the odour of data involves…

With the online holiday shopping season heavily underway and expected to reach $83 billion this year, online stores need to run smoothly, including having prices in order. It may sound basic, but simple computer glitches can cause pricing errors that may excite consumers, but cause major problems down the line. For example, for a few hours this summer, visitors…

Ashley Madison, IRS, Target, Sony…What do they have in common? Here we only name a few but of the most tremendous crisis of data breach in recent years - yes, it is happening and it is happening everywhere. The cost of data breach comes to a new high at $154 per record of stolen or leaked data, adding up to millions of data for each incident, including the law suits,…

The idea of environmental determinism once made a lot of sense. Hostile climates and habitats prevented the expansion of human populations. The conceptual opposite of determinism is called possibilism. These days, human populations can found living in many inhospitable habitats. This isn't because humans have physically evolved. But rather, we normally occupy built-environments. We exist through our technologies and advanced forms of social interaction: a person might not be able to build a…

Above is a distribution of price differentials for the Dow Jones Industrial Average from the 1930s. The image was generated by one of my programs called Storm. I posted a few images from the same application in other blogs. If I recall correctly, the more volatile differentials (closer to the action) are at top; the more stable differentials (further from the…

Embodiment is comparable to the idea of an “ecosystemic” or “holistic” approach. In an ecosystem, each thing affects everything else. In light of the interrelationship, a person would not attempt to correct a problem by considering only a single piece of the puzzle. Instead, there is a need to bring together many aspects of the body. To understand embodiment, it is necessary to recognize how “the body” separates an organism from its environment; in a manner of speaking, the body represents…