Let’s explore the complexity and vulnerability of IT infrastructure and how to build a modern IT infrastructure monitoring solution, using a combination of time series databases with machine learning.

IT Infrastructure: Complex and Vulnerable…

We investigate a large class of auto-correlated, stationary time series, proposing a new statistical test to measure departure from the base model, known as Brownian motion. We also discuss a methodology to deconstruct these time series, in order to identify the root mechanism that generates the observations. The time series studied here can be discrete or continuous in time, they  can have various degrees of smoothness (typically measured using the Hurst exponent) as well as long-range or…

Summary: Recurrent Neural Nets (RNNs) are at the core of the most common AI applications in use today but we are rapidly recognizing broad time series problem types where they don’t fit well.  Several alternatives are already in use and one that’s just been introduced, ODE net is a radical departure from our way of thinking about the solution.

 …

Understanding customer transactional behaviour pays well for any business. With the tsunami of start ups in recent times and the immense money flow in businesses, customers find lucrative offers from companies for acquisition, retention & referrals strategies. Understanding transactional behaviour of a customer has become even more complex with the invent of new business houses everyday. Although, with the rise of powerful machines, one can…

Recently Kaggle master Kazanova along with some of his friends released a "How to win a data science competition" Coursera course. The Course involved a final project which itself was a time series prediction problem. Here I will describe how I got a top 10 position as of writing this…

In my most recent blog, I discussed the idea of aligning the supply of services to market demand.  My conceptualization of “alignment” specifically relates to time intervals: i.e. having people at the right place and at the right time - for example, to take advantage of opportunities - is a sign of alignment.  Alignment for me is often about the relationship between capacity and incapacity:  the ability to supply services versus the inability to satisfy the market demand…

In the “Ecology of Metrics,” I wrote about “alignment” being a type of metric; alignment can measure the extent to which an organization’s supply or capacity is matched against the demands or needs of the market.  For instance, in a call centre, it would be highly desirable to have agents available to respond to calls at “precisely” the same time that clients are making calls.  If alignment is off even by only 15 to 30 seconds, impatient clients might hang up and never call again.  Similarly…

The majority of industry and academic numeric predictive projects deal with deterministic or point forecasts of expected values of a random variable given some conditional information. In some cases, these predictions are enough for decision making. However, these predictions don’t say much about the uncertainty of your underlying stochastic process. A common desire of all data scientists is to make predictions for an uncertain future. Clearly then, forecasts should…

Sales prediction is an important part of modern business intelligence.  First approaches one can apply to predict sales time series are such conventional methods of forecasting as ARIMA and  Holt-Winters. But there are several challenges while using these methods. They are: multilevel daily/weekly/monthly/yearly seasonality,  many exogenous  factors which impact sales, complex trends in different time periods. In such cases, it is not easy to apply conventional methods. Of course, there is…

Time series forecasting is hardly a new problem in data science and statistics. The term is self-explanatory and has been on business analysts’ agenda for decades now: The very first practices of time series analysis and forecasting trace back to the early 1920s.

The underlying idea of time series forecasting is to look at historical data from the time perspective, define the patterns, and yield short or long-term predictions on how –  considering the captured patterns – target…

In 2006, marketing commentator Michael Palmer had blogged, “Data is just like crude. It’s valuable, but if unrefined it cannot really be used.”

After nine years, the statement still holds true across any industry that depends on large volumes of data. It is true that until and unless, data is not broken down into pieces and analyzed, it holds little value.

As the world becomes more receptive to the advantages of big data, the oil industry does not seem to be far…

Time-series data arise in many fields including finance, signal processing, speech recognition and medicine. A standard approach to time-series problems usually requires manual engineering of features which can then be fed into a machine learning algorithm. Engineering of features generally requires some  domain knowledge of the discipline where the data has originated from. For example, if one is dealing with signals (i.e. classification of EEG signals), then possible features would involve…

Data modeling has been a fixture of enterprise architecture since the 1970’s with ANSI defined conceptual, logical, and physical data schema.    As data models developed, so did the availability of templates for business use.   Retail banks use similar data models, as do other industries.  A shared approach to data modeling advanced the discussion and planning of solutions. 

Growth in unstructured data has led to development of tools to search data to identify context or bring…

In this post, we consider different approaches for time series modeling. The forecasting approaches using linear models, ARIMA alpgorithm, XGBoost machine learning algorithm are described. Results of different model combinations are shown. For probabilistic modeling the approaches using copulas and Bayesian inference are considered.

INTRODUCTION

Time series analysis, especially forecasting, is an important problem of modern…

PREAMBLE:  

This video was built as a result of our internal hackathon using Teradata Listener to absorb real time small messages from Transformers and other devices on the Power Grid in Southern California. The video demonstrates a real time predictive analytic showcasing proactive repairs of the power grid to reduce costs and avoid disruptions of power service.…

The world moves with faster speed every day. Folk, companies, and entrepreneurs try to react at an ever-increasing speed. Reaching the limits of a human's ability to reaction, tools are build to process the massive and big amounts of data available to decision makers, analyze and present it. The processing of this data has a number of different application areas.…

Collaborative business intelligence is an environment In which users can communicate and collaborate each other with ease, they are able to sharing information, ideas, and decision making in their communities.

Information retention

Each and every day, no one holds the millions of data items of intellectual property (telephone calls , conversations, and e-mails) in companies and organizations across the world. Using important collaborative software to…

Did you know that Big Data can help us predict who the future football stars will be? This is precisely what happened in Riyad Mahrez’s case, PFA Player of the Year, who has conquered the English Premier League with Leicester City.

During last year’s Big Data Week Conference in London, Malta-based entrepreneur Valery Bollier correctly foresaw…

The need for info within the twenty first century continues to intensify—and shows no sign of subsiding. Today’s decision manufacturers would like to be of an incredible volume and style of info, leading more companies to deploy analytics that not solely facilitate them sense and respond to key business problems, but additionally facilitate them build predictions and act based mostly on period of…