Let’s explore the complexity and vulnerability of IT infrastructure and how to build a modern IT infrastructure monitoring solution, using a combination of time series databases with machine learning.

IT Infrastructure: Complex and Vulnerable…

We investigate a large class of auto-correlated, stationary time series, proposing a new statistical test to measure departure from the base model, known as Brownian motion. We also discuss a methodology to deconstruct these time series, in order to identify the root mechanism that generates the observations. The time series studied here can be discrete or continuous in time, they  can have various degrees of smoothness (typically measured using the Hurst exponent) as well as long-range or…

Summary: Recurrent Neural Nets (RNNs) are at the core of the most common AI applications in use today but we are rapidly recognizing broad time series problem types where they don’t fit well.  Several alternatives are already in use and one that’s just been introduced, ODE net is a radical departure from our way of thinking about the solution.

 …

1. Background

New home construction plays a significant role in housing economy, while simultaneously impacting other sectors such as timber, furniture and home appliances. New house sales is also an important indicator of country’s overall economic health and direction. In the last 50 years there has been few significant bumps and turning points in this sector that shaped the trajectory of the overall economy.  Here I review the…

Recently Kaggle master Kazanova along with some of his friends released a "How to win a data science competition" Coursera course. The Course involved a final project which itself was a time series prediction problem. Here I will describe how I got a top 10 position as of writing this…

In a previous blog I wrote about 6 potential applications of time series data. To recap, they are the following:

1. Trend analysis
2. Outlier/anomaly detection
3. Examining shocks/unexpected variation
4. Association analysis
5. Forecasting
6. Predictive analytics

Here I am focusing on outlier…

A time series is a sequence of data points recorded at specific time points - most often in regular time intervals (seconds, hours, days, months etc.). Every organization generates a high volume of data every single day – be it sales figure, revenue, traffic, or operating cost. Time series data mining can generate valuable information for long-term business decisions, yet they are underutilized in most organizations. Below is a list of few possible ways to…

The majority of industry and academic numeric predictive projects deal with deterministic or point forecasts of expected values of a random variable given some conditional information. In some cases, these predictions are enough for decision making. However, these predictions don’t say much about the uncertainty of your underlying stochastic process. A common desire of all data scientists is to make predictions for an uncertain future. Clearly then, forecasts should…

Sales prediction is an important part of modern business intelligence.  First approaches one can apply to predict sales time series are such conventional methods of forecasting as ARIMA and  Holt-Winters. But there are several challenges while using these methods. They are: multilevel daily/weekly/monthly/yearly seasonality,  many exogenous  factors which impact sales, complex trends in different time periods. In such cases, it is not easy to apply conventional methods. Of course, there is…

Time series forecasting is hardly a new problem in data science and statistics. The term is self-explanatory and has been on business analysts’ agenda for decades now: The very first practices of time series analysis and forecasting trace back to the early 1920s.

The underlying idea of time series forecasting is to look at historical data from the time perspective, define the patterns, and yield short or long-term predictions on how –  considering the captured patterns – target…

Missing data present significant challenges to trend analysis of time series. Straightforward approaches consisting of supplementing missing data with constant or zero values or with linear trends can severely degrade the quality of the trend analysis, which significantly reduces the reliability of the trend analysis. …

Time-series data arise in many fields including finance, signal processing, speech recognition and medicine. A standard approach to time-series problems usually requires manual engineering of features which can then be fed into a machine learning algorithm. Engineering of features generally requires some  domain knowledge of the discipline where the data has originated from. For example, if one is dealing with signals (i.e. classification of EEG signals), then possible features would involve…

In this post, we consider different approaches for time series modeling. The forecasting approaches using linear models, ARIMA alpgorithm, XGBoost machine learning algorithm are described. Results of different model combinations are shown. For probabilistic modeling the approaches using copulas and Bayesian inference are considered.

INTRODUCTION

Time series analysis, especially forecasting, is an important problem of modern…