Quite often, non-technical executives have difficulties understanding what programming, on a very fundamental level, is all about. Because of that knowledge-gap, they tend to hire and overburden experienced data professionals with tasks which they are hopelessly overqualified for. Such as, for example, doing ad-hoc SQL queries on CRM data: "You're the go-to-guy for all things data, and we need the results for the board meeting tomorrow." That's a quite humbling and frustrating…

Over the last few years, Excel has been redesigned from the ground up. Currently, Microsoft is making the new Excel core-features available to every user, regardless of your Office 365 license. Thanks to the Microsoft naming conventions, it is easy to confuse the new features with existing ones. That being said, Power Query and Power Pivot are not the same things as Pivot Tables, which you have likely been using for years.…

Here is a short blog I was asked to make about making a personal Wiki from Wikipedia. It shows the basic steps in text processing so I hope it will be useful for data scientists. It also requires some knowledge of MediaWiki setup on a web server, and some (not very advanced) knowledge of the Python programming language. It takes only several days to create this Wiki with Wikipedia articles if you know…

Summary: Python’s open-source and high-level nature, as well as its comprehensive libraries, make it the perfect fit to solve the numerous real-life ML challenges.

The increasing popularity and accessibility of Artificial Intelligence solutions is rapidly reshaping many industries, from healthcare through finance to aviation. Although the application of the latest technologies has always been an essential consideration for companies striving to get…

Why would a data scientist use Kafka Jupyter Python KSQL TensorFlow all together in a single notebook?

There is an impedance mismatch between model development using Python and its Machine Learning tool stack and a scalable, reliable data platform. The former is what you need for quick and easy prototyping to build analytic models. The latter is what you need to use for data ingestion, preprocessing, model deployment and monitoring at scale. It…

October is historically the most volatile month for stocks, but is this a persistent signal or just noise in the data?

Stocks, Significance Testing & p-Hacking. Follow me on Twitter (twitter.com/pdquant) for more. Over the past 32 years, October has been the most volatile month on average for the S&P500 and December the least, in this article we will use simulation to assess the…

Electronic spreadsheets have been around for nearly 40 years now. They were invented by Bob Frankston and Dan Bricklin, founders of VisiCalc, and I had a chance to chat with both gentlemen a couple of months ago. I highly recommend watching this TED talk with Dan Bricklin:

It's important to understand for…

My writing engagement at Data Science Central came up unexpectedly. Back in August 2018, I stumbled upon an excellent write-up on Data Science Central. The author, Bill Vorhies, shared his thoughts on career transitioning toward data science. I wrote him an email, complimenting him on his blog post, and I dropped a few lines about my own transition. Here's his response:

"Congratulations on your remarkable journey. Perhaps you’d like to write one or more articles…

Is not a secret that Data Science tools like Jupyter, Apache Zeppelin or the more recently launched …

Over the last years, my small business has undergone a digital transformation from a marketing service company to a data literacy consultancy. What does a data literacy consultancy do? We teach business users within large enterprises to work with data, and we help them acquire the necessary skills from state of the art Excel to Python, querying structured, semi-structured and unstructured databases, as well as math, statistics, and probability.

Throughout our…

English is becoming the official language in the global business world, being currently spoken by approximately 1.75 billion people worldwide according to Harvard Business Review. While English is the fastest spreading language in human history, a significant proportion of businesses are still resistant to giving up…

Latest update: November 16, 2018

Microsoft Excel has been around for over 30 years now, and chances are it's not going to change in the foreseeable future. In fact, Excel is facing immense competition from challengers such as Google Spreadsheets and well-funded start-ups like Airtable, which are both going after Excel's massive user base of approximately 500 million worldwide. Tech-savvy small and mid-sized businesses embrace innovative alternatives to Excel. However,…

At the time of writing this post, I am nine months into my learning sabbatical. You can read about my journey here: “Career Transition Towards Data Analytics & Science”. Today I will share with you how you can plan your own, unique learning sabbatical, regardless of its scope and duration –…

With the rise of IoT devices (Internet of Things), being able to analyze and visualize live streams of data is becoming more and more important. For example, you could have sensors like thermometers in machines or portable medical devices like pacemakers, continuously streaming data to a streaming service like Kafka. PixieDust makes it easier to work with live data inside Jupyter Notebooks by providing simple integration APIs to both the PixieApp…

Introduction

Did you know that you can write R and Python code within your T-SQL statements? Machine Learning Services   in SQLServer eliminates the need for data movement. Instead of transferring large and sensitive data over the network or losing accuracy with sample csv files, you can have your R/Python code execute within your database. Easily deploy your R/Python code with SQL stored procedures making them accessible in your…

Stochastic Signal Analysis is a field of science concerned with the processing, modification and analysis of (stochastic) signals.

Anyone with a background in Physics or Engineering knows to some degree about signal analysis techniques, what these technique are and how they can be used to analyze, model and classify signals.

Data Scientists coming from a different fields, like Computer Science or Statistics, might not be aware of the analytical power these techniques bring with…

Many of us are bombarded with various recommendations in our day to day life, be it on e-commerce sites or social media sites. Some of the recommendations look relevant but some create range of emotions in people, varying from confusion to anger.

There are basically two types of recommender systems, Content based and Collaborative filtering. Both have their pros and cons depending upon the…

Summary

In machine learning, a convolutional neural network (CNN, or ConvNet) is a class of neural networks that has successfully been applied to image recognition and analysis. In this project I've approached this class of models trying to apply it to stock market prediction, combining stock prices with sentiment analysis. The implementation of the network has been made…