A key strength of NLP (natural language processing) is being able to process large amounts of texts and then summarise them to extract meaningful insights.

In this example, a selection of economic bulletins in PDF format from 2018 to 2019 are analysed in order to gauge economic sentiment. The bulletins in question are sourced from the European Central Bank website. tf-idf is used to rank…

The key to perform any text mining operation, such as topic detection or sentiment analysis, is to transform words into numbers, sequences of words into sequences of numbers. Once we have numbers, we are back in the well-known game of data analytics, where machine learning algorithms can help us with classifying and clustering.

We will focus here exactly on that part of the analysis that transforms words…

I.  Introduction

This is a continuation of my previous blog, “Natural Language Understanding – Application Notes with Context Discriminant”. 

Background:

Natural Language Understanding (NLU) is a subtopic of Natural Language Processing (NLP). Successful implementations of NLU are difficult because of limitations in prevailing technology. SiteFocus solved these limitations with a new approach to NLU. This approach has been successfully…

Summary:  The addition of AI capabilities to our personal devices, applications, and even self-driving cars has caused us to take a much deeper look at what we call ‘User Experience’ (Ux).  A more analytical framework identified as Cognitive Ergonomics is becoming an important field for data scientists to understand and implement.

I…

Introduction

Business ventures based on existing or disruptive business models taking on the route of Initial Public Offering are always a challenge to investors who want to profit from early investment into those would be “unicorn IPO”. A good investment may get worse before it gets better. Others may get worse and never recover. Aside from the macroeconomics and consumer trends that could affect the outcome of such investments, the fundamentals of these new public offerings…

Introduction

Deep Learning can be used to automate just about every repetitive task that is currently or formerly performed by humans. Factory robots, autonomous cars, Internet of Things are example of these automations. Yet, mentally challenging tasks such as conducting research or strategic planning with natural language textual documents remain a daunting task for automation. We look into the root cause of this challenge and have implemented a solution to automate these…

Text Analytics with Python -- A Practical Real-World Approach to Gaining Actionable Insights from your Data

Text analytics can be a bit overwhelming and frustrating at times with the unstructured and noisy nature of textual data and the vast amount of information available. "Text Analytics with Python" published by Apress\Springer, is a book packed with 385 pages of useful information based on techniques, algorithms,…

This post covers the following tasks using R programming:

• cleans the texts,
• sorts and aggregates by publisher names
• creates word clouds and word…

Context Matters When Text Mining

This post highlights a number of important applications found for deep learning so far. It is well known that 80% of data is unstructured. Unstructured data is the messy stuff every quantitative analyst tries to traditionally stay away from. It can include images of accidents, text notes of loss adjusters, social media comments, claim documents and review of medical doctors etc. Unstructured data has massive potential but has never been traditionally considered as a source of insight before.…

How many times a day do we ourselves, or hear someone else, utter the phrase “Google it”? It’s hard to imagine that a phrase so ubiquitous and universally understood has been around for less than two decades. The word “Google” has become synonymous with online search, and when we think about why this, it’s because Google yields the most relevant, comprehensive results, quickly. Essentially, it has changed the way we find and interact with content and information.

We’ve seen the…

Organizations are struggling with a fundamental challenge – there’s far more data than they can handle.  Sure, there’s a shared vision to analyze structured and unstructured data in support of better decision making but is this a reality for most companies?  The big data tidal wave is transforming the database management industry, employee skill sets, and business strategy as organizations race to unlock meaningful connections between disparate sources of…

Summary: Gartner says that predictive analytics is a mature technology yet only one company in eight is currently utilizing this ability to predict the future of sales, finance, production, and virtually every other area of the…