Given the nature of the community, presumably many visitors already have a strong understanding of the nature of quantitative data. Perhaps more mysterious is the idea of qualitative data especially since it can sometimes be expressed in quantitative terms. For instance, "stress" as an internal response to an externality differs from person to person; yet it would be possible to canvas a large number of people and express stress levels as an aggregate based on a perceptual gradient: minimal,…

Being the son of a mechanic, I have spent many years handling power tools. I'm especially fond of a couple of hammer-drills in my possession. They can effortlessly drill holes through concrete. At least, this is what my father once claimed. He handed down his most treasured tools to me. I'm big on pliers and screwdrivers. This might be due to my vocational training as a technician. Even today - long after I completed my diploma and continued to further my education - I still carry a licence…

Somebody once mentioned to me that there is a need for a standard method of performance evaluation that can be applied to all employees regardless of their exact duties: e.g. to compare a janitor to an accountant. In my jurisdiction, there is a regulatory requirement for "equal pay for work of equal value" that can affect companies with government contracts. I consider the concept of "equal value" complicated due to its subjective nature. Certainly two people handling exactly the same work…

A few semesters into my undergraduate studies, I decided to move away from campus in order to escape the incessant party atmosphere. I chose to rent an old but roomy apartment in Kitchener, Ontario. I read somewhere that this city was formerly called New Munich. One day in the dead of morning, I heard loud banging and smashing downstairs. It seemed that neither I nor my cat could sleep, so I went to investigate. I saw the owner of the building at the front entrance of the apartment holding a…

I was often the lone wolf among my peers in university because I supported a prominent place in society for corporations and an important social role for capital. I questioned whether the directors and executives of companies entered into boardrooms really intending to “oppress” people such as minorities and people with disabilities. Did they deliberately make bathrooms inaccessible to people in wheelchairs perhaps to advance their preconceptions of who gets to go to the bathroom, I pondered…

If I want to build a house, wouldn't it be wise to learn carpentry? Does the analogy hold for data-analytic multivariate models? Or is it simply enough to let a machine do it, with no knowledge by the machine operator of how to interpret the results from those modeling efforts? Or is it true, as one person has recently asserted, that he could replicate ALL statistical procedures and techniques using MapReduce, without knowing anything about statistics and probability, or the vast collection…

Embodiment is comparable to the idea of an “ecosystemic” or “holistic” approach. In an ecosystem, each thing affects everything else. In light of the interrelationship, a person would not attempt to correct a problem by considering only a single piece of the puzzle. Instead, there is a need to bring together many aspects of the body. To understand embodiment, it is necessary to recognize how “the body” separates an organism from its environment; in a manner of speaking, the body represents…

Illegal fishing is a significant economic and environmental challenge for countries around the world.  Up to 40% of fishing catch in certain parts of the world is unlawful or unregulated, resulting in approximately $10B to $20B in economic losses and significantly depleting…

Excerpt reprinted with permission from ckmadvisors.com

This weekend sees Super Bowl XLVIII come to New York (yes, we're well aware that the stadium is technically in New Jersey). Earlier this week one of our data scientists noticed the Empire State Building lights putting on quite a show. A quick search revealed that the iconic building's…

Much more devices held, much more messages sent, much more data up in the air.

Upon the arrival of IOT (Internet of Things) era, number of connected devices grows rapidly, the signals of which stack up mines containing valuable, hidden insights.

We used to analyze log files for risk management, spotting the anomalies and exceptions based on outlined records. Via the ever-richer meta data and context, we are entitled to weave more story from the strings now. By union or…

Excerpt reprinted with permission from ckmadvisors.com

By CKM Advisors Natural Language Analysis Team, 

Last year we posted a popular piece offering our view on…

By: Nicholas Hartman, Director at CKM Advisors

Today we'd like to share with you some fun charts that have come out of our internal linguistics research efforts. Specifically, studying weather events by analyzing social media traffic from Twitter. 

We do not specialize in social media and most of our data analytics work focuses on the…

We have been using tables in the relational database, mostly for the transactional purposes, and that proves effective. Considering the data size and analytic purpose, however, the data structure might need to be redesigned for better efficiency.

To determine how to decompose the complexity of big data, we have observed the way the organisms function. In the physical world, the universe is organized into a hierarchy of…

To be short, in-memory computing takes advantage of physical memory, which is expected to process data much faster than disk. In-place, on the other hand, fully utilizes the address space of 64bit architecture. Both are gifts from the modern computer science; both are essences of the BigObject. 

In-place computing only becomes possible upon the introduction of 64bit architecture, whose address space is big enough to hold the entire data set for most of cases we are dealing with today.…

Hi all,

This is my first post here. I'm glad to introduce this newly launched big data analytic engine, the BigObject. In the past 2 years we have been working on an optimal approach to handle big data for analytic purposes and challenging the existed models, some assumptions of which are no longer valid. For example, as the data size grows so rapidly, is it still practical that we stick to the relational models neglecting the time spending in data retrievals? What impact did…

Data Mining and Cookie Butter

There’s been a great deal of discussion over the past several weeks regarding data mining and predictive models.  Terms like “meta data” and “algorithm” are fast moving from the domain of IT practitioners and into the realm of water cooler discussion.  This might be a good opportunity to briefly review some of these concepts in order to better understand data mining practices and standards.

First, some terms.  

Meta Data - refers…

During analysis of movements of individuals in public places, there are only two dimensions that can represent movement of an individual, shown via data saved between starting and end point, even incorporating elevators and stairs to different (shop) levels). That is a multi-linear way of looking at movements of individuals in crowds in a specific environment. Most big shopping corporates use these kinds of analysis methods.

But what if (like in 3D environments) movement can be up and…

Big Data can help in mapping and understanding customer behaviors, and in developing one-to-one marketing programs or innovative services. However, Big Data is too often presented as a technological capability subsequently requiring armies of data scientists to mine and analyse data.

Yes, managing and exploiting the growing amount of internal and external data is a necessary condition to steer business performance. But it is far from a sufficient condition.

In a recent meeting…

From MIT's Technology Review - 

"Data science is so new that there are no textbooks on the subject, and no university curricula designed to turn out data scientists. Yet it's integral to everything from quantitative trading on Wall Street to ad targeting on the web and the optimization of real-world supply chains."

Read more...