When I read a blog, I often find myself in deep thought as I approach the end, trying to determine if the author has said anything that I might be able to use. A blog doesn't have to say anything. Nor does it have to be useful to me specifically. It might simply offer a personal reflection on life. As a person who also writes blogs, I…

Skills, skills, skills!!! Which ones should I learn? Which ones do I need to land the job, to impress the client, to prepare for the future, to stay relevant? What programming languages should I learn? What technologies should I master? What business books should I read? Is there a course I can take, or a certification I can enroll in? Should I focus on being a…

Given the nature of the community, presumably many visitors already have a strong understanding of the nature of quantitative data. Perhaps more mysterious is the idea of qualitative data especially since it can sometimes be expressed in quantitative terms. For instance, "stress" as an internal response to an externality differs from person to person; yet it would be possible to canvas a large number of people and express stress levels as an aggregate based on a perceptual gradient: minimal,…

Being the son of a mechanic, I have spent many years handling power tools. I'm especially fond of a couple of hammer-drills in my possession. They can effortlessly drill holes through concrete. At least, this is what my father once claimed. He handed down his most treasured tools to me. I'm big on pliers and screwdrivers. This might be due to my vocational training as a technician. Even today - long after I completed my diploma and continued to further my education - I still carry a licence…

We've given Hadoop almost 10 years to mature, invested billions, and very few companies are  seeing the return on investment.  Several companies have tried to make Hadoop a real-time analytical platform, incorporating SQL-like facades on top, but the latency is still not where it needs to be for interactive applications.  Even Google, a true big data user, has moved on and is using more dataflow / flow-based programming approaches.    Why?  It just makes sense...

• Why should I…

The idea of environmental determinism once made a lot of sense. Hostile climates and habitats prevented the expansion of human populations. The conceptual opposite of determinism is called possibilism. These days, human populations can found living in many inhospitable habitats. This isn't because humans have physically evolved. But rather, we normally occupy built-environments. We exist through our technologies and advanced forms of social interaction: a person might not be able to build a…

This blog is about the peculiar nature in which software sometimes gets developed. I hope that many readers will recognize the relevance of data science in the examples taken from my own projects. I propose that development is the product of creativity more than accreditation. Creativity is something complicated that interacts with a person over his or her life circumstances. Many people know how to write . . . sentences and paragraphs. However, the ability to write well does not necessarily…

In this post, I discuss the basic characteristics of code that I have personally used to extract online data - in a process these days often called data-mining.  I intend to cover some general features.  Those that wish to do so can also compile the coding samples.

Over the years, I have programmed in a number of computer programming languages including Visual Basic, Perl, Python, and LISP (AutoLISP).  The coding samples on this blog are written in Java, my language of…