The following advice is built from my experience working as a data scientist on a variety of projects across different data & engineering teams. Many data scientists (myself included) do not come from a computer science or software development background, so may not have formal training or good habits in code writing. These tips should help data scientists work collaboratively to write good code and build models in a way that will be easier to…

In order to prevent my programs from freezing up while running long calculations, I generally run the calculations on separate threads.  In Java, this process can be accomplished by separating the GUI from processing.  In the code below, a thread for an instance of MyProcessing would be invoked using start(): e.g. “(new MyProcessing()).start();” would run indefinitely until T is made null.  T can be made null by calling stop() or by directly making T null.  Often when the GUI is closing, I…

 This blog contains some snippets of code that I tend to use in Java.  I acknowledge that somebody else writing this blog might include different code.  Except for a short course at Sun Educational Services, most of my Java programming skills are self-taught.  I’m unsure if people with formal backgrounds in computer science might have different styles and conventions.  Mine have been shaped primarily by my needs.

Creating a Graphical User Interface…

When the performance of an employee is evaluated, ideally there are no externalities to complicate the analysis. If the employee has a computer that is constantly freezing up - or the servers in the company frequently operate slowly - the employee's performance data will reflect the functionality and effectiveness of these systems. If the company occupies a highly competitive market, declining sales data is attributable at least in part to competition rather than the behaviours of employees.…

If I want to build a house, wouldn't it be wise to learn carpentry? Does the analogy hold for data-analytic multivariate models? Or is it simply enough to let a machine do it, with no knowledge by the machine operator of how to interpret the results from those modeling efforts? Or is it true, as one person has recently asserted, that he could replicate ALL statistical procedures and techniques using MapReduce, without knowing anything about statistics and probability, or the vast collection…

In this post, I discuss the basic characteristics of code that I have personally used to extract online data - in a process these days often called data-mining.  I intend to cover some general features.  Those that wish to do so can also compile the coding samples.

Over the years, I have programmed in a number of computer programming languages including Visual Basic, Perl, Python, and LISP (AutoLISP).  The coding samples on this blog are written in Java, my language of…