At the Data Science Association our members often complain about the major data engineering problem of finding the right tools and programming models to build both robust data processing pipelines and efficient ETL processes for data transformation and integration.…
Added by Michael Walker on May 19, 2016 at 10:00pm — No Comments
A new 2016 survey entitled "Big Data Executive Survey 2016" concludes that data variety is the top data priority for most firms. Seasoned data science practitioners have long known that …Continue
Watch out folks - there is a new breed of health analytics firms sprouting like weeds - mining "big data" about you and making broad and bold predictions about your state of health to make workplace decisions.
Added by Michael Walker on February 18, 2016 at 12:34am — No Comments
Does human intelligence have any connection to the type of music a person listens to? Can we define human intelligence? How do you measure human intelligence? Do SAT scores accurately measure human intelligence? Is there any evidence that SAT scores accurately predict educational or workplace performance?
I am skeptical 1) that we…
Last week we discussed the importance of data scientists prioritizing client confidentiality and the concern of exposing high-value information to…Continue
Added by Michael Walker on October 1, 2015 at 8:07pm — No Comments
The Data Science Association (DSA) and Google is interested in learning more about your experience with tools and training. Click on the link below to take a 10-minute survey for data scientists.…Continue
Added by Michael Walker on October 1, 2015 at 7:53pm — No Comments
Many organizations are reluctant to create data science teams (internally or externally) because of information confidentiality and privacy concerns. It is dangerous to open the kimono to competition - disclosing high-value information about inner workings of the firm may cause…Continue
Added by Michael Walker on September 24, 2015 at 3:00pm — No Comments
Added by Michael Walker on June 1, 2015 at 7:30am — No Comments
United States President Barack Obama recently introduced DJ Patil as the new Chief Data Scientist of the United States Government. See video: Data Science: Where are We Going?…Continue
I often receive phone calls from organizations, aspiring data scientists and reporters about whether data science would be a good career choice for women. My response is absolutely yes, for the following reasons:
Women are great contrarian…Continue
There is often confusion between the definitions of "data veracity" and "data quality".
Data veracity is sometimes thought as uncertain or imprecise data, yet may be more precisely defined as false or inaccurate data. The data may be intentionally, negligently or mistakenly falsified. Data veracity may be distinguished from data quality,…Continue
We all know models are always an illusion of reality - yet may or may not be useful. Data scientists should build both simple and complex models with reasonable and testable assumptions that capture what is…Continue
Added by Michael Walker on December 5, 2014 at 6:31pm — No Comments
While large data sets may provide significant value in certain cases, data diversity and integrating smart data points will provide more consistent actionable insights and high value intelligence leading to better decision-making.
For example, consider NFL football data. Focusing on large football game data sets is usually not helpful and often misleading creating…
The current (November 2014) United States election reminds us that sophisticated data science techniques are employed on the public in attempt to influence opinion and persuade votes. The slick television advertising, debate prevarications, and policy position distortions and exaggerations have soured many citizens on the current state of modern democracy. Indeed, most feel we are not getting the straight scoop - the…Continue
A new paper entitled "Gaining Trust as well as Respect in Communicating Science Topics" published in the Proceedings of the National…Continue
Added by Michael Walker on September 24, 2014 at 5:07pm — No Comments
The "Big Data" marketing hype obscures the fact that more actionable, valuable insights are likely to be found in the right smaller "Smart Data" sets in contrast to large data sets.
While the term "Big Data" is properly defined as data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time - the…
The "big data" technology vendor market is ripe for consolidation. The myriad of vendors and technologies is causing market confusion. Matt Turck created a nice visualization entitled "…Continue
According to recent survey by Burtch Works.
Added by Michael Walker on August 25, 2014 at 12:10pm — No Comments