Data science development is very different from software development, and getting the two to mesh is sometimes like trying to cobble together Tinker Toys with Lincoln Logs. One data science expert once described the differences as:
Software development is “Measure twice; cut once,” while Data Science is “Cut, cut, cut!”
The methodologies and processes that support successful software development do not work for data science projects according to one…Continue
Added by Bill Schmarzo on October 17, 2018 at 4:23am — No Comments
Organizations looking for justification to move beyond legacy reporting, should review this little ditty from the healthcare industry:
The report identified six major areas of waste: unnecessary services ($210 billion annually); inefficient…Continue
Added by Bill Schmarzo on October 2, 2018 at 2:30pm — No Comments
AI has snuck into our daily lives and there is nothing to stop it. Not only does AI power autonomous vehicles, but AI already decides what products you should buy, what movies you should watch, what music you should listen to, and whom you should date. If you talk to your iPhone, Google Home, and Amazon Echo, you are talking to an AI engine that powers these personal virtual assistants. AI decides whether you are approved for a loan, determines the outcome of a job applications, identifies…Continue
Added by Bill Schmarzo on September 30, 2018 at 2:11pm — No Comments
Oh,the lowly data engineer. Harvard Business Review declared the role of the data scientist as “the sexiest job in the 21stcentury.” But the data engineer labors away in near obscurity acquiring, transforming, enriching, munging and preparing data for the data scientist to do their black magic.
In addition to building data pipelines –…Continue
Added by Bill Schmarzo on September 26, 2018 at 6:22am — No Comments
There are so many confusing and sometimes even counter-intuitive concepts in statistics. I mean, come on…even explaining the differences between Null Hypothesis and Alternative Hypothesis can be an ordeal. All I want to do is to understand and quantify the cost of my analytical models being wrong.
For example, let’s say that I’m a shepherd who has bad eyesight and have a hard time distinguishing between a wolf and a sheep dog. That’s obviously…Continue
Perhaps to no one’s surprise, the growth in companies implementing Big Data and Analytics projects continues to climb – as evidenced by the continued growth in data lakes. As most companies begin to implement their Big Data and Analytics strategy, they struggle to show value for their efforts. This can come from several areas:
Added by Bill Schmarzo on September 22, 2018 at 7:08am — No Comments
Today’s Artificial Intelligence (AI) discussions remind me of a Steve Martin skit from the early Saturday Night Live days (1979). In the skit titled “What the Hell is that?”, Steve Martin, later joined by Bill Murray, is looking in the distance at something, repeatedly asking the question “What the hell is that?” The skit reminds me of today’s AI discussions about “What the hell is AI?”, which distracts from…Continue
Added by Bill Schmarzo on September 18, 2018 at 4:30am — No Comments
Reviving from the dead an old but popular blog on Understanding Type I and Type II Errors
I recently got an inquiry that asked me to clarify the difference between type I and type II errors when doing statistical testing. Let me use this blog to clarify the difference as well as discuss the potential cost ramifications of type I and type II errors. I have also provided some examples at the end of the blog.
In statistical test theory, the…Continue
Updated from original posted on April 17, 2014
The importance of metadata only continues to grow as organizations are realizing that to fully exploit the business and operational potential of machine learning, deep learning and artificial intelligence requires that the raw data be enhanced with metadata. And while we have growing volumes of actual data, there is even more data, or metadata, around the usage and source of the actual data.
Added by Bill Schmarzo on July 23, 2018 at 4:30am — No Comments
Will I ever get this digital transformation thing right? The more work I do with clients on their digital transformation initiatives, the more I realize how much I don’t know. For example, first there was the “4 Laws of Digital Transformation”:
Added by Bill Schmarzo on July 11, 2018 at 9:02am — No Comments
In an attempt to put the patient first in healthcare, Congress and President Obama in 2015 approved a bipartisan bill for United States healthcare reform. The bill is known as “Medicare Access and CHIP Reauthorization Act of 2015”, or MACRA. Among the major provisions of MACRA is the Quality Payment Program. Under the Quality Payment Program, physicians, and nurses receive positive, neutral, or negative Medicare payment adjustments based upon a…Continue
The key of any organization’s digital transformation is becoming more effective at leveraging data and analytics to power their business models. That is, how can organizations exploit the growing bounty of internal and external data sources to uncover new sources of customer, product, service, operational and market insights that they can use to optimize key business and operational processes, mitigate compliance and cybersecurity risks, uncover new monetization opportunities,…Continue
Gartner’s recently released “Magic Quadrant for Industrial IoT Platforms” outlines how organizations can leverage the Internet of Things (IoT) to drive their digital transformation initiatives. In particular, Gartner believes that “By 2020, on-premises Internet of Things (IoT) platforms coupled with edge computing will account for up to 60% of industrial IoT (IIoT) analytics, up from less than 10% today.”
More real-time sensor and…Continue
Added by Bill Schmarzo on June 20, 2018 at 1:30pm — No Comments