Subscribe to DSC Newsletter

All Blog Posts Tagged '#BigData' (137)

Data Science “Paint by the Numbers” with the Hypothesis Development Canvas

 When I was a kid, I use to love “Paint by the Numbers” sets.  Makes anyone who can paint or color between the lines a Rembrandt or Leonardo da Vinci (we can talk later about the long-term impact of forcing kids to “stay between the lines”).

 

Well, the design world is applying the “Paint by the Numbers” concept using design canvases. …

Continue

Added by Bill Schmarzo on October 23, 2018 at 7:00am — No Comments

Critical Thinking and Becoming “Students of Data Science”

Social media and internet search scare me because it is so easy to acquire information without also acquiring the associated knowledge. It’s easy to find almost any information via search or social media, but an effective “student of data scientist” (as well as any effective future leader) must learn to discern easily acquired information with a sense of skepticism. To quote the character Dr. Ian Malcolm (played skillfully by Jeffrey Goldblum) in the movie “Jurassic Park:”…

Continue

Added by Bill Schmarzo on October 20, 2018 at 12:29pm — No Comments

Embracing Conflict to Fuel Digital Innovation

When talking to clients about their business goals, most business executives are pretty clear as to what they want to accomplish, such as reducing customer churn or reducing inventory costs or improving quality of care or improving product line profitability.  But these “one dimensional” business initiatives really don’t push the organization’s innovative thinking.  For example, I can easily reduce marketing costs if I significantly reduce advertising and…

Continue

Added by Bill Schmarzo on October 20, 2018 at 4:03am — No Comments

Why Is Data Science Different than Software Development?  It Starts with Data…Lots o’ DATA!!

Data science development is very different from software development, and getting the two to mesh is sometimes like trying to cobble together Tinker Toys with Lincoln Logs.  One data science expert once described the differences as:

Software development is “Measure twice; cut once,” while Data Science is “Cut, cut, cut!”

The methodologies and processes that support successful software development do not work for data science projects according to one…

Continue

Added by Bill Schmarzo on October 17, 2018 at 4:23am — No Comments

Predict ►Prescribe ►Prevent Analytics Value Cycle

Organizations looking for justification to move beyond legacy reporting, should review this little ditty from the healthcare industry:

The Institute of Medicine (IOM) estimates that the United States loses $750 billion annually to medical fraud, inefficiencies, and other siphons in the health-care system[1].

The report identified six major areas of waste: unnecessary services ($210 billion annually); inefficient…

Continue

Added by Bill Schmarzo on October 2, 2018 at 2:30pm — No Comments

AI and the Future of Millennials

AI has snuck into our daily lives and there is nothing to stop it. Not only does AI power autonomous vehicles, but AI already decides what products you should buy, what movies you should watch, what music you should listen to, and whom you should date. If you talk to your iPhone, Google Home, and Amazon Echo, you are talking to an AI engine that powers these personal virtual assistants. AI decides whether you are approved for a loan, determines the outcome of a job applications, identifies…

Continue

Added by Bill Schmarzo on September 30, 2018 at 2:11pm — No Comments

Data Engineers: Nobody Puts Baby in a Corner!

Oh,the lowly data engineer.  Harvard Business Review declared the role of the data scientist as “the sexiest job in the 21stcentury.” But the data engineer labors away in near obscurity acquiring, transforming, enriching, munging and preparing data for the data scientist to do their black magic.

In addition to building data pipelines –…

Continue

Added by Bill Schmarzo on September 26, 2018 at 6:22am — No Comments

Using Confusion Matrices to Quantify the Cost of Being Wrong

There are so many confusing and sometimes even counter-intuitive concepts in statistics.  I mean, come on…even explaining the differences between Null Hypothesis and Alternative Hypothesis can be an ordeal.  All I want to do is to understand and quantify the cost of my analytical models being wrong.

For example, let’s say that I’m a shepherd who has bad eyesight and have a hard time distinguishing between a wolf and a sheep dog.  That’s obviously…

Continue

Added by Bill Schmarzo on September 25, 2018 at 8:42am — 1 Comment

How do you know if you’re getting value from your data?

Perhaps to no one’s surprise, the growth in companies implementing Big Data and Analytics projects continues to climb – as evidenced by the continued growth in data lakes.  As most companies begin to implement their Big Data and Analytics strategy, they struggle to show value for their efforts.  This can come from several areas:

  • According to…
Continue

Added by Bill Schmarzo on September 22, 2018 at 7:08am — No Comments

Aggregated Data Dilemma

Okay, I am weird (tell me something that I don’t know, say most of my friends).  For Christmas I wanted a Nike Apple Watch to go with my existing FitBit and Garmin fitness trackers (I look sort of like a cyborg in the photo below…which is always cool). 

While I was intrigued by the ability to do all sorts of cool things on the…

Continue

Added by Bill Schmarzo on September 18, 2018 at 9:29am — No Comments

Defining AI Not as Important as Exploiting AI

Today’s Artificial Intelligence (AI) discussions remind me of a Steve Martin skit from the early Saturday Night Live days (1979). In the skit titled “What the Hell is that?”, Steve Martin, later joined by Bill Murray, is looking in the distance at something, repeatedly asking the question “What the hell is that?”  The skit reminds me of today’s AI discussions about “What the hell is AI?”, which distracts from…

Continue

Added by Bill Schmarzo on September 18, 2018 at 4:30am — No Comments

Embracing Conflict to Fuel Digital Innovation

When talking to clients about their business goals, most business executives are pretty clear as to what they want to accomplish, such as reducing customer churn or reducing inventory costs or improving quality of care or improving product line profitability.  But these “one dimensional” business initiatives really don’t push the organization’s innovative thinking.  For example, I can easily reduce marketing costs if I significantly reduce advertising and…

Continue

Added by Bill Schmarzo on September 12, 2018 at 4:21am — No Comments

Why Accept the Hype? Time to Transform How We Approach Emerging Technology

It’s like a rite of passage – a new technology pops onto the scene and the “pundits,” analysts and venture capitalists (among others) immediately start to over-hype the technology:

It’ll stop world hunger

It’ll cure cancer

It’ll end poverty

It’ll help the Cubs win the World Series (okay, that one was true)

And the ultimate result? The technology is forced to traverse the infamous Gartner Hype Cycle, soaring to the Peak of…

Continue

Added by Bill Schmarzo on September 7, 2018 at 5:10am — No Comments

A Winning Game Plan For Building Your Data Science Team

One of the most exciting challenges I have at Hitachi as the Vice-Chairmen of Hitachi’s “Data Science 部会” is to help lead the development of Hitachi’s data science capabilities. We have a target number of people who we want trained and operational by 2020, so there is definitely a sense of urgency. And I like urgency because it’s required to sweep aside the inhibitors and resistors to change.

I started this assignment with a blog titled “…

Continue

Added by Bill Schmarzo on August 31, 2018 at 7:30pm — No Comments

Prioritization Matrix:  Aligning Business and IT On The Big Data Journey

One key challenge to a successful Big Data journey is gaining consensus and alignment between the business and IT stakeholders in identifying the initial big data business use cases that 1) deliver sufficient value to the business, while 2) possessing a high probability of success.  One can find multiple business use cases where big data and advanced analytics can deliver compelling business value.  However, many of these use cases have a low probability of…

Continue

Added by Bill Schmarzo on August 22, 2018 at 8:51am — No Comments

3 Stages of Creating Smart

Tomorrow’s market winners will win with the smartest products.  It’s not enough to just build insanely great products; winners must have the smartest products!” – Bill Schmarzo

Okay, that’s a pretty bold statement on my part (especially to challenge the famous Steve Jobs statement about building insanely great products), but then again I’m an analytics dude and think that analytics should be a part of every product and space – smart cities, smart cars, smart…

Continue

Added by Bill Schmarzo on August 22, 2018 at 8:12am — No Comments

Understanding Type I and Type II Errors

Reviving from the dead an old but popular blog on Understanding Type I and Type II Errors

I recently got an inquiry that asked me to clarify the difference between type I and type II errors when doing statistical testing.  Let me use this blog to clarify the difference as well as discuss the potential cost ramifications of type I and type II errors. I have also provided some examples at the end of the blog[1]

In statistical test theory, the…

Continue

Added by Bill Schmarzo on August 18, 2018 at 3:35pm — 2 Comments

Why Digital Transformation Is So Dang Hard?

I recently talked to Clint Boulton, author of “Why digital transformations are lagging” about why Digital Transformation is so hard. His article raises lots of interesting points about the inhibitors of Digital Transformation, so I thought I’d share my thoughts about how to overcome some of these challenges.

Forrester published a report entitled “…

Continue

Added by Bill Schmarzo on August 9, 2018 at 6:00am — No Comments

Leveraging Agent-based Models (ABM) and Digital Twins to Prevent Injuries

On the surface, preventing injuries to professional-caliber athletes would seem to have little in common with preventing operational failures for a machine (i.e., autonomous vehicle, locomotive, airplane, CT Scan).  However, both athletes and machines deal with inter-twined complex systems (where the interactions of one complex system can have a ripple effect on others) that can have significant impact on their operational effectiveness.

My son Max, the Director of…

Continue

Added by Bill Schmarzo on August 6, 2018 at 11:19am — No Comments

Open Peer-to-Peer Communications to Facilitate Real-time Insights Sharing

My blog “Blockchain + Analytics: Enabling Smart IOT” drew some great feedback asking me to clarify my autonomous vehicle example that used blockchain as a means of near real-time, peer-to-peer communications between clusters of intelligent devices and machines.  But first, some background.

Edge analytics within an Internet of Things (IOT) world is very…

Continue

Added by Bill Schmarzo on August 4, 2018 at 6:08am — No Comments

Monthly Archives

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

1999

Videos

  • Add Videos
  • View All

© 2020   TechTarget, Inc.   Powered by

Badges  |  Report an Issue  |  Privacy Policy  |  Terms of Service

console.log("HostName");