I recently conducted a 3-day, remote “Data Monetization: Thinking Like a Data Scientist” workshop for a transportation agency in the Middle East. Doing this training remotely is a personal challenge as I miss the face-to-face interaction in ideating, validating, and prioritizing the business areas that can benefit from data and analytics. However, conducting the workshop remotely did provide some valuable learnings for me.
One learning was my “Thinking Like a Data Scientist” visual was outdated (Figure 1).
Figure 1: Original “Thinking Like a Data Scientist” (TLADS) Visual
Figure 1 portrayed the “Thinking Like a Data Scientist” (TLADS) process as a linear process, where you would complete one step and then cleanly move onto the next step. But in reality, the process is highly iterative where it is common for learnings from one step to impact an earlier step such as refining the KPIs against which the targeted business initiative’s progress and success will be measured. So, I created an updated “Thinking Like a Data Scientist” visual in Figure 2 to reflect the highly iterative nature of the TLADS process.
Figure 2: Updated “Thinking Like a Data Scientist” Visual
One other learning from the workshop was the need to spend more time thoroughly understanding and defining the “value” that the business initiative sought to create. And that’s where things get tricky. Too many organizations limit how they define and measure “value”. And defining a robust and diverse set of KPIs and metrics against which to measure business initiative progress and success becomes critically important as we apply AI to continuously-optimize the business initiative.
To understand the AI “value” quandary, one must first understand how an AI model works:
Bottom-line: the AI model determines “right and wrong” actions based upon the definition of “value” as articulated in the AI Utility Function (Figure 3).
Consequently, we must invest the effort across a diverse set of stakeholders to thoroughly explore and validate a diverse set of metrics and KPIs against which the AI model is seeking to optimize. And that’s exactly one of the key objectives of the TLADS process.
If we want AI to work for us humans (versus us humans working for AI), then we must thoroughly define “value” before we start building our AI ML models. Consequently, I expanded the TLADS process to drive a more thorough exploration and definition of “value” (Figure 4).
Figure 4: How Does Your Organization Define “Value”
A more holistic suite of “value” dimensions that today’s organizations need to consider are represented in red in Figure 4. To support the ideation around the exploration of these expanded value dimensions, I updated TLADS Step #1 and Template #1 (Figure 5) to include:
Note: capturing a robust set of KPIs, benefits and impediments should not be difficult if 1) you have a diverse group of stakeholders participating in the brainstorming process and 2) you ensure that everyone has an equal voice in the ideation process.
See Figure 5 for an updated Template #1 using my traditional Chipotle example, where the items in red are related to the expanded definition of “value” for Chipotle.
Figure 5: Updated Template 1 of the “Thinking Like a Data Scientist” methodology
By the way, how your organization defines “value” probably says more about your organization than whatever your charter and mission statement states. Or said another way:
You are what you measure, and you measure what you reward
Yea, you may say that your organization’s charter is such and such, but your organization’s charter is actually defined by the metrics and KPIs against which you measure (and reward) business success. Period.
Value definition is critical from an AI execution perspective. Unfortunately, in a rush to get to the fun part of the AI job and start playing with the AI algorithms, organizations sometimes shortchange the upfront work in thoroughly and holistically defining the value (metrics and KPIs) against which business initiative progress and success will be measured.
Organizations must be thoughtful and thorough in how it defines the values against which the operations of the business will be measured. Getting those “values” wrong can lead to unintended, biased, and disastrous consequences in your AI models (check out Terminators, VIKI, and ARIIA…your homework assignment for my next blog).