Home » Business Topics » Data Strategist

AI Effectiveness Starts by Understanding User Intent

  • Bill Schmarzo 
Slide6

If you want any more proof about how much AI has integrated itself into our daily lives, go no further than the map on your smart phone.  Whether you use Google Maps or Apple Maps or Waze (also owned by Google), these AI-infused apps are amazing at getting you from Point A to Point B in the shortest amount of time.  The give you step-by-step directions and can update those directions based upon any problems (traffic accidents, weather, potholes, special events, etc.) that pop up during your trip (Figure 1).

Slide1-1

Figure 1: AI-powered Maps (GPS) Provides Fastest Route From Point A to Point B

Yes, the AI systems that power the Apple and Google map apps are optimized to get you from Point A to Point B as quickly as possible.

But what if getting from Point A to Point B in the fastest time was not your intent?  Based upon your intentions, the AI model might give you a different route considering the variables and metrics against which you would measure trip or route effectiveness.

The effectiveness of your AI model starts by understanding user intent. That is, you must understand what the user is trying to accomplish, their desired outcomes, the decisions that they need to make, and the KPIs and metrics against which the user will measure progress and success.

Relevant and responsible AI model optimization starts by understanding user intent.

Importance of Understanding User Intent

I’m shocked at how many AI projects start without first trying to understand the context of what is trying to be accomplished.  Without context, it is impossible to distinguish signal from noise in the data; that is, what’s valuable and what’s not valuable in the data!  And you can’t determine context without first understanding Intent; that is, what is the user trying to accomplish and how will they measure effectiveness.

Ensuring that your AI model is delivering relevant, accurate, and responsible outcomes requires determining and quantifying user intent which includes (Figure 2):

  • Intent Determination. What is the user trying to accomplish (their Intent), their objectives, their desired outcomes, and how will the user measure effectiveness of their intent?
  • Value Definition (AI Utility Function). What are the variables and metrics (features) against which the AI model will seek to optimize in order to achieve user intent?
  • Value Creation (AI Model). What actions or decisions does the AI model need to make to optimize the variables and metrics against which intent success will be measured?
Slide2

Figure 2: The Power of Determining User Intent

The article “Adding Context Will Take AI to the Next Level” by Emil Eifrem states:

“Essentially AI needs context in order to mimic human levels of intelligence. Context is the information that frames something to give it meaning, after all. For example, a person who says “Get out!” may either be expressing a friendly note of surprise, or angrily demanding that someone leave the room. However, you can’t decide that simply by reading the text alone.”

Intent and context are closely interrelated. Intent refers to a user’s desired outcome or purpose, while context is the circumstances surrounding intent and against which intent can be fully understood and assessed.

Variables, Weights, and the AI Model Optimization

AI Utility Function is comprised of the variables and metrics – and their associated weights – against which the AI model will seek to optimize as the basis for its decisions and actions to achieve the stated outcome. An AI model’s preferences over possible actions can be defined by a function that maps these outcomes to a utility value; the higher the value, the more the AI model likes that action (Figure 3)

Slide3-1

Figure 3: Role of AI Utility Function

The AI Model’s variables and metrics (Machine Learning features) and their relative weights are used by the AI model to determine what actions to take or decisions to make to achieve the optimal outcome based upon the user’s intent.

Going back to our GPS example, we need to consider the different variables and metrics that might be used to help optimize our journey. To help the AI-powered GPS optimize our journey, we might have two classes of variables (Figure 4):

Driver’s Driving Preferences which include variables and metrics considering:

  • Road Conditions considering current weather, road maintenance, surface conditions of the road, local events, etc.)
  • Route Safety considering neighborhood crime activity, location and severity of car accidents, time of day (twilight and dawn can be difficult driving times), seasonality (don’t want to be driving by a big mall right before Christmas or my birthday), etc.
  • Scenic sites considering scenic views, museums, historical sites, etc.

Vehicle Operating Efficiencies which include variables and metrics considering

  • Miles-to-Destination in seeking to minimize wear-and-tear on the car
  • Time-to-Destination estimate in seeking to get to your destination as fast as possible
Slide4-1

Figure 4: Variables, Weights, and AI-power GPS Optimization

Note, the features in Figure 4 could themselves be driven by an AI Utility Function in order to determine their optimal outcomes. For example,

  • Road Conditions could be a function of current weather, road maintenance, reported debris on the road, reported potholes, black ice conditions, surface conditions, icy weather forecast, etc.
  • Route Safety could be a function of neighborhood crime, history of car accidents, time of data, holiday traffic patterns, weather conditions, traffic lights, stop signs, deer crossings, railroad crossings, etc.
  • Scenic Sites could be a function scenic views, museums, art galleries, opera houses, birth places, historical crime sites, historical buildings, places of interest, statues, etc.

But what if our intent isn’t to get from Point A to Point B as quickly as possible? What if your driving intentions were different than just getting you from Point A to Point B as quickly as possible?  What if you had different intentions such as get to the destination on the least expensive path (considering tolls), or on the safest route (considering neighborhood crimes like carjacking), or on the most scenic route (considering picturesque viewing options or historical sites of interest), or the most comfortable route (based upon the conditions of the roads)?

The weights that we assign (and which the AI model will seek to continuously finetune) would change dramatically based upon our driving intent.  For example (Figure 5):

  • Option A: Minimize Time From Point A to Point B
  • Option B: Provide safest Route From Point A to Point B
  • Option C: Provide Scenic or Historic Site Seeking
Slide5

Figure 5: Determining Variable Weights Depending Upon Driver Intent

Incorporating the driver’s intent into the AI-driven GPS map would yield different route recommendations and recommended changes or detours along that route as conditions changed.  For example, Route C in Figure 5 – the Scenic and Historical Sites option – would take me to some scenic sites such as the Vermeer Windmill & Scholte House in Pella, the Louis Sullivan Jewel Box Bank in Grinnell, the John Wayne Memorial in Brooklyn, and the Amana Colonies (Figure 6).

Slide6-1

Figure 6: Different Recommended Routes Based Upon Intent

Intent Powers AI Effectiveness Summary

Today’s AI-driven maps are great in getting us from Point A to Point B as quickly as possible.  But what if that’s not our intent?

Creating AI models that deliver relevant and responsible recommendations, decisions, and actions requires understanding the user’s intent and ensuring that the variables, metrics, and weights to support that intent are properly reflected in the AI Utility Function.

You see, there is really nothing magical about AI. AI it will constantly seek to optimize and refine its operations based upon the variables and metrics – and their associated weights – as defined by the AI Utility Function.  It’s just that AI will constantly seek to optimize and refine at a speed billions of times faster than the human mind. 

So, if we don’t properly and holistically define the variables and metrics that comprise our AI Utility Function based upon the user’s intent, then the probability that the AI model can go bad is very high…and it could go bad billions of times faster than the human mind could monitor, comprehend, manage, and correct.