Home » Sector Topics » Energy Tech

Today’s Data Purgatory and a History Lesson from the Energy Industry

Today’s Data Purgatory and a History Lesson from the Energy Industry
Marathon Refinery in Anacortes, WA. Wikimedia Commons, 2008

Now I shall sing the second kingdom there where the soul of man is cleansed, made worthy to ascend to Heaven.

Dante Alighieri, The Divine Comedy, Purg. I.4–9

In its most recent (April 2022) Data and Analytics Trends report, Gartner made a number of good points in and around the subject of AI and how to improve it.

Their findings as I interpreted them from the Information Week article I read summarizing the report’s highlights came down to this:

  1. For the right models, you need the right data. For AI models, that’s even more the case. Rather than focusing so much attention on the models, though, companies need to change the way they collect, refine, use and manage data throughout the data lifecycle.
  2. To address this issue, leading companies are weaving together metadata-rich data fabrics to place the data in context and using AI to bolster their nascent connected data governance efforts.
  3. Minimize the influence of data cartels or hoarders and shift to a culture of data sharing.
  4. Broaden organization-wide data literacy and analytics relevance.

These are not new business imperatives. Before the tech media made AI a synonym for powerful things you could do with data, enterprises were talking about data fabrics, for example. Business executives have been talking about getting the right data since the dawn of computing in the 1940s.

So why haven’t more organizations already mobilized more guerrilla teams and retrained their workforces to create a new data culture and transform data architecture? 

Because they aren’t getting the vision. The vision is ephemeral. It’s representational. It’s abstracted. You can’t touch it. And the language that refers to it is full of jargon. 

That’s the reason most enterprises are still in the 21st century version of Dante’s Purgatory. Because they haven’t really left the 20th century. They’re thinking in vague 20th century terms and not taking full advantage of the compute, networking and storage that’s in their hands right now. They need concrete visions and a sense of the scope of the opportunities ahead of them to inspire them.

Freeing Up Future Imaginings with a 19th and 20th Century Example

It might help to create an analogy to make the vision and the opportunities more compelling. Let’s take an energy example. Humanity has used coal as fuel since the Stone Age. The Chinese have been smelting copper with coal since 1000 BC. Europeans have been mining, shipping and selling coal since the Romans were exploiting coal deposits in England and Wales in 200 AD.

However, though petroleum has a similar long history of use, the oil and gas industry began in earnest much later. Drilling the first commercial oil wells and building the first refineries beginning in the 1850s triggered the start of the industry, with both Europe and North America innovating simultaneously.

Now think about where the world’s oil industry stood in the 1850s and where it stands today. The global industry produced 1,400 barrels a day in 1860. In 1870, daily global oil production was 55,000 barrels a day.  Imagine wooden oil derricks, local oil distribution via horse drawn carts, and transport by sea via steamship.

Back then, the primary demand was for heating oil and kerosene. But in the 20th century, demand ramped up quickly. In 1920, 12 years after the introduction of Ford’s mass produced Model T, global oil production had reached 1.2 million barrels a day.

Today’s Data Purgatory and a History Lesson from the Energy Industry

US Energy Information Administration, other sources and estimates, 2000 – 2010

Worldwide oil supply peaked at 100 million barrels a day in 2019. The International Energy Agency then observed a downturn due to the outbreak of COVID, and estimates the next peak will be in 2026. Twenty different countries across five continents supply most of the world’s crude oil, but Russia and Saudi Arabia together have until recently accounted for roughly 30 percent of the world’s oil production.

Demand for Artificial Intelligence: Just Getting Started

The global market in 2021 for AI was $93.5 billion, according to GrandView Research.

In other words, demand for AI, even generously defined, is a drop in the bucket of $84 trillion overall demand for global goods and services. 

I’d argue we’re still in a nascent stage of AI, despite the huge volume of media verbiage and apparent activity in the tech sector. 

What will get us beyond the data equivalent of wooden oil derricks, horse drawn carts and steamships to pipeline networks and diverse, rich machine-readable information refined for a variety of purposes? Behaving more like grown ups when it comes to data, more like the energy and transportation industry pioneers who powered the industrial revolution.

So much purposeful planning and action needs to happen at the level of infrastructure and architecture over the next decades. One thing we shouldn’t do is downplay what has to be built and how different it has to be from application-centric, monolithic IT to be successful.