Home » Uncategorized

What’s missing from ChatGPT and other LLMs?

Recent developments in artificial intelligence remind me of the automotive industry in the late 19th and early 20th century. In that case, it took the industry several decades to commit to internal combustion engines. And while that picture was still unclear, there were over 250 different car manufacturers, some of whom were producing steam-powered cars. Electric cars were soon developed too but proved infeasible at that point. And every vehicle built was hand-assembled, more or less.

By the late 1920s, the shape of a more mature market had become evident. Ford’s assembly line, inspired by the systematic approach Swift took to butchering and meat packing, was so superior to the old guild-style manufacturing that assembly lines quickly took over.

Those carmakers who couldn’t adapt to assembly line processes simply went under. The number of carmakers declined to 44 by 1929. The Great Depression then forced even more makers to merge or go out of business.

Lots of affordable cars appeared, and buyers snapped them up, even during the Depression. Their utility was too obvious to ignore.

What took even longer to develop was the larger infrastructure market: scaling up oil and gasoline production and distribution for cars, paving roads, building bridges capable of supporting cars, gas and repair stations, and “motor hotels” or motels for those who all of a sudden wanted to take longer and longer driving trips. 

Not to mention creating dealership networks, puncture-resistant tires, and national and international road networks. It wasn’t until the 1950s that the US Interstate Highway System was born and funded.

The Timeline Comparison to Today’s Narrow AI

In terms of this timeline, AI, to my mind, is still in the auto industry equivalent of the 1880s. ChatGPT on the automotive industry timeline is akin to the first car with Karl Benz’s internal combustion engine–the Benz Patent Motorwagen. 

What’s missing from ChatGPT and other LLMs?

1886 Benz Patent Motorwagen (Wikimedia Commons)

The Motorwagen pointed the way forward, in some ways, but there wouldn’t be a vision of the larger, transformed transportation system for a while yet. And there was a serious issue that would come back to haunt us later on: a rigid, long-term commitment to the internal combustion engine that was, in retrospect, a fateful decision with a huge impact on carbon emissions levels. 

In retrospect, we should have given ourselves the flexibility to pivot to electric motors, electric mass transit, and then renewable energy as soon as we could. Electric motors worked, and we could have focused more resources on mobile and fixed battery technology (for mass transit) to boost storage capacity. And we could have refined and decentralized both batteries and nuclear power generation.

But we weren’t focused on energy efficiency or environmental concerns at that point. 

In retrospect, our failure to do better by the environment decades ago demonstrated that we ignored the need to make decisions factoring in impacts at an ecosystem level. 

We created a crisis of our own making. That’s the kind of crisis we definitely want to avoid when it comes to general rather than just narrow AI.

Today’s AI Motorwagen within the Missing Data-Centric Frame

In a nutshell, the AI that’s getting the most attention today is the equivalent of the three-wheel Benz Motorwagen: statistical machine learning in the form of neural networks and prompt interfaces. These add up to a form of natural language processing (NLP) or image processing and generation and a chatbot interface that together can help automate some recognition and transformation processes with the help of humans in the loop.

What doesn’t get attention is deterministic rule and reasoning capabilities that can complement what NLP does on the probabilistic side today. These are long-developed capabilities that need to be repurposed within a new, data-centric architecture so that they can be harnessed in conjunction with NLP and prompt interfaces.

There’s a symbiosis implied by such a data-centric architecture:

What’s missing from ChatGPT and other LLMs?

Back in 2017, John Launchbury of the Defense Advanced Research Projects Agency (DARPA) stepped back and described another view of AI symbiosis in terms of three AI waves. The third wave, he pointed out, blends the deterministic – or symbolic – first wave (the 1980s decision science wave, in other words) with the second wave of probabilistic neural nets.

What’s missing from ChatGPT and other LLMs?

Why We Aren’t in Wave III Yet – Tribalism

Unfortunately, tribalism often gets in the way when it comes to technological development. Tribalism is a big problem and has been for decades. Pedro Domingos, now a Computer Science Emeritus at the University of Washington, published a book in 2015 called the Master Algorithm that described machine learning efforts in terms of five tribes that didn’t work together. His assertion was that artificial general intelligence is needed to harness the collaborative power of those five tribes.

What’s missing from ChatGPT and other LLMs?

Domingos’ book gained some attention when it was published, but most of those involved in AI engineering today are either unaware of its insight about machine learning tribes or aren’t really thinking in terms of the larger AI picture.

A More Complete AI with a Contextualized Data Foundation

The chatbot buzz we’re hearing today doesn’t factor in the totality of all the elements needed to make AI trustworthy, reliable, or real-world responsible in any larger sense. We hear lots of complaints about the lack of these capabilities, and we’re seeing regulatory action ramp up as a result. 

Most of all, the buzz doesn’t seem to reflect much strategic interest in the data itself, which after all should provide the foundation for simulated AI worlds. How that data is created and managed will determine the effectiveness of AI governance. 

Managing data holistically is critical to truth-telling, verifiable AI. It is data that can be developed and stored in a humanistic way that values data sovereignty. The resulting contextualized data (which enables contextual computing of AI’s third wave) can be organic, efficient, and reusable. 

In fact, data and how it’s managed is key to AI at scale. Better data, Stanford computer science professor and entrepreneur Andrew Ng said years ago, beats better algorithms. But I suspect that even Andrew Ng doesn’t know how to develop or future-proof data for the role that it will be playing soon. That’s because he’s been immersed in a single tribe himself. 

Intertribal collaboration will help us create powerful, sustainable AI. As such, we have a political challenge ahead of us to win the minds of engineers, scientists, and users.