The Idea In 60 Seconds :

  • Emergent Properties are something an AI system knows that you haven’t taught it.
  • How can that happen?
  • There is a structure to information.
  • In fact, there’s a lot more structure to information than we’re aware of.
  • AI systems decipher this structure mathematically, across very large datasets.
  • Since computers are much better at maths than humans, AI can sometimes surprise you with its capabilities.
  • The results are valuable with commercial applications including healthcare and predictive analytics.
  • Which, unfortunately, creates problems to do with how we control AI systems which demonstrate emergent qualities?

AI And The Importance of Information Structure

I think the key concept I’ve found so far in all of the AI research I’ve done is structure.

If you think about it, there is structure to almost everything, whether we see it or not. Since much of our Internet Bot research for the new AI tools and websites we’re building has been based around Large Language Models (LLMs), I’ll start with the structure of language.

If I showed you an English word, told you it began with ‘Q’ and asked you what the next letter was, you’d likely say ‘u’. We all know the structure of words beginning with ‘Q’ in English. Similarly, If I asked you the minimum numbers of vowels in a word, you’d tell me ‘one’ and then tell me the exceptions. If I told you a story had a beginning and middle and a ? You’d say ‘end.’ Each of these is an example of structure which you know at some level. You might have taken those rules for granted and not even thought of them as structure. They are and much of the language you use is subject to these structures whether we are aware of those structures or not.

There is structure to everything – including music.

There is more structure than we know to almost everything we look at. Figuring out Emergent properties involves considering this.

Music has structure. Images have structure. Videos have structure. When you ask Chat GPT to tell you a story about a teddy bear as a Haiku, it starts with structure. It applies the structure of a story and it applies the structure of a Haiku and there’s your result. Generative AI image apps take the structure of typed sentences and apply them to the structure of images. And so on.

Where Does the Structure Come From & How Does A Computer ‘Know’ What it ‘Knows’ ?

AI systems capture structure in the vector database of an AI Product. Vector databases store data in what’s called a Tensor.

A tensor is an ‘n’ dimensional matrix. I think of it a bit like a brain. We encode thoughts and memories in neurons in 3 dimensional space in our brains. Groups of neurons fire together to create thoughts and those neurons actually exist, physically, in a 3 dimensional model in our heads, creating a structure.

Sticking with the LLM I was discussing, words are encoded for the LLM using ‘Embeddings’. The relationships between the words are stored not in 3 dimensions but in many, many dimensions.

Embeddings Store the Structure in Vector Databases

To take some concept with structure and turn it in to the sort of thing a vector database can store, embeddings are created. OpenAI has a tool which does this although others are available.

Embeddings take each ‘thing’ (concept, word, idea) and attribute mathematical values to it which separate it from other, similar ‘things’. The closer the ‘thing’ is to another ‘thing’, mathematically, the closer those ‘things’ are mathematically in the Vector Database. This is where the structure is established from which Emergent Properties grow.

Since it’s maths, computers are much better and faster at doing this we are. Open AI’s embeddings have 1500 coordinates assigned to them. Think of all that structure being captured. Think of all the ways Vector Databases store the structure of concepts that we take for granted or know intuitively. Now, think of all the structure surrounding us that we’ve never noticed.

How All That Structure Leads To ‘Emergent Qualities’

So much structure can be captured by this process of producing embeddings that the AI model you’re working with can take what seem like incredible steps you weren’t anticipating.

You might have heard, when Google taught its AI 240 languages and how to translate between them, it figured out for itself how to translate Bengali. They didn’t have to teach it, it knew enough about the structure of translation and the structure of languages to figure it out. This an emergent property.

Many Humans Have the Same Ability

Humans do it all the time. Gladwell wrote a book called ‘Blink’ in which he described the ability of human experts to accurately predict an outcome (whether art was fake, whether a tennis player was about to double fault) without much information. They just ‘knew’. They’d seen so many examples of right and wrong in their field that subconsciously, they had internalized the structure of the circumstances preceding the event and, as a result, could predict what was going to happen next. You’ve probably felt an intuition yourself which turned out to be accurate.

Consciousness May Also Be An Emergent Property

Some speculate hat human (and other animal) consciousness is an emergent property. We don’t know what causes consciousness yet. My guess is that the reason we don’t know what causes it is that we’re blind to some of the structural relationships between the elements we see in the mind / brain (and perhaps some that we don’t.)

So What?

These emergent properties are useful things. They can be used in a variety of ways to make money including :

  • Predictive Analytics: Identifying patterns and correlations in large datasets that humans might overlook. Useful in finance for market trend analysis, in healthcare for predicting disease outbreaks or patient outcomes, and in retail for consumer behavior prediction.
  • Autonomous Systems: In robotics and autonomous vehicles, emergent properties can enhance decision-making capabilities in complex, unpredictable environments.
  • Personalization in Technology: AI can use emergent properties to tailor experiences and recommendations to individual users e.g. Targetted Advertising.
  • Creative Arts: In music, visual arts, and literature, AI can exhibit emergent creativity, generating new compositions, artworks, or literary pieces that may not have been explicitly programmed. Every AI is a borrower and a mimic. (Link to https://quotefancy.com/quote/894488/Ralph-Waldo-Emerson-Every-man-is-a-borrower-and-a-mimic-life-is-theatrical-and-literature )

Possibly most importantly, Emergent Properties might ultimately help us with problem solving and Innovation in important areas like Climate Change. Its possible emergent properties will contribute to novel solutions in engineering, environmental science, and other fields. AI systems can discover new materials, optimize complex systems (like power grids or logistic networks), and propose innovative approaches to longstanding problems.

Its not all good. While emergent properties can be incredibly useful, they also pose challenges. If the ‘rule’ about a relationship exists only in maths a computer understands, how can we know if it will remain good at predicting results? How can we control a situation without turning control over to an AI system. As AI continues to evolve, the management and ethical consideration of these emergent properties can only become increasingly important.