Conceptualization plays a crucial role in the development of Artificial General Intelligence (AGI), as it will enable that super-intelligence to understand and handle abstract information in a manner similar to how humans do, but...
What is a concept from a neuronal point of view?
A concept can be defined as a latent, abstract, and multi-modal representation that integrates sensory information from different sources: sight, hearing, smell, touch, encoded in a hyper-dimensional space, whose structure emerges from the activation pattern of multiple neurons.
This representation is compositional, in the sense that it is formed by hierarchical combinations of simpler representations, and relational, as it is topologically connected to similar or functionally related concepts, depending on the network's training.
Similar to linguistic embedding, the "distance" between concepts in latent space reflects their degree of similarity: the closer they are, the more they share in terms of meaning, structure, or function.
Many concepts are associated with a linguistic label, but this is not strictly necessary; there are many things we cannot name. On the other hand, linguistic labels — words — are themselves concepts that are also encoded in that neural network's latent vector space. It could not be otherwise, since language is processed in the same network.
In other words, our concepts are not purely linguistic. They are based on direct experience: what we see, hear, touch, smell, imagine, or experience in the real world. Language serves as a tool to refer to and share these concepts, but it does not define them by itself. The foundation of conceptual thinking is multi-modal, not merely verbal, and words are just a modality, a meta-modality.
Interestingly, we have studied language in great depth, to the point of creating an entire discipline dedicated to its meaning: semantics. However, we have paid very little attention to the process of conceptualization. In fact, we do not even have a word to name that discipline: we do not have a "conceptmatics" or equivalent. Everything that escapes the linguistic realm — that cannot be easily reduced to words or equations — seems to slip through our fingers like water, difficult to catch, difficult to understand and analyze. It is only with the advent of artificial neural networks that we have begun to understand conceptualization.
What is a concept from a neuronal point of view?
A concept can be defined as a latent, abstract, and multi-modal representation that integrates sensory information from different sources: sight, hearing, smell, touch, encoded in a hyper-dimensional space, whose structure emerges from the activation pattern of multiple neurons.
This representation is compositional, in the sense that it is formed by hierarchical combinations of simpler representations, and relational, as it is topologically connected to similar or functionally related concepts, depending on the network's training.
Similar to linguistic embedding, the "distance" between concepts in latent space reflects their degree of similarity: the closer they are, the more they share in terms of meaning, structure, or function.
Many concepts are associated with a linguistic label, but this is not strictly necessary; there are many things we cannot name. On the other hand, linguistic labels — words — are themselves concepts that are also encoded in that neural network's latent vector space. It could not be otherwise, since language is processed in the same network.
In other words, our concepts are not purely linguistic. They are based on direct experience: what we see, hear, touch, smell, imagine, or experience in the real world. Language serves as a tool to refer to and share these concepts, but it does not define them by itself. The foundation of conceptual thinking is multi-modal, not merely verbal, and words are just a modality, a meta-modality.
Interestingly, we have studied language in great depth, to the point of creating an entire discipline dedicated to its meaning: semantics. However, we have paid very little attention to the process of conceptualization. In fact, we do not even have a word to name that discipline: we do not have a "conceptmatics" or equivalent. Everything that escapes the linguistic realm — that cannot be easily reduced to words or equations — seems to slip through our fingers like water, difficult to catch, difficult to understand and analyze. It is only with the advent of artificial neural networks that we have begun to understand conceptualization.