Categories in Conceptual Spaces
Categorizing and identification of objects is a fundamental functionality we exercise regularly. In this post I'd like to describe the way Gardenfors proposed to address it. I will base it on "Conceptual Spaces", 2000, "Reasoning about Categories in Conceptual Spaces", 2001 of Gardenfors, his book "Geometry of Meaning", and add a bit of mine.
In his "Conceptual Spaces" paper, Gardenfors proposed a conceptual space theory that can assist both in explaining cognitive processes, but also provide guidelines for how to construct them. In his view, the symbolic approach for Conceptual Spaces is not sufficient to represent concepts as it is too coarse, whereas the connectionist one (aka unsupervised learnings) is too fine grained. Wordnet, a well known representative of the symbolic approach produces very limited descriptions of objects mainly defined by their relationships with predecessors, and ancestors. However, objects are multi-faceted entities having different attributes, and wider types of relationships, yet this graphical representation reduces them greatly and limiting the information we may learn about concepts. On the other hand, embedding representations stemming from connectionist approaches, are problematic for various other reasons. First, they require vast amounts of data for training. Second, they are vague in the type of relationships they produce, e.g. the degree of similarity is produced solely on the basis of textual context. Third, text is limited, we categorize concepts in ways that go beyond text, and often text would not inform us on "bananas are yellow", as it tend to add new information built on top of our common sense. Fourth, embedding representation model often struggles to continue producing new concepts, (after a model was trained), moreover, modifying the space as a result of expansion of a new category remains unclear. In this post, I will entertain the notion of combining both the symbolic, and the connectionist approaches enabling us to overcome the limitations I described.
Forming a Space
In Gardenfors' view, creating these spaces has to do with creating regions of concepts. These regions may change over time as we learn new concepts by partitioning our space. The partitioning would follow the Voronoi Tessellation process shown in the GIF below. The partitioning will happen gradually as more knowledge is obtained. Similarly, a child can't recognize the difference between a dog and a calf until a certain developmental stage, as both seem to look very similar, yet as she grows she starts paying attention to finer details that help her partitioning the previously united concept into two separated ones representing two different type of animals. Every concept region in this space has a prototypical entity (say "bird"), and more specific exemplars that are placed as a function of their distance/similarity to the "prototypical/ideal" entity ("pinguin" would be farther, and "crow" would be closer to the prototype "bird").
an example for partitioning the space through forming new concepts
When more knowledge is obtained about special types of "crows", concept regions can be formed around exemplars too, another reason I think they could be formed is when we create instances of that exemplar. The boundaries of an exemplar concept can overlap other regions. For instance, "lichen" is categorized as a hybrid of both a "fungi", and as "weed" within the same organism, so it may share both regions. However, as a side note, a multi-sense exemplar is not addressed in his theory. I think that homographs should be represented by different vectors, in regions that define them best. "Apple" vector for the company would be closer to "Microsoft" representation and "software company" concept region, and the fruit will be found in its appropriate fruit region.
The reason "Apple" homographs will be different is that each will be represented by a different vector, with different dimensions. Every concept and exemplar (or instance) are represented by a vector. Each vector in this space is described by "quality" dimensions or meaningful attributes to characterize a concept/exemplar. The type of dimensions vary depending on the vector's attributes, the number of dimensions may not be identical, and may grow as we learn more about the object. For instance, early in our childhood we tend to confuse hight with volume, thinking that the higher a liquid in a container, the more volume it consumes. This happens since we haven't yet developed the dimensions to characterize the difference of the two, but over time we learn to distinguish between both to describe different (while related) things. It also happens later on in our life when we encounter "mass" and "weight" and since this happens not through a grounded experience we add another dimension that is more abstract.
Each vector maintains different dimensions corresponding to different attributes
(they may correlate to some degree like height and volume)
Depending on the context, different dimensions within a vector, would be given rise to in the process of categorization. A simplistic example of exemplar retrieval can be when a child asking their mom "I want to play with the brown thing", perhaps looking for the concept region of "toy", and finding the toy by the color dimension (assuming they exist in space) would be sufficient, alternatively, the parent may ask to refine the query by adding another dimension to the search process or propose a list of entities that address or partially address this requirement. On a related note on weight assignment to vectors, Gardenfors describes that when children learn new concepts and develop gradually their concept space, they may not always assign sufficient weights to "important" dimensions' vector of a new object. Research shows that a child would make attempts at getting into a toy car (while the latter being too small to fit her), suggesting that the child learns the functionality of the toy through creating reasonable dimensions to characterize its affordances/functionalities, yet assigned a small weight to the size dimension.
A new concept therefore can be formed, not requiring massive amounts of co-occurrence in text, but via characterizing it through various sensors (generating a grounded experience), or in more advanced stages building on that towards characterizion of abstract concepts (a challenge on its own right). I think it is possible to see how to create an abstract concept, if it can be grounded in a more physical one or a previously learned one. The text "Apple is a software company selling digital devices" could lead to forming a concept of "software company", containing digital devices concept region (assuming its prior existence the space), and producing a vector to represent "Apple" close to the prototype of "software company", it could be created next to categories that are characterized by selling such as groceries (this may be reflected in their dimensions), but also with a constraint of digital devices within them. New concepts, in general, may not always be squeezed into a well known region, but form one in the outskirts of the space. If we learn how to characterize them, they will be more connected to other concepts we already know about, but also may form new dimensions we weren't familiar with before. In addition, in our experience as humans, there are also other dimensions we characterize concepts by, some entities in our life are characterized by emotions.
I know that this may seem hand-wavy, but I think this refreshing view opens the door to overcome shortcomings of previous approaches and delineates ways of learning concept spaces in a more natural and cognitively inspired approach. I personally would love to take part in developing this theory into practice. The papers and the book could inform you further on the various types of relationships, similarity metrics, and other aspects that were left out. Will be happy for your comments :-)