Multi-agents learning and mind: An evolving Minsky’s Society
“without having designed a device yourself, you may very well make completely the wrong functional decomposition by simply observing its behaviour”, Rodney A. Brooks
The term `agent' plays a major role in the recent developments in AI and computer science, and it is used in many working in different areas.
An agent is generally identified as a computational entity, which operates in an environment with a behaviour that partially depends on its experience.
An agent is therefore composed of four basic modules, a monitoring module, an actions module, an information base, and some reasoning engine.
It is normally assumed that sensor and motor components enable an agent to interact with its environment (e.g., by carrying out an action or by exchanging data with other agents). What is called the agent’s information base contains some information the agent has, about environmental regularities and possibly other agents'. The reasoning engine represents the “rules” that permit to implement the agent’s policy in the environment. See the fundamental work by Minsky [1], reference [2] for an updated description of most recent advances. The original papers by Brooks represent alsor a very interesting discussion of the basic ideas [3, 4, 5].
There exist different “interpretations” of agents, which give rise to different sets of minimal properties an agent should have. We ignore here the more anthropomorphic vision, where properties like belief, intention, desire, and so on, are attributed to the agents. According to the weak notion, we adopt here, an agent is characterized by the following properties:
. Autonomy,
. Reactivity,
. Proactiveness.
Three classes of mechanisms which make multiagent learning different from single agent learning are commonly identified:
Multiplication,
Division
Interaction.
Multiplication mechanism can be characterised by the fact that each individual agent pursues its own learning task without taking care of the other agents' learning goals. The multiplication’s effects on learning are due essentially to the number of agents or their heterogeneity. This paradigm corresponds to the metaphor of the performance of group independent and different learners.
Division corresponds to the case where a learning task is divided among several agents. The division may be intended in many different ways, according to functionality aspects of the algorithm or to characteristics of the data to be processed in order to achieve learning effects. Individual agents act as `specialist', which implements subset of the actions needed by the overall learning process.
The division of the tasks reduces the computational load of each agent. Agents must interact in order to create a correct information flux between the different tasks and reach the target of learning. So we are lead to discuss to the next mechanism, interaction. While division implies a minimal form of interaction to implement the agents society global learning task, more advanced form of interaction can be considered. Actually agents interact during learning as well.
This kind of co-operation considerably improves the learning result in many situations. One of the major problems associated with complex interaction between agents is the slow down learning processes as well as the decision taking activities. The mechanism of mutual regulation requires that each agent has some knowledge of the world representation of the agents it interacts with (e.g. the generalization hierarchy that they use.) Other major problems associated with interacting multiple agents societies are associated with conflicts resolutions.
Consider an agent a. A given symbol X , coming from the environment or from another agent, will be interpreted on the basis of a set of examples Wa. Another agent b may connect X with a different set of examples Wb not overlapping or partially overlapping with Wa. As a consequence a and b must negotiate the situation taking into account both the formal aspects and the consequences of the results of the negotiation process. One can then be lead to introduce integrators or regulating super agents, which often alter the sense of the multi agents approach.
In a recent paper [6] Brooks et al. have put into evidence how the concept of adaptive, interacting agent can be still of fundamental importance for understanding how mammalian mind transforms the information it receives from the environment into a sequence of actions. This approach is based on a number of experimental evidences. Multiple internal representations characterise human mind. In references [7, 8, 9] one can find evidences of multiple internal representations, which do not have full access to one another. Experiments have also shown how different internal representations can be associated to different independent nervous inputs [10, 11].
Biological evolution is the basic framework to understand the organisation of a society of agents by which modelling mammalian mind. So doing some fundamental constraints can be pointed out that can support us in identifying the specific characteristics of such agents society:
1) Genetic constraints are represented by the modular doubling-specialisation mechanism.
2) Neuronal competition is another important mechanism, which, by its influence on the structural organisation of the nervous system, can indirectly affect the agents society organisation.
3) The evolutionary framework also make necessary to focus on the interaction between knowledge and monitoring modules, as their improvements turns out to have been strictly correlated.
4) The evolutionary aspects of environmental states of affairs (SOA) classification are also of fundamental importance to identify the global performance of an agents society capable of representing the mammalian mind. Evolutionary aspects of SOA’s classification can be easily understood in terms of the interaction between local (in time) survival capabilities safeguarding and the need of improvement on longer time scales.
Point (4) is of great importance and has not been investigated in detail.
In the following we discuss some simple ideal experiments that can help in focusing on the nature of the problems affecting point (4).
Assume that a certain external state of affairs implies a survival risk for the living system and that the already existing knowledge module is capable to identify it and produce a competent primitive reaction. Such a reaction represents a useful competence. Why should we call it primitive? This judgement refers necessarily to an external, omniscient observer, who could judge the reaction from the exterior. Such an observer could for example “observe” that the reaction is too expensive from the energetic point of view and would give rise in the future to negative consequence. In any case, the observer would conclude that, if the living beings had been able to take into proper account other information, it would have adopted a more appropriate choice. This is why the observer would have called the reaction “competent, but primitive”. This leads to the following interesting question: “why should the primitive being give up with its implicit judgement that was considered correct also by an omniscient external observer?” Of course we can translate the previous quite rhetoric question into a more pragmatic statement: there are no evolutionary reasons to modify or cancel the knowledge module that has produced a correct classification of a dangerous state of affairs, while there exist excellent evolutionary reasons to improve its reaction to such a state of affairs. The problem posed here, becomes even more serious if one considers the time that would normally be needed to acquire a good statistical knowledge about block formed by a vocabulary with larger cardinality. This means essentially that, to optimise survival together with its possible improving, the system must, in many senses, build the new knowledge on the base of old one.
Consider the simple case where the genetic evolution makes available a new hardware module to the knowledge system of a simple evolving system. In principle at least, the system has acquired a higher computing capability. As survival remains essential, the strategy of re-learning the whole environment taking full advantage of the new information crunching potential would lead to the extinction of the system. During the
re-learning phase. A simple doubling of the competences would represent, in a condition of relative scarcity of computing resources, would give only a small redundancy advantage (this will not be the case once scarcity shall disappear). The best strategy will therefore be that of implementing a support activity of the new module without giving up all the knowledge stored in the pre-existing one. The previous knowledge will therefore represent the basis on which the new one will be built. The interaction between survival needs and the in time evolution of the hardware creates a strong constraint. Such a constraint is responsible for the overwhelming complexity of any interpretation of knowledge structure of a living system, in terms of its functional analysis at a fixed time of its evolution. The principal reason for that apparent gigantic complexity is due to the fact that the most efficient use of new available hardware modules is defined by the local in-time needs of the knowledge system. So, spatial localisation of knowledge function will be adopted whenever satisfying to some advantage at a given point of the evolution while , spatially random, structures will be adopted when they will represent a good solution.
References
[1] Minsky M., (1985), The society of Mind, Simon & Shuster, New York
[2] Weiss G. ed. (1999), Multiagent Systems, a modern approach to distributed artificial intelligence, The MIT Press, Cambridge
[3] Brooks R. A., Intelligence without representation, Artificial Intelligence 47 (1991), 139--159.
[4] Brooks R. A., (1991), Intelligence Without Reason, Proc. 12th International Joint Conference on Artificial Intelligence pp:569-595
[5] Brooks R. A., (1990), Also Elephants don’t play chess, in P. Maes, editor: Designing Autonomous Agents 5-15 MIT press,
[6] Brooks R. A., Breazeal C., Irie R., Kemp C. C., Marjanovic M., Scassellati B., Williamson M. M., (1999), Alternative Essences of Intelligence
[7] Gazzaniga, M. S. & LeDoux, J. E. (1978), The Integrated Mind, Plenum Press, New York.
[8] Weiskrantz, L. (1986), Blindsight: A Case Study and Implications, Clarendon Press, Oxford.
[9] Rensink, R., O'Regan, J. & Clark, J. (1997), `To See or Not to See: The Need for Attention to Perceive Changes in Scenes', Psychological Science 8, pp:368-373.
[10] [LeDoux J. (1996) "The Emotional Brain", Touchstone Book, New York
[11] J. S. Morris, A. Öhman, and R. J. Dolan, (1999), A subcortical pathway to the right amygdala mediating "unseen" fear, Vol. 96, Issue 4, pp:1680-1685,