Engineering Computational Emotion - A Reference Model for Emotion in Artificial Systems

Emotion is generally argued to be an influence on the behavior of life systems, largely concerning flexibility and adaptivity. The way in which life systems acts in response to a particular situations of the environment, has revealed the decisive and crucial importance of this feature in the success of behaviors. And this source of inspiration has influenced the way of thinking artificial systems. During the last decades, artificial systems have undergone such an evolution that each day more are integrated in our daily life. They have become greater in complexity, and the subsequent effects are related to an increased demand of systems that ensure resilience, robustness, availability, security or safety among others. All of them questions that raise quite a fundamental challenges in control design. This thesis has been developed under the framework of the Autonomous System project, a.k.a the ASys-Project. Short-term objectives of immediate application are focused on to design improved systems, and the approaching of intelligence in control strategies. Besides this, long-term objectives underlying ASys-Project concentrate on high order capabilities such as cognition, awareness and autonomy. This thesis is placed within the general fields of Engineery and Emotion science, and provides a theoretical foundation for engineering and designing computational emotion for artificial systems. The starting question that has grounded this thesis aims the problem of emotion--based autonomy. And how to feedback systems with valuable meaning has conformed the general objective. Both the starting question and the general objective, have underlaid the study of emotion, the influence on systems behavior, the key foundations that justify this feature in life systems, how emotion is integrated within the normal operation, and how this entire problem of emotion can be explained in artificial systems. By assuming essential differences concerning structure, purpose and operation between life and artificial systems, the essential motivation has been the exploration of what emotion solves in nature to afterwards analyze analogies for man--made systems. This work provides a reference model in which a collection of entities, relationships, models, functions and informational artifacts, are all interacting to provide the system with non-explicit knowledge under the form of emotion-like relevances. This solution aims to provide a reference model under which to design solutions for emotional operation, but related to the real needs of artificial systems. The proposal consists of a multi-purpose architecture that implement two broad modules in order to attend: (a) the range of processes related to the environment affectation, and (b) the range or processes related to the emotion perception-like and the higher levels of reasoning. This has required an intense and critical analysis beyond the state of the art around the most relevant theories of emotion and technical systems, in order to obtain the required support for those foundations that sustain each model. The problem has been interpreted and is described on the basis of AGSys, an agent assumed with the minimum rationality as to provide the capability to perform emotional assessment. AGSys is a conceptualization of a Model-based Cognitive agent that embodies an inner agent ESys, the responsible of performing the emotional operation inside of AGSys. The solution consists of multiple computational modules working federated, and aimed at conforming a mutual feedback loop between AGSys and ESys. Throughout this solution, the environment and the effects that might influence over the system are described as different problems. While AGSys operates as a common system within the external environment, ESys is designed to operate within a conceptualized inner environment. And this inner environment is built on the basis of those relevances that might occur inside of AGSys in the interaction with the external environment. This allows for a high-quality separate reasoning concerning mission goals defined in AGSys, and emotional goals defined in ESys. This way, it is provided a possible path for high-level reasoning under the influence of goals congruence. High-level reasoning model uses knowledge about emotional goals stability, letting this way new directions in which mission goals might be assessed under the situational state of this stability. This high-level reasoning is grounded by the work of MEP, a model of emotion perception that is thought as an analogy of a well-known theory in emotion science. The work of this model is described under the operation of a recursive-like process labeled as R-Loop, together with a system of emotional goals that are assumed as individual agents. This way, AGSys integrates knowledge that concerns the relation between a perceived object, and the effect which this perception induces on the situational state of the emotional goals. This knowledge enables a high-order system of information that provides the sustain for a high-level reasoning. The extent to which this reasoning might be approached is just delineated and assumed as future work. This thesis has been studied beyond a long range of fields of knowledge. This knowledge can be structured into two main objectives: (a) the fields of psychology, cognitive science, neurology and biological sciences in order to obtain understanding concerning the problem of the emotional phenomena, and (b) a large amount of computer science branches such as Autonomic Computing (AC), Self-adaptive software, Self-X systems, Model Integrated Computing (MIC) or the paradigm of models@runtime among others, in order to obtain knowledge about tools for designing each part of the solution. The final approach has been mainly performed on the basis of the entire acquired knowledge, and described under the fields of Artificial Intelligence, Model-Based Systems (MBS), and additional mathematical formalizations to provide punctual understanding in those cases that it has been required. This approach describes a reference model to feedback systems with valuable meaning, allowing for reasoning with regard to (a) the relationship between the environment and the relevance of the effects on the system, and (b) dynamical evaluations concerning the inner situational state of the system as a result of those effects. And this reasoning provides a framework of distinguishable states of AGSys derived from its own circumstances, that can be assumed as artificial emotion.

Desenvolvimento artificial autônomo de um grafo sensório-motor auto-organizável.

A teoria de Jean Piaget sobre o desenvolvimento da inteligência tem sido utilizada na área de inteligência computacional como inspiração para a proposição de modelos de agentes cognitivos. Embora os modelos propostos implementem aspectos básicos importantes da teoria de Piaget, como a estrutura do esquema cognitivo, não consideram o problema da fundamentação simbólica e, portanto, não se preocupam com os aspectos da teoria que levam à aquisição autônoma da semântica básica para a organização cognitiva do mundo externo, como é o caso da aquisição da noção de objeto. Neste trabalho apresentamos um modelo computacional de esquema cognitivo inspirado na teoria de Piaget sobre a inteligência sensório-motora que se desenvolve autonomamente construindo mecanismos por meio de princípios computacionais pautados pelo problema da fundamentação simbólica. O modelo de esquema proposto tem como base a classificação de situações sensório-motoras utilizadas para a percepção, captação e armazenamento das relações causais determiníscas de menor granularidade. Estas causalidades são então expandidas espaço-temporalmente por estruturas mais complexas que se utilizam das anteriores e que também são projetadas de forma a possibilitar que outras estruturas computacionais autônomas mais complexas se utilizem delas. O modelo proposto é implementado por uma rede neural artificial feed-forward cujos elementos da camada de saída se auto-organizam para gerar um grafo sensóriomotor objetivado. Alguns mecanismos computacionais já existentes na área de inteligência computacional foram modificados para se enquadrarem aos paradigmas de semântica nula e do desenvolvimento mental autônomo, tomados como base para lidar com o problema da fundamentação simbólica. O grafo sensório-motor auto-organizável que implementa um modelo de esquema inspirado na teoria de Piaget proposto neste trabalho, conjuntamente com os princípios computacionais utilizados para sua concepção caminha na direção da busca pelo desenvolvimento cognitivo artificial autônomo da noção de objeto.

Augmenting web service discovery by cognitive semantics and abduction

We argue that web service discovery technology should help the user navigate a complex problem space by providing suggestions for services which they may not be able to formulate themselves as (s)he lacks the epistemic resources to do so. Free text documents in service environments provide an untapped source of information for augmenting the epistemic state of the user and hence their ability to search effectively for services. A quantitative approach to semantic knowledge representation is adopted in the form of semantic space models computed from these free text documents. Knowledge of the user’s agenda is promoted by associational inferences computed from the semantic space. The inferences are suggestive and aim to promote human abductive reasoning to guide the user from fuzzy search goals into a better understanding of the problem space surrounding the given agenda. Experimental results are discussed based on a complex and realistic planning activity.

Agent-based model of passenger flows in airport terminals

Passenger flow studies in airport terminals have shown consistent statistical relationships between airport spatial layout and pedestrian movement, facilitating prediction of movement from terminal designs. However, these studies are done at an aggregate level and do not incorporate how individual passengers make decisions at a microscopic level. Therefore, they do not explain the formation of complex movement flows. In addition, existing models mostly focus on standard airport processing procedures such as immigration and security, but seldom consider discretionary activities of passengers, and thus are not able to truly describe the full range of passenger flows within airport terminals. As the route-choice decision-making of passengers involves many uncertain factors within the airport terminals, the mechanisms to fulfill the capacity of managing the route-choice have proven difficult to acquire and quantify. Could the study of cognitive factors of passengers (i.e. human mental preferences of deciding which on-airport facility to use) be useful to tackle these issues? Assuming the movement in virtual simulated environments can be analogous to movement in real environments, passenger behaviour dynamics can be similar to those generated in virtual experiments. Three levels of dynamics have been devised for motion control: the localised field, tactical level, and strategic level. A localised field refers to basic motion capabilities, such as walking speed, direction and avoidance of obstacles. The other two fields represent cognitive route-choice decision-making. This research views passenger flow problems via a "bottom-up approach", regarding individual passengers as independent intelligent agents who can behave autonomously and are able to interact with others and the ambient environment. In this regard, passenger flow formation becomes an emergent phenomenon of large numbers of passengers interacting with others. In the thesis, first, the passenger flow in airport terminals was investigated. Discretionary activities of passengers were integrated with standard processing procedures in the research. The localised field for passenger motion dynamics was constructed by a devised force-based model. Next, advanced traits of passengers (such as their desire to shop, their comfort with technology and their willingness to ask for assistance) were formulated to facilitate tactical route-choice decision-making. The traits consist of quantified measures of mental preferences of passengers when they travel through airport terminals. Each category of the traits indicates a decision which passengers may take. They were inferred through a Bayesian network model by analysing the probabilities based on currently available data. Route-choice decision-making was finalised by calculating corresponding utility results based on those probabilities observed. Three sorts of simulation outcomes were generated: namely, queuing length before checkpoints, average dwell time of passengers at service facilities, and instantaneous space utilisation. Queuing length reflects the number of passengers who are in a queue. Long queues no doubt cause significant delay in processing procedures. The dwell time of each passenger agent at the service facilities were recorded. The overall dwell time of passenger agents at typical facility areas were analysed so as to demonstrate portions of utilisation in the temporal aspect. For the spatial aspect, the number of passenger agents who were dwelling within specific terminal areas can be used to estimate service rates. All outcomes demonstrated specific results by typical simulated passenger flows. They directly reflect terminal capacity. The simulation results strongly suggest that integrating discretionary activities of passengers makes the passenger flows more intuitive, observing probabilities of mental preferences by inferring advanced traits make up an approach capable of carrying out tactical route-choice decision-making. On the whole, the research studied passenger flows in airport terminals by an agent-based model, which investigated individual characteristics of passengers and their impact on psychological route-choice decisions of passengers. Finally, intuitive passenger flows in airport terminals were able to be realised in simulation.

The effect of autonomy transparency in human-robot interactions : a preliminary study on operator cognitive workload and situation awareness in multiple heterogeneous UAV management

The autonomous capabilities in collaborative unmanned aircraft systems are growing rapidly. Without appropriate transparency, the effectiveness of the future multiple Unmanned Aerial Vehicle (UAV) management paradigm will be significantly limited by the human agent’s cognitive abilities; where the operator’s CognitiveWorkload (CW) and Situation Awareness (SA) will present as disproportionate. This proposes a challenge in evaluating the impact of robot autonomous capability feedback, allowing the human agent greater transparency into the robot’s autonomous status - in a supervisory role. This paper presents; the motivation, aim, related works, experiment theory, methodology, results and discussions, and the future work succeeding this preliminary study. The results in this paper illustrates that, with a greater transparency of a UAV’s autonomous capability, an overall improvement in the subjects’ cognitive abilities was evident, that is, with a confidence of 95%, the test subjects’ mean CW was demonstrated to have a statistically significant reduction, while their mean SA was demonstrated to have a significant increase.

Religion Explained? : A Philosophical Appraisal of the Cognitive Science of Religion

This study examines philosophically the main theories and methodological assumptions of the field known as the cognitive science of religion (CSR). The study makes a philosophically informed reconstruction of the methodological principles of the CSR, indicates problems with them, and examines possible solutions to these problems. The study focuses on several different CSR writers, namely, Scott Atran, Justin Barrett, Pascal Boyer and Dan Sperber. CSR theorising is done in the intersection between cognitive sciences, anthropology and evolutionary psychology. This multidisciplinary nature makes CSR a fertile ground for philosophical considerations coming from philosophy of psychology, philosophy of mind and philosophy of science. The study begins by spelling out the methodological assumptions and auxiliary theories of CSR writers by situating these theories and assumptions in the nexus of existing approaches to religion. The distinctive feature of CSR is its emphasis on information processing: CSR writers claim that contemporary cognitive sciences can inform anthropological theorising about the human mind and offer tools for producing causal explanations. Further, they claim to explain the prevalence and persistence of religion by cognitive systems that undergird religious thinking. I also examine the core theoretical contributions of the field focusing mainly on the (1) “minimally counter-intuitiveness hypothesis” and (2) the different ways in which supernatural agent representations activate our cognitive systems. Generally speaking, CSR writers argue for the naturalness of religion: religious ideas and practices are widespread and pervasive because human cognition operates in such a way that religious ideas are easy to acquire and transmit. The study raises two philosophical problems, namely, the “problem of scope” and the “problem of religious relevance”. The problem of scope is created by the insistence of several critics of the CSR that CSR explanations are mostly irrelevant for explaining religion. Most CSR writers themselves hold that cognitive explanations can answer most of our questions about religion. I argue that the problem of scope is created by differences in explanation-begging questions: the former group is interested in explaining different things than the latter group. I propose that we should not stick too rigidly to one set of methodological assumptions, but rather acknowledge that different assumptions might help us to answer different questions about religion. Instead of adhering to some robust metaphysics as some strongly naturalistic writers argue, we should adopt a pragmatic and explanatory pluralist approach which would allow different kinds of methodological presuppositions in the study of religion provided that they attempt to answer different kinds of why-questions, since religion appears to be a multi-faceted phenomenon that spans over a variety of fields of special sciences. The problem of religious relevance is created by the insistence of some writers that CSR theories show religious beliefs to be false or irrational, whereas others invoke CSR theories to defend certain religious ideas. The problem is interesting because it reveals the more general philosophical assumptions of those who make such interpretations. CSR theories can (and have been) interpreted in terms of three different philosophical frameworks: strict naturalism, broad naturalism and theism. I argue that CSR theories can be interpreted inside all three frameworks without doing violence to the theories and that these frameworks give different kinds of results regarding the religious relevance of CSR theories.

Nash equilibria in multi-agent motor interactions.

Social interactions in classic cognitive games like the ultimatum game or the prisoner's dilemma typically lead to Nash equilibria when multiple competitive decision makers with perfect knowledge select optimal strategies. However, in evolutionary game theory it has been shown that Nash equilibria can also arise as attractors in dynamical systems that can describe, for example, the population dynamics of microorganisms. Similar to such evolutionary dynamics, we find that Nash equilibria arise naturally in motor interactions in which players vie for control and try to minimize effort. When confronted with sensorimotor interaction tasks that correspond to the classical prisoner's dilemma and the rope-pulling game, two-player motor interactions led predominantly to Nash solutions. In contrast, when a single player took both roles, playing the sensorimotor game bimanually, cooperative solutions were found. Our methodology opens up a new avenue for the study of human motor interactions within a game theoretic framework, suggesting that the coupling of motor systems can lead to game theoretic solutions.

Improving effects of huperzine A on spatial working memory in aged monkeys and young adult monkeys with experimental cognitive impairment

Our previous studies demonstrated that huperzine A, a reversible and selective acetylcholinesterase inhibitor, exerts beneficial effects on memory deficits in various rodent models of amnesia. To extend the antiamnesic action of huperzine A to nonhuman primates, huperzine A was evaluated for its ability to reverse the deficits in spatial memory produced by scopolamine in young adult monkeys or those that are naturally occurring in aged monkeys using a delayed-response task. Scopolamine, a muscarinic receptor antagonist, dose dependently impaired performance with the highest dose (0.03 mg/kg, i.m.) producing a significant reduction in choice accuracy in young adult monkeys. The delayed performance changed from an average of 26.8/30 trials correct on saline control to an average of 20.2/30 trials correct after scopolamine administration. Huperzine A (0.01-0.1 mg/kg, i.m.) significantly reversed deficits induced by scopolamine in young adult monkeys on a delayed-response task; performance after an optimal dose (0.1 mg/kg) averaged 25.0/30 correct. In four aged monkeys, huperzine A (0.001-0.01 mg/kg, i.m.) significantly increased choice accuracy from 20.5/30 on saline control to 25.2/30 at the optimal dose (0.001 mg/kg for two monkeys and 0.01 mg/kg for the other two monkeys). The beneficial effects of huperzine A on delayed-response performance were long lasting; monkeys remained improved for about 24 h after a single injection of huperzine A. This study extended the findings that huperzine A improves the mnemonic performance requiring working memory in monkeys, and suggests that huperzine A may be a promising agent for clinical therapy of cognitive impairments in patients with Alzheimer's disease.

Implementation of cognitive mapping, spatial representation and wayfinding behaviours of people within evacuation modelling tools

Within the building evacuation context, wayfinding describes the process in which an individual located within an arbitrarily complex enclosure attempts to find a path which leads them to relative safety, usually the exterior of the enclosure. Within most evacuation modelling tools, wayfinding is completely ignored; agents are either assigned the shortest distance path or use a potential field to find the shortest path to the exits. In this paper a novel wayfinding technique that attempts to represent the manner in which people wayfind within structures is introduced and demonstrated through two examples. The first step is to encode the spatial information of the enclosure in terms of a graph. The second step is to apply search algorithms to the graph to find possible routes to the destination and assign a cost to the routes based on their personal route preferences such as "least time" or "least distance" or a combination of criteria. The third step is the route execution and refinement. In this step, the agent moves along the chosen route and reassesses the route at regular intervals and may decide to take an alternative path if the agent determines that an alternate route is more favourable e.g. initial path is highly congested or is blocked due to fire.

Embodied Autonomy in Digital Ecosystems: From Bio-Inspired Agents to Cognitive Systems

This paper proposes to promote autonomy in digital ecosystems so that it provides agents with information to improve the behavior of the digital ecosystem in terms of stability. This work proposes that, in digital ecosystems, autonomous agents can provide fundamental services and information. The final goal is to run the ecosystem, generate novel conditions and let agents exploit them. A set of evaluation measures must be defined as well. We want to provide an outline of some global indicators, such as heterogeneity and diversity, and establish relationships between agent behavior and these global indicators to fully understand interactions between agents, and to understand the dependence and autonomy relations that emerge between the interacting agents. Individual variations, interaction dependencies, and environmental factors are determinants of autonomy that would be considered. The paper concludes with a discussion of situations when autonomy is a milestone

Introspection on control-grounded capabilities. An agent inspired approach for control

Introspecció sobre la dinàmica dels agents té un important impacte en decisions individuals i cooperatives en entorns multi-agent. Introspecció, una habilitat cognitiva provinent de la metàfora "agent", permet que els agents siguin conscients de les seves capacitats per a realitzar correctament les tasques. Aquesta introspecció, principalment sobre capacitats relacionades amb la dinàmica, proporciona als agents un raonament adequat per a assolir compromisos segurs en sistemes cooperatius. Per a tal fi, les capacitats garanteixen una representació adequada i explícita de tal dinàmica. Aquest enfocament canvia i millora la manera com els agents poden coordinar-se per a portar a terme tasques i com gestionar les seves interaccions i compromisos en entorns cooperatius. L'enfocament s'ha comprovat en escenaris on la coordinació és important, beneficiosa i necessària. Els resultats i les conclusions són presentats ressaltant els avantatges de la introspecció en la millora del rendiment dels sistemes multi-agent en tasques coordinades i assignació de tasques.

Can social interaction skills be taught by a social agent? The role of a robotic mediator in autism therapy

Increasingly socially intelligent agents (software or robotic) are used in education, rehabilitation and therapy. This paper discusses the role of interactive, mobile robots as social mediators in the particular domain of autism therapy. This research is part of the project AURORA that studies how mobile robots can be used to teach children with autism basic interaction skills that are important in social interactions among humans. Results from a particular series of trials involving pairs of two children and a mobile robot are described. The results show that the scenario with pairs of children and a robot creates a very interesting social context which gives rise to a variety of different social and non-social interaction patterns, demonstrating the specific problems but also abilities of children with autism in social interactions. Future work will include a closer analysis of interactional structure in human-human and robot-human interaction. We outline a particular framework that we are investigating.

Conceptualizing change in the Portuguese footwear industry: regional voices and cognitive spaces of agency

Expanding national services sectors and global competition aggravate current and perceived future market pressures on traditional manufacturing industries. These perceptions of change have provoked a growing intensification of geo-political discourses on technological innovation and ‘learning’, and calls for competency in design among other professional skills. However, these political discourses on innovation and learning have paralleled public concerns with the apparent ‘growth pains’ from factory closures and subsequent increases in unemployment, and its debilitating social and economic implications for local and regional development. In this respect the following investigation sets out to conceptualize change through the complementary and differing perceptions of industry and regional actors’ experiences or narratives, linking these perceptions to their structure-determined spheres of agent-environment interactivity. It aims to determine whether agents’ differing perceptions of industry transformation can have a role in the legitimization of their interests in, and in sustaining their organizational influence over the process of industry-regional transformation. It argues that industry and regional agent perceptions are among the cognitive aspects of agent-environment interactivity that permeate agency. It stresses agents’ ability to reason and manipulate their work environments to preserve their self-regulating interests in, and task representative influence over the multi-jurisdictional space of industry-regional transformation. The contributions of this investigation suggest that agents’ varied perceptions of industry and regional change inform or compete for influence over the redirection of regional, industry and business strategies. This claim offers a greater appreciation for the reflexive and complex institutional dimensions of industry planning and development, and the political responsibility to socially just forms of regional development. It positions the outcomes of this investigation at the nexus of intensifying geo-political discourses on the efficiency and equity of territorial development in Europe.

CASCADE: An Agent Based Framework For Modeling The Dynamics Of Smart Electricity Systems

The Complex Adaptive Systems, Cognitive Agents and Distributed Energy (CASCADE) project is developing a framework based on Agent Based Modelling (ABM). The CASCADE Framework can be used both to gain policy and industry relevant insights into the smart grid concept itself and as a platform to design and test distributed ICT solutions for smart grid based business entities. ABM is used to capture the behaviors of diff erent social, economic and technical actors, which may be defi ned at various levels of abstraction. It is applied to understanding their interactions and can be adapted to include learning processes and emergent patterns. CASCADE models ‘prosumer’ agents (i.e., producers and/or consumers of energy) and ‘aggregator’ agents (e.g., traders of energy in both wholesale and retail markets) at various scales, from large generators and Energy Service Companies down to individual people and devices. The CASCADE Framework is formed of three main subdivisions that link models of electricity supply and demand, the electricity market and power fl ow. It can also model the variability of renewable energy generation caused by the weather, which is an important issue for grid balancing and the profi tability of energy suppliers. The development of CASCADE has already yielded some interesting early fi ndings, demonstrating that it is possible for a mediating agent (aggregator) to achieve stable demandfl attening across groups of domestic households fi tted with smart energy control and communication devices, where direct wholesale price signals had previously been found to produce characteristic complex system instability. In another example, it has demonstrated how large changes in supply mix can be caused even by small changes in demand profi le. Ongoing and planned refi nements to the Framework will support investigation of demand response at various scales, the integration of the power sector with transport and heat sectors, novel technology adoption and diffusion work, evolution of new smart grid business models, and complex power grid engineering and market interactions.

Agent-based hybrid framework for decision making on complex problems

Electronic commerce and the Internet have created demand for automated systems that can make complex decisions utilizing information from multiple sources. Because the information is uncertain, dynamic, distributed, and heterogeneous in nature, these systems require a great diversity of intelligent techniques including expert systems, fuzzy logic, neural networks, and genetic algorithms. However, in complex decision making, many different components or sub-tasks are involved, each of which requires different types of processing. Thus multiple such techniques are required resulting in systems called hybrid intelligent systems. That is, hybrid solutions are crucial for complex problem solving and decision making. There is a growing demand for these systems in many areas including financial investment planning, engineering design, medical diagnosis, and cognitive simulation. However, the design and development of these systems is difficult because they have a large number of parts or components that have many interactions. From a multi-agent perspective, agents in multi-agent systems (MAS) are autonomous and can engage in flexible, high-level interactions. MASs are good at complex, dynamic interactions. Thus a multi-agent perspective is suitable for modeling, design, and construction of hybrid intelligent systems. The aim of this thesis is to develop an agent-based framework for constructing hybrid intelligent systems which are mainly used for complex problem solving and decision making. Existing software development techniques (typically, object-oriented) are inadequate for modeling agent-based hybrid intelligent systems. There is a fundamental mismatch between the concepts used by object-oriented developers and the agent-oriented view. Although there are some agent-oriented methodologies such as the Gaia methodology, there is still no specifically tailored methodology available for analyzing and designing agent-based hybrid intelligent systems. To this end, a methodology is proposed, which is specifically tailored to the analysis and design of agent-based hybrid intelligent systems. The methodology consists of six models - role model, interaction model, agent model, skill model, knowledge model, and organizational model. This methodology differs from other agent-oriented methodologies in its skill and knowledge models. As good decisions and problem solutions are mainly based on adequate information, rich knowledge, and appropriate skills to use knowledge and information, these two models are of paramount importance in modeling complex problem solving and decision making. Follow the methodology, an agent-based framework for hybrid intelligent system construction used in complex problem solving and decision making was developed. The framework has several crucial characteristics that differentiate this research from others. Four important issues relating to the framework are also investigated. These cover the building of an ontology for financial investment, matchmaking in middle agents, reasoning in problem solving and decision making, and decision aggregation in MASs. The thesis demonstrates how to build a domain-specific ontology and how to access it in a MAS by building a financial ontology. It is argued that the practical performance of service provider agents has a significant impact on the matchmaking outcomes of middle agents. It is proposed to consider service provider agents' track records in matchmaking. A way to provide initial values for the track records of service provider agents is also suggested. The concept of ‘reasoning with multimedia information’ is introduced, and reasoning with still image information using symbolic projection theory is proposed. How to choose suitable aggregation operations is demonstrated through financial investment application and three approaches are proposed - the stationary agent approach, the token-passing approach, and the mobile agent approach to implementing decision aggregation in MASs. Based on the framework, a prototype was built and applied to financial investment planning. This prototype consists of one serving agent, one interface agent, one decision aggregation agent, one planning agent, four decision making agents, and five service provider agents. Experiments were conducted on the prototype. The experimental results show the framework is flexible, robust, and fully workable. All agents derived from the methodology exhibit their behaviors correctly as specified.