Cogprints: No conditions. Results ordered -Date, Title. 2018-01-17T14:23:59ZEPrintshttp://cogprints.org/images/sitelogo.gifhttp://cogprints.org/2014-08-24T21:08:27Z2015-04-20T11:40:52Zhttp://cogprints.org/id/eprint/9753This item is in the repository with the URL: http://cogprints.org/id/eprint/97532014-08-24T21:08:27ZA Quantitative Neural Coding Model of Sensory Memory
The coding mechanism of sensory memory on the neuron scale is one of the most
important questions in neuroscience. We have put forward a quantitative neural network model,
which is self-organized, self-similar, and self-adaptive, just like an ecosystem following
Darwin's theory. According to this model, neural coding is a “mult-to-one”mapping from
objects to neurons. And the whole cerebrum is a real-time statistical Turing Machine, with
powerful representing and learning ability. This model can reconcile some important disputations,
such as: temporal coding versus rate-based coding, grandmother cell versus population coding,
and decay theory versus interference theory. And it has also provided explanations for some key
questions such as memory consolidation, episodic memory, consciousness, and sentiment.
Philosophical significance is indicated at last.
PHD Peilei Liulpl1520@163.comProfessor Ting Wangtingwang1970@163.com2013-05-04T23:24:45Z2013-05-04T23:24:45Zhttp://cogprints.org/id/eprint/8966This item is in the repository with the URL: http://cogprints.org/id/eprint/89662013-05-04T23:24:45ZHow to Solve Classification and Regression Problems on High-Dimensional Data with a Supervised Extension of Slow Feature AnalysisSupervised learning from high-dimensional data, e.g., multimedia data, is a challenging task. We propose an extension of slow feature analysis (SFA) for supervised dimensionality reduction called graph-based SFA (GSFA). The algorithm extracts a label-predictive low-dimensional set of features that can be post-processed by typical supervised algorithms to generate the final label or class estimation. GSFA is trained with a so-called training graph, in which the vertices are the samples and the edges represent similarities of the corresponding labels. A new weighted SFA optimization problem is introduced, generalizing the notion of slowness from sequences of samples to such training graphs. We show that GSFA computes an optimal solution to this problem in the considered function space, and propose several types of training graphs. For classification, the most straightforward graph yields features equivalent to those of (nonlinear) Fisher discriminant analysis. Emphasis is on regression, where four different graphs were evaluated experimentally with a subproblem of face detection on photographs. The method proposed is promising particularly when linear models are insufficient, as well as when feature selection is difficult.Alberto-N. Escalante-B.alberto.escalante@ini.rub.deProf. Dr. Laurenz Wiskottlaurenz.wiskott@ini.rub.de2012-11-09T19:56:57Z2013-02-18T15:13:25Zhttp://cogprints.org/id/eprint/8715This item is in the repository with the URL: http://cogprints.org/id/eprint/87152012-11-09T19:56:57ZAn Automated Petri-Net Based Approach for Change Management in Distributed Telemedicine EnvironmentThe worldwide healthcare industry is facing a number of daunting challenges which are forcing healthcare systems worldwide to adapt and transform, and will ultimately completely redefine the way they do business and deliver care for patients. In this paper, we present a distributed telemedicine environement reaping from both the benefits of Service Oriented Approach (SOA) and the strong telecoms capabilities. We propose an automated approach to handle changes in a distributed telemedicine environement. A combined Petri nets model to handle changes and Reconfigurable Petri nets model to react to these changes are used to fulfill telemedicine functional and non functional requirements.S MtibaaM Tagina2012-04-25T12:29:38Z2012-04-25T12:29:38Zhttp://cogprints.org/id/eprint/8149This item is in the repository with the URL: http://cogprints.org/id/eprint/81492012-04-25T12:29:38ZCategory of Metabolic-Replication Systems
in Biology and MedicineMetabolic-repair models, or (M,R)-systems were introduced in Relational Biology by Robert Rosen. Subsequently, Rosen represented such (M,R)-systems (or simply MRs) in terms of categories of sets, deliberately selected without any structure other than the discrete topology of sets. Theoreticians of life’s origins postulated that Life on Earth has begun with the simplest possible organism, called the primordial. Mathematicians interested in biology attempted to answer this important question of the minimal living organism by defining the functional relations that would have made life possible in such a minimal system- a grandad and grandma of all living organisms on Earth.Prof.Dr. I.C. Baianuibaianu@illinois.edu2012-04-25T12:30:10Z2012-04-25T12:30:10Zhttp://cogprints.org/id/eprint/8144This item is in the repository with the URL: http://cogprints.org/id/eprint/81442012-04-25T12:30:10ZQuantum Genetics and Quantum Automata Models of Quantum-Molecular Selection Processes Involved in the Evolution of Organisms and Species Previous theoretical or general approaches (Rosen, 1960; Shcherbik and Buchatsky, 2007) to the problems of Quantum Genetics and Molecular Evolution are considered in this article from the point of view of Quantum Automata Theory first published by the author in 1971 (Baianu,1971a, b) , and further developed in several recent articles (Baianu, 1977, 1983, 1987, 2004, 2011).The representation of genomes and Interactome networks in categories of many-valued logic LMn –algebras that are naturally transformed during biological evolution, or evolve through interactions with the environment provide a new insight into the mechanisms of molecular evolution, as well as organismal evolution, in terms of sequences of quantum automata. Phenotypic changes are expressed only when certain environmentally-induced quantum-molecular changes are coupled with an internal re-structuring of major submodules of the genome and Interactome networks related to cell cycling and cell growth. Contrary to the commonly held view of `standard’ Darwinist models of evolution, the evolution of organisms and species occurs through coupled multi-molecular transformations induced not only by the environment but actually realized through internal re-organizations of genome and interactome networks. The biological, evolutionary processes involve certain epigenetic transformations that are responsible for phenotypic expression of the genome and Interactome transformations initiated at the quantum-molecular level. It can thus be said that only quantum genetics can provide correct explanations of evolutionary processes that are initiated at the quantum—multi-molecular levels and propagate to the higher levels of organismal and species evolution. Biological evolution should be therefore regarded as a multi-scale process which is initiated by underlying quantum (coupled) multi-molecular transformations of the genomic and interactomic networks, followed by specific phenotypic transformations at the level of organism and the variable biogroupoids associated with the evolution of species which are essential to the survival of the species. The theoretical framework introduced in this article also paves the way to a Quantitative Biology approach to biological evolution at the quantum-molecular, as well as at the organismal and species levels. This is quite a substantial modification of the `established’ modern Darwinist, and also of several so-called `molecular evolution’ theories.Professor I.C. Baianu, ibaianu@illinois.edu2011-12-16T00:58:48Z2011-12-16T00:58:48Zhttp://cogprints.org/id/eprint/7739This item is in the repository with the URL: http://cogprints.org/id/eprint/77392011-12-16T00:58:48ZNonlinear Models of Neural and Genetic Network Dynamics:
Natural Transformations of Łukasiewicz Logic LM-Algebras in a Łukasiewicz-Topos as Representations of Neural Network Development and Neoplastic Transformations
A categorical and Łukasiewicz-Topos framework for Algebraic Logic models of nonlinear dynamics in complex functional systems such as Neural Networks, Cell Genome and Interactome Networks is introduced. Łukasiewicz Algebraic Logic models of both neural and genetic networks and signaling pathways in cells are formulated in terms of nonlinear dynamic systems with n-state components that allow for the generalization of previous logical models of both genetic activities and neural networks. An algebraic formulation of variable next-state/transfer functions is extended to a Łukasiewicz Topos with an N-valued Łukasiewicz Algebraic Logic subobject classifier description that represents non-random and nonlinear network activities as well as their transformations in developmental processes and carcinogenesis.
Professor I.C. Baianuibaianu@illinois.edu2014-02-25T12:49:04Z2014-02-25T12:49:04Zhttp://cogprints.org/id/eprint/9187This item is in the repository with the URL: http://cogprints.org/id/eprint/91872014-02-25T12:49:04ZPattern Recognition by Hierarchical Temporal MemoryHierarchical Temporal Memory (HTM) is still largely unknown by the pattern recognition community and only a few studies have been published in the scientific literature. This paper reviews HTM architecture and related learning algorithms by using formal notation and pseudocode description. Novel approaches are then proposed to encode coincidence-group membership (fuzzy grouping) and to derive temporal groups (maxstab temporal clustering). Systematic experiments on three line-drawing datasets have been carried out to better understand HTM peculiarities and to extensively compare it against other well-know pattern recognition approaches. Our results prove the effectiveness of the new algorithms introduced and that HTM, even if still in its infancy, compares favorably with other existing technologies.Prof. Davide Maltonidavide.maltoni@unibo.it2011-02-16T19:49:50Z2011-03-11T08:57:50Zhttp://cogprints.org/id/eprint/7179This item is in the repository with the URL: http://cogprints.org/id/eprint/71792011-02-16T19:49:50ZUsing Feature Weights to Improve Performance of Neural NetworksDifferent features have different relevance to a particular learning problem. Some features are less relevant; while some very important. Instead of selecting the most relevant features using feature selection, an algorithm can be given this knowledge of feature importance based on expert opinion or prior learning. Learning can be faster and more accurate if learners take feature importance into account. Correlation aided Neural Networks (CANN) is presented which is such an algorithm. CANN treats feature importance as the correlation coefficient between the target attribute and the features. CANN modifies normal feed-forward Neural Network to fit both correlation values and training data. Empirical evaluation shows that CANN is faster and more accurate than applying the two step approach of feature selection and then using normal learning algorithms.Ridwan Al Iqbalstopofeger@yahoo.com2013-09-17T14:26:14Z2013-09-17T14:26:14Zhttp://cogprints.org/id/eprint/8980This item is in the repository with the URL: http://cogprints.org/id/eprint/89802013-09-17T14:26:14ZMind: meet network. Emergence of features in conceptual metaphor.As a human product, language reflects the psychological experience of man (Radden and Dirven, 2007). One model of language and human cognition in general is connectionism, by many linguists is regarded as mathematical and, therefore, too reductive. This opinion trend seems to be reversing, however, due to the fact that many cognitive researchers begin to appreciate one attribute of network models: feature emergence. In the course of a network simulation properties emerge that were neither inbuilt nor intended by its creators (Elman, 1998), in other words, the whole becomes more than just the sum of its parts. Insight is not only drawn from the network's output, but also the means that the network utilizes to arrive at the output.
It may seem obvious that the events of life should be meaningful for human beings, yet there is no widely accepted theory as to how do we derive that meaning. The most promising hypothesis regarding the question how the world is meaningful to us is that of embodied cognition (cf. Turner 2009), which postulates that the functions of the brain evolved so as to ‘understand’ the body, thus grounding the mind in an experiential foundation. Yet, the relationship between the body and the mind is far from perspicuous, as research insight is still intertwined with metaphors specific for the researcher’s methodology (Eliasmith 2003). It is the aim of this paper to investigate the conceptual metaphor in a manner that will provide some insight with regard to the role that objectification, as defined by Szwedek (2002), plays in human cognition and identify one possible consequence of embodied cognition.
If the mechanism for concept formation, or categorization of the world, resembles a network, it is reasonable to assume that evidence for this is to be sought in language. Let us then postulate the existence of a network mechanism for categorization and concept formation present in the human mind and initially developed to cope with the world directly accessible to the early human (i.e. tangible). Such a network would convert external inputs to form an internal, multi modal representation of a perceived object in the brain. The sheer amount of available information and the computational restrictions of the brain would force some sort of data compression, or a computational funnel. It has been shown that a visual perception network of this kind can learn to accurately label patterns (Elman, 1998). What is more, the compression of data facilitated the recognition of prototypes of a given pattern category rather than its peripheral representations, an emergent property that supports the prototype theory of the mental lexicon (cf. Radden and Dirven, 2007).
The present project proposes that, in the domain of cognition, the process of objectification, as defined by Szwedek (2002), would be an emergent property of such a system, or that if an abstract notion is computed by a neural network designed to cope with tangible concepts the data compression mechanism would require the notion to be conceptualized as an object to permit further processing. The notion of emergence of meaning from the operation of complex systems is recognised as an important process in a number of studies on metaphor comprehension. Feature emergence is said to occur when a non-salient feature of the target and the vehicle becomes highly salient in the metaphor (Utsumi 2005). Therefore, for example, should objectification emerge as a feature in the metaphor KNOWLEDGE IS A TREASURE, the metaphor would be characterised as having more
features of an object than either the target or vehicle alone. This paper focuses on providing a theoretical connectionist network based on the Elman-type network (Elman, 1998) as a model of concept formation where objectification would be an emergent feature. This is followed by a psychological experiment whereby the validity of this assumption is tested through a questionnaire where two groups of participants are asked to evaluate either metaphors or their components. The model proposes an underlying relation between the mechanism for concept formation and the omnipresence of conceptual metaphors, which are interpreted as resulting from the properties of the proposed network system.
Thus, an evolutionary neural mechanism is proposed for categorization of the world, that is able to cope with both concrete and abstract notions and the by-product of which are the abstract language-related phenomena, i.e. metaphors. The model presented in this paper aims at providing a unified account of how the various types of phenomena, objects, feelings etc. are categorized in the human mind, drawing on evidence from language.
References:
Szwedek, Aleksander. 2002. Objectification: From Object Perception To Metaphor Creation. In B. Lewandowska-Tomaszczyk and K. Turewicz (eds). Cognitive Linguistics To-day, 159-175. Frankfurt am Main: Peter Lang.
Radden, Günter and Dirven, René. 2007. Cognitive English Grammar. Amsterdam/ Philadelphia: John Benjamins Publishing Company
Eliasmith, Chris. 2003. Moving beyond metaphors: understanding the mind for what it is. Journal of Philosophy. C(10):493- 520.
Elman, J. L. et al. 1998. Rethinking innateness: A connectionist perspective on development. Cambridge, MA: MIT Press
Turner, Mark. 2009. Categorization of Time and Space Through Language. (Paper presented at the FOCUS2009 conference "Categorization of the world through language". Serock, 25-28 February 2009).
Utsumi, Akira. 2005. The role of feature emergence in metaphor appreciation, Metaphor and Symbol, 20(3), 151-172.Anna Jelecajelec@wa.amu.edu.plDorota Jaworska2011-12-16T00:58:12Z2011-12-16T00:58:12Zhttp://cogprints.org/id/eprint/7751This item is in the repository with the URL: http://cogprints.org/id/eprint/77512011-12-16T00:58:12ZŁukasiewicz-Moisil Many-Valued Logic Algebra of Highly-Complex SystemsA novel approach to self-organizing, highly-complex systems (HCS), such as living organisms and artificial intelligent systems (AIs), is presented which is relevant to Cognition, Medical Bioinformatics and Computational Neuroscience. Quantum Automata (QAs) were defined in our previous work as generalized, probabilistic automata with quantum state spaces (Baianu, 1971). Their next-state functions operate through transitions between quantum states defined by the quantum equations of motion in the Schroedinger representation, with both initial and boundary conditions in space-time. Such quantum automata operate with a quantum logic, or Q-logic, significantly different from either Boolean or Łukasiewicz many-valued logic. A new theorem is proposed which states that the category of quantum automata and automata--homomorphisms has both limits and colimits. Therefore, both categories of quantum automata and classical automata (sequential machines) are bicomplete. A second new theorem establishes that the standard automata category is a subcategory of the quantum automata category. The quantum automata category has a faithful representation in the category of Generalized (M,R)--Systems which are open, dynamic biosystem networks with defined biological relations that represent physiological functions of primordial organisms, single cells and higher organisms.Professor I.C. Baianuibaianu@illinois.eduProfessor George Georgescugeorgescu@funinf.cs.unibuc.roProfessor James F. Glazebrookjfglazebrook@eiu.edu2010-07-29T01:51:34Z2011-03-11T08:57:38Zhttp://cogprints.org/id/eprint/6880This item is in the repository with the URL: http://cogprints.org/id/eprint/68802010-07-29T01:51:34ZInformation Processing, Computation and CognitionComputation and information processing are among the most fundamental notions in cognitive science. They are also among the most imprecisely discussed. Many cognitive scientists take it for granted that cognition involves computation, information processing, or both – although others disagree vehemently. Yet different cognitive scientists use ‘computation’ and ‘information processing’ to mean different things, sometimes without realizing that they do. In addition, computation and information processing are surrounded by several myths; first and foremost, that they are the same thing. In this paper, we address this unsatisfactory state of affairs by presenting a general and theory-neutral account of computation and information processing. We also apply our framework by analyzing the relations between computation and information processing on one hand and classicism and connectionism/computational neuroscience on the other. We defend the relevance to cognitive science of both computation, at least in a generic sense, and information processing, in three important senses of the term. Our account advances several foundational debates in cognitive science by untangling some of their conceptual knots in a theory-neutral way. By leveling the playing field, we pave the way for the future resolution of the debates’ empirical aspects.Dr. Gualtiero Piccininipiccininig@umsl.eduDr. Andrea Scarantinoascarantino@gsu.edu2011-08-30T04:24:23Z2011-08-30T04:24:23Zhttp://cogprints.org/id/eprint/6801This item is in the repository with the URL: http://cogprints.org/id/eprint/68012011-08-30T04:24:23ZDual-hop transmissions with fixed-gain relays over Generalized-Gamma fading channelsIn this paper, a study on the end-to-end performance of dual-hop wireless communication systems equipped with fixed-gain relays and operating over Generalized-Gamma (GG) fading channels is presented. A novel closed form expression for the moments of the end-to-end signal-to-noise ratio (SNR) is derived. The average bit error probability for coherent and non-coherent modulation schemes as well as the end-to-end outage probability of the considered system are also studied. Extensive numerically evaluated and computer simulations results are presented that verify the accuracy of the proposed mathematical analysis.
Kostas P. PeppasAkil MansourGeorge S. Tombras2009-11-14T11:29:03Z2011-03-11T08:57:33Zhttp://cogprints.org/id/eprint/6720This item is in the repository with the URL: http://cogprints.org/id/eprint/67202009-11-14T11:29:03ZANN-based Innovative Segmentation Method for Handwritten text in AssameseArtificial Neural Network (ANN) s has widely been used for recognition of optically scanned character, which partially emulates human thinking in the domain of the Artificial Intelligence. But prior to recognition, it is necessary to segment the character from the text to sentences, words etc. Segmentation of words into individual letters has been one of the major problems in handwriting recognition. Despite several successful works all over the work, development of such tools in specific languages is still an ongoing process especially in the Indian context. This work explores the application of ANN as an aid to segmentation of handwritten characters in Assamese- an important language in the North Eastern part of India. The work explores the performance difference obtained in applying an ANN-based dynamic segmentation algorithm compared to projection- based static segmentation. The algorithm involves, first training of an ANN with individual handwritten characters recorded from different individuals. Handwritten sentences are separated out from text using a static segmentation method. From the segmented line, individual characters are separated out by first over segmenting the entire line. Each of the segments thus obtained, next, is fed to the trained ANN. The point of segmentation at which the ANN recognizes a segment or a combination of several segments to be similar to a handwritten character, a segmentation boundary for the character is assumed to exist and segmentation performed. The segmented character is next compared to the best available match and the segmentation boundary confirmed.Kaustubh BhattacharyyaKandarpa Kumar Sarma2009-11-14T11:32:17Z2011-03-11T08:57:32Zhttp://cogprints.org/id/eprint/6708This item is in the repository with the URL: http://cogprints.org/id/eprint/67082009-11-14T11:32:17ZNaive Bayes vs. Decision Trees vs. Neural Networks in the Classification of Training Web PagesWeb classification has been attempted through many different technologies. In this study we concentrate on the comparison of Neural Networks (NN), Naïve Bayes (NB) and Decision Tree (DT) classifiers for the automatic analysis and classification of attribute data from training course web pages. We introduce an enhanced NB classifier and run the same data sample through the DT and NN classifiers to determine the success rate of our classifier in the training courses domain. This research shows that our enhanced NB classifier not only outperforms the traditional NB classifier, but also performs similarly as good, if not better, than some more popular, rival techniques. This paper also shows that, overall, our NB classifier is the best choice for the training courses domain, achieving an impressive F-Measure value of over 97%, despite it being trained with fewer samples than any of the classification systems we have encountered.Daniela XHEMALIChristopher J. HINDERoger G. STONE2009-11-14T11:31:55Z2011-03-11T08:57:32Zhttp://cogprints.org/id/eprint/6711This item is in the repository with the URL: http://cogprints.org/id/eprint/67112009-11-14T11:31:55ZPassword Based a Generalize Robust Security System Design Using Neural NetworkAmong the various means of available resource protection including biometrics, password based system is most simple, user friendly, cost effective and commonly used. But this method having high sensitivity with attacks. Most of the advanced methods for authentication based on password encrypt the contents of password before storing or transmitting in physical domain. But all conventional cryptographic based encryption methods are having its own limitations, generally either in terms of complexity or in terms of efficiency. Multi-application usability of password today forcing users to have a proper memory aids. Which itself degrades the level of security. In this paper a method to exploit the artificial neural network to develop the more secure means of authentication, which is more efficient in providing the authentication, at the same time simple in design, has given. Apart from protection, a step toward perfect security has taken by adding the feature of intruder detection along with the protection system. This is possible by analysis of several logical parameters associated with the user activities. A new method of designing the security system centrally based on neural network with intrusion detection capability to handles the challenges available with present solutions, for any kind of resource has presented.Manoj Kumar Singh2010-01-30T03:40:56Z2011-03-11T08:57:35Zhttp://cogprints.org/id/eprint/6769This item is in the repository with the URL: http://cogprints.org/id/eprint/67692010-01-30T03:40:56ZHow Creative Should Creators be to Optimize the Evolution of Ideas? A Computer ModelThere are both benefits and drawbacks to creativity. In a social group it is not necessary for all members to be creative to benefit from creativity; some merely imitate or enjoy the fruits of others' creative efforts. What proportion should be creative? This paper outlines investigations of this question carried out using a computer model of cultural evolution referred to as EVOC (for EVOlution of Culture). EVOC is composed of neural network based agents that evolve fitter ideas for actions by (1) inventing new ideas through modification of existing ones, and (2) imitating neighbors' ideas. The ideal proportion with respect to fitness of ideas is found to depend on the level of creativity of the creative agents. For all levels or creativity, the diversity of ideas in a population is positively correlated with the ratio of creative agents.Stefan Leijnenstefanleijnen@gmail.comDr. Liane Gaboraliane.gabora@ubc.ca2008-05-11T02:41:13Z2011-03-11T08:57:07Zhttp://cogprints.org/id/eprint/6046This item is in the repository with the URL: http://cogprints.org/id/eprint/60462008-05-11T02:41:13ZIterative Application of the aiNET Algorithm in the Construction of a Radial Basis Function Neural NetworkThis paper presents some of the procedures adopted in the construction of a Radial Basis Function Neural Network by iteratively applying the aiNET, an Artificial Immune Systems Algorithm. These procedures have shown to be effective in terms of i) the free determination of centroids inspired by an immune heuristics; and ii) the achievement of appropriate minimal square errors after a number of iterations. Experimental and empirical results are compared aiming at confirming (or not) some hypotheses.Sandro Rautenbergsrautenberg@egc.ufsc.brLuciano Frontino de Medeiroslfm@egc.ufsc.brWagner Igarashiwigarashi@bol.com.brFernando Ostuni Gauthiergauthier@inf.ufsc.brRogério Cid Bastosrogerio@inf.ufsc.brJosé Leomar Todescotite@egc.ufsc.br2009-10-15T22:58:25Z2011-03-11T08:57:27Zhttp://cogprints.org/id/eprint/6638This item is in the repository with the URL: http://cogprints.org/id/eprint/66382009-10-15T22:58:25ZLogical openness in Cognitive Models It is here proposed an analysis of symbolic and sub-symbolic models for studying cognitive processes, centered on emergence and logical openness notions. The Theory of logical openness connects the Physics of system/environment relationships to the system informational structure. In this theory, cognitive models can be ordered according to a hierarchy of complexity depending on their logical openness degree, and their descriptive limits are correlated to Gödel-Turing Theorems on formal systems. The symbolic models with low logical openness describe cognition by means of semantics which fix the system/environment relationship (cognition in vitro), while the sub-symbolic ones with high logical openness tends to seize its evolutive dynamics (cognition in vivo). An observer is defined as a system with high logical openness. In conclusion, the characteristic processes of intrinsic emergence typical of “bio-logic” - emerging of new codes-require an alternative model to Turing-computation, the natural or bio-morphic computation, whose essential features we are going here to outline.Prof. Ignazio Licataignazio.licata@ejtp.info2009-02-13T01:12:02Z2011-03-11T08:57:19Zhttp://cogprints.org/id/eprint/6354This item is in the repository with the URL: http://cogprints.org/id/eprint/63542009-02-13T01:12:02ZPredictive Coding as a Model of Biased Competition in Visual AttentionAttention acts, through cortical feedback pathways, to enhance the response of cells encoding expected or predicted information. Such observations are inconsistent with the predictive coding theory of cortical function which proposes that feedback acts to suppress information predicted by higher-level cortical regions. Despite this discrepancy, this article demonstrates that the predictive coding model can be used to simulate a number of the effects of attention. This is achieved via a simple mathematical rearrangement of the predictive coding model, which allows it to be interpreted as a form of biased competition model. Nonlinear extensions to the model are proposed that enable it to explain a wider range of data.Michael W Spratling2009-02-13T01:12:14Z2011-03-11T08:57:19Zhttp://cogprints.org/id/eprint/6353This item is in the repository with the URL: http://cogprints.org/id/eprint/63532009-02-13T01:12:14ZReconciling Predictive Coding and Biased Competition Models of Cortical FunctionA simple variation of the standard biased competition model is shown, via some trivial mathematical manipulations, to be identical to predictive coding. Specifically, it is shown that a particular implementation of the biased competition model, in which nodes compete via inhibition that targets the inputs to a cortical region, is mathematically equivalent to the linear predictive coding model. This observation demonstrates that these two important and influential rival theories of cortical function are minor variations on the same underlying mathematical model.Michael W Spratling2008-10-22T01:17:40Z2011-03-11T08:57:13Zhttp://cogprints.org/id/eprint/6237This item is in the repository with the URL: http://cogprints.org/id/eprint/62372008-10-22T01:17:40ZEvolution of Prehension Ability in an Anthropomorphic Neurorobotic ArmIn this paper we show how a simulated anthropomorphic robotic arm controlled by an artificial neural network can develop effective reaching and grasping behaviour through a trial and error process in which the free parameters encode the control rules which regulate the fine-grained interaction between the robot and the environment and variations of the free parameters are retained or discarded on the basis of their effects at the level of the global behaviour exhibited by the robot situated in the environment. The obtained results demonstrate how the proposed methodology allows the robot to produce effective behaviours thanks to its ability to exploit the morphological properties of the robot’s body (i.e. its anthropomorphic shape, the elastic properties of its muscle-like actuators, and the compliance of its actuated joints) and the properties which arise from the physical interaction between the robot and the environment mediated by appropriate control rules.Prof Angelo Cangelosiacangelosi@plymouth.ac.ukGianluca MasseraStefano Nolfi2007-01-19Z2011-03-11T08:56:45Zhttp://cogprints.org/id/eprint/5357This item is in the repository with the URL: http://cogprints.org/id/eprint/53572007-01-19ZTowards Avatars with Artificial Minds: Role of Semantic Memoryhe first step towards creating avatars with human-like artificial minds is to give them human-like memory structures with an access to general knowledge about the world. This type of knowledge is stored in semantic memory. Although many approaches to modeling of semantic memories have been proposed they are not very useful in real life applications because they lack knowledge comparable to the common sense that humans have, and they cannot be implemented in a computationally efficient way. The most drastic simplification of semantic memory leading to the simplest knowledge representation that is sufficient for many applications is based on the Concept Description Vectors (CDVs) that store, for each concept, an information whether a given property is applicable to this concept or not. Unfortunately even such simple information about real objects or concepts is not available. Experiments with automatic creation of concept description vectors from various sources, including ontologies, dictionaries, encyclopedias and unstructured text sources are described. Haptek-based talking head that has an access to this memory has been created as an example of a humanized interface (HIT) that can interact with web pages and exchange information in a natural way. A few examples of applications of an avatar with semantic memory are given, including the twenty questions game and automatic creation of word puzzles.
Julian SzymanskiTomasz SarnatowiczWlodzislaw Duch2008-01-08T00:28:59Z2011-03-11T08:57:02Zhttp://cogprints.org/id/eprint/5891This item is in the repository with the URL: http://cogprints.org/id/eprint/58912008-01-08T00:28:59ZTowards comprehensive foundations of computational intelligenceAlthough computational intelligence (CI) covers a vast variety of different methods it still lacks an integrative theory. Several proposals for CI foundations are discussed: computing and cognition as compression, meta-learning as search in the space of data models, (dis)similarity based methods providing a framework for such meta-learning, and a more general approach based on chains of transformations. Many useful transformations that extract information from features are discussed. Heterogeneous adaptive systems are presented as particular example of transformation-based systems, and the goal of learning is redefined to facilitate creation of simpler data models. The need to understand data structures leads to techniques for logical and prototype-based rule extraction, and to generation of multiple alternative models, while the need to increase predictive power of adaptive models leads to committees of competent models. Learning from partial observations is a natural extension towards reasoning based on perceptions, and an approach to intuitive solving of such problems is presented. Throughout the paper neurocognitive inspirations are frequently used and are especially important in modeling of the higher cognitive functions. Promising directions such as liquid and laminar computing are identified and many open problems presented.
Prof Wlodzislaw Duchwduch@is.umk.pl2007-12-10T21:42:07Z2011-03-11T08:57:01Zhttp://cogprints.org/id/eprint/5869This item is in the repository with the URL: http://cogprints.org/id/eprint/58692007-12-10T21:42:07ZNotes on Convolutional Neural NetworksWe discuss the derivation and implementation of convolutional neural networks, followed by an extension which allows one to learn sparse combinations of feature maps. The derivation we present is specific to two-dimensional data and convolutions, but can be extended without much additional effort to an arbitrary number of dimensions. Throughout the discussion, we emphasize
efficiency of the implementation, and give small snippets of MATLAB code to accompany the equations.Jake Bouvrie2009-11-14T11:30:26Z2011-03-11T08:57:33Zhttp://cogprints.org/id/eprint/6718This item is in the repository with the URL: http://cogprints.org/id/eprint/67182009-11-14T11:30:26ZAsimov's Coming BackEver since the word ‘ROBOT’ first appeared in a science
fiction in 1921, scientists and engineers have been trying
different ways to create it. Present technologies in
mechanical and electrical engineering makes it possible
to have robots in such places as industrial manufacturing
and assembling lines. Although they are
essentially robotic arms or similarly driven by electrical
power and signal control, they could be treated the
primitive pioneers in application. Researches in the
laboratories go much further. Interdisciplines are
directing the evolution of more advanced robots. Among these are artificial
intelligence, computational neuroscience, mathematics and robotics. These disciplines
come closer as more complex problems emerge.
From a robot’s point of view, three basic abilities are needed. They are thinking
and memory, sensory perceptions, control and behaving. These are capabilities we
human beings have to adapt ourselves to the environment. Although
researches on robots, especially on intelligent thinking, progress slowly, a revolution
for biological inspired robotics is spreading out in the laboratories all over the world.Mr. L. Wangliyu.wang@wadh.oxon.org2006-03-16Z2011-03-11T08:56:21Zhttp://cogprints.org/id/eprint/4764This item is in the repository with the URL: http://cogprints.org/id/eprint/47642006-03-16ZThe Missing Link between Morphemic Assemblies and Behavioral Responses:a Bayesian Information-Theoretical model of lexical processingWe present the Bayesian Information-Theoretical (BIT) model of lexical processing: A mathematical model illustrating a novel approach to the modelling of language processes. The model shows how a neurophysiological theory of lexical processing relying on Hebbian association and neural assemblies can directly account for a variety of effects previously observed in behavioural experiments. We develop two information-theoretical measures of the distribution of usages of a morpheme or word, and use them to predict responses in three visual lexical decision datasets investigating inflectional morphology and polysemy. Our model offers a neurophysiological basis for the effects of
morpho-semantic neighbourhoods. These results demonstrate how distributed patterns of activation naturally result in the arisal of symbolic structures. We conclude by arguing that the modelling framework exemplified here, is
a powerful tool for integrating behavioural and neurophysiological results.Dr Fermin Moscoso del Prado MartinProf Aleksandar KosticDusica Filipovic-Djurdjevic2005-05-19Z2011-08-30T04:20:20Zhttp://cogprints.org/id/eprint/4362This item is in the repository with the URL: http://cogprints.org/id/eprint/43622005-05-19ZKnowledge-based Neural Network for Line Flow Contingency Selection and RankingThe Line flow Contingency Selection and Ranking (CS & R) is performed to rank the critical contingencies in order of their severity. An Artificial Neural Network based method for MW security assessment corresponding to line outage events have been reported by various authors in the literature. One way to provide an understanding of the behaviour of Neural Networks is to extract rules that can be provided to the user. The domain knowledge (fuzzy rules extracted from Multi-layer Perceptron model trained by Back Propagation algorithm) is integrated into a Neural Network for fast and accurate CS & R in an IEEE 14-bus system, for unknown load patterns and are found to be suitable for on-line applications at Energy Management Centers. The system user is provided with the capability to determine the set of conditions under which a line-outage is critical, and if critical, then how severe it is, thereby providing some degree of transparency of the ANN solution.Mr. Nitin Maliknitinmal@yahoo.comDr. Laxmi Srivastavalaxmi@sancharnet.in2006-03-06Z2011-03-11T08:56:21Zhttp://cogprints.org/id/eprint/4754This item is in the repository with the URL: http://cogprints.org/id/eprint/47542006-03-06ZThe Missing Link between Morphemic Assemblies and Behavioral Responses:a Bayesian Information-Theoretical model of lexical processingWe present the Bayesian Information-Theoretical (BIT) model of lexical processing: A mathematical model illustrating a novel approach to the modelling of language processes. The model shows how a neurophysiological theory of lexical processing relying on Hebbian association and neural assemblies can directly account for a variety of eects previously observed in behavioral experiments. We develop two information-theoretical measures of the distribution of usages of a word or morpheme. These measures are calculated through unsupervised means from corpora. We show that our measures succesfully predict responses in three visual lexical decision datasets investigating the processing of in
ectional morphology in Serbian and English languages, and the eects of polysemy and homonymy in English. We discuss how our model provides a neurophysiological grounding for the facilitatory and inhibitory eects of dierent types of lexical neighborhoods. In addition, our results show how, under a model based on neural assemblies, distributed patterns of activation naturally result in the arisal of discrete symbol-like structures. Therefore, the BIT model oers a point of reconciliation in the debate between distributed connectionist and discrete localist models. Finally, we argue that the modelling framework exemplied by the BIT model, is a powerful tool for integrating the different levels of the description of the human language
processing system.Fermin Moscoso del Prado MartinKostic AleksandarFilipovic-Djurdjevic Dusica2006-05-25Z2011-03-11T08:56:25Zhttp://cogprints.org/id/eprint/4881This item is in the repository with the URL: http://cogprints.org/id/eprint/48812006-05-25ZModelling and control of chaotic processes
through their Bifurcation Diagrams generated
with the help of Recurrent Neural Networks
models Part 2 - Industrial StudyMany real-world processes tend to be chaotic and are not amenable to satisfactory
analytical models. It has been shown here that for such chaotic processes represented
through short chaotic noisy observed data, a multi-input and multi-output recurrent
neural network can be built which is capable of capturing the process trends and
predicting the behaviour for any given starting condition. It is further shown that
this capability can be achieved by the recurrent neural network model when it is
trained to very low value of mean squared error. Such a model can then be used
for constructing the Bifurcation Diagram of the process leading to determination
of desirable operating conditions. Further, this multi-input and multi-output model
makes the process accessible for control using open-loop / closed-loop approaches
or bifurcation control etc.Krishnaiah JalluC.S. KumarM.A. Faruqi2006-10-15Z2011-03-11T08:56:39Zhttp://cogprints.org/id/eprint/5220This item is in the repository with the URL: http://cogprints.org/id/eprint/52202006-10-15ZArms races and car racesEvolutionary car racing (ECR) is extended to the case of two cars racing on the same track. A sensor representation is devised, and various methods of evolving car controllers for competitive racing are explored. ECR can be combined with co-evolution in a wide variety of ways, and one aspect which is explored here is the relative-absolute fitness continuum. Systematical behavioural differences are found along this continuum; further, a tendency to specialization and the reactive nature of the controller architecture are found to limit evolutionary progress.Julian TogeliusSimon M. Lucas2006-10-15Z2011-03-11T08:56:39Zhttp://cogprints.org/id/eprint/5222This item is in the repository with the URL: http://cogprints.org/id/eprint/52222006-10-15ZEvolution of Neural Networks for Helicopter Control: Why Modularity MattersThe problem of the automatic development of controllers for vehicles for which the exact characteristics are not known is considered in the context of miniature helicopter flocking. A methodology is proposed in which neural network based controllers are evolved in a simulation using a dynamic model qualitatively similar to the physical helicopter. Several network architectures and evolutionary sequences are investigated, and two approaches are found that can evolve very competitive controllers. The division of the neural network into modules and of the task into incremental steps seems to be a precondition for success, and we analyse why this might be so.Renzo De NardiJulian TogeliusOwen HollandSimon M. Lucas2006-10-15Z2011-03-11T08:56:39Zhttp://cogprints.org/id/eprint/5219This item is in the repository with the URL: http://cogprints.org/id/eprint/52192006-10-15ZEvolving robust and specialized car racing skillsNeural network-based controllers are evolved for racing simulated R/C cars around several tracks of varying difficulty. The transferability of driving skills acquired when evolving for a single track is evaluated, and different ways of evolving controllers able to perform well on many different tracks are investigated. It is further shown that such generally proficient controllers can reliably be developed into specialized controllers for individual tracks. Evolution of sensor parameters together with network weights is shown to lead to higher final fitness, but only if turned on after a general controller is developed, otherwise it hinders evolution. It is argued that simulated car racing is a scalable and relevant testbed for evolutionary robotics research, and that the results of this research can be useful for commercial computer games.Julian TogeliusSimon M. Lucas2006-05-25Z2011-03-11T08:56:26Zhttp://cogprints.org/id/eprint/4885This item is in the repository with the URL: http://cogprints.org/id/eprint/48852006-05-25ZA feedback model of perceptual learning and categorisationTop-down, feedback, influences are known to have significant effects on visual information processing. Such influences are also likely to affect perceptual learning. This article employs a computational model of the cortical region interactions underlying visual perception to investigate possible influences of top-down information on learning. The results suggest that feedback could bias the way in which perceptual stimuli are categorised and could also facilitate the learning of sub-ordinate level representations suitable for object identification and perceptual expertise.
Michael W SpratlingMark H Johnson2006-05-25Z2011-03-11T08:56:26Zhttp://cogprints.org/id/eprint/4886This item is in the repository with the URL: http://cogprints.org/id/eprint/48862006-05-25ZLearning image components for object recognitionIn order to perform object recognition it is necessary to learn representations of the underlying components of images. Such components correspond to objects, object-parts, or features. Non-negative matrix factorisation is a generative model that has been specifically proposed for finding such meaningful representations of image data, through the use of non-negativity constraints on the factors. This article reports on an empirical investigation of the performance of non-negative matrix factorisation algorithms. It is found that such algorithms need to impose additional constraints on the sparseness of the factors in order to successfully deal with occlusion. However, these constraints can themselves result in these algorithms failing to identify image components under certain conditions. In contrast, a recognition model (a competitive learning neural network algorithm) reliably and accurately learns representations of elementary image features without such constraints.
Michael W Spratling2006-10-15Z2011-03-11T08:56:39Zhttp://cogprints.org/id/eprint/5221This item is in the repository with the URL: http://cogprints.org/id/eprint/52212006-10-15ZMaking Racing Fun Through Player Modeling and Track EvolutionThis paper addresses the problem of automatically constructing tracks tailor-made to maximize the enjoyment of individual players in a simple car racing game. To this end, some approaches to player modeling are investigated, and a method of using evolutionary algorithms to construct racing tracks is presented. A simple player-dependent metric of entertainment is proposed and used as the fitness function when evolving tracks. We conclude that accurate player modeling poses some significant challenges, but track evolution works well given the right track representation.Julian TogeliusRenzo De NardiSimon M. Lucas2006-06-10Z2011-03-11T08:56:27Zhttp://cogprints.org/id/eprint/4914This item is in the repository with the URL: http://cogprints.org/id/eprint/49142006-06-10ZSEPARATING NONLINEAR IMAGE MIXTURES USING A PHYSICAL MODEL TRAINED WITH ICAThis work addresses the separation of real-life nonlinear
mixtures of images, which occur when a paper document is
scanned and the image from the back page shows through.
A physical model of the mixing process, based on the consideration of the halftoning process used to print grayscale images, is presented. The corresponding inverse model is then used to perform image separation. The parameters of the inverse model are optimized through the MISEP technique of nonlinear ICA, which uses an independence criterion based on minimal mutual information.
The quality of the separated images is competitive with
the one achieved by other techniques, namely by MISEP
with a generic MLP-based separation network and by Denoising
Source Separation. The separation results show that
MISEP is an appropriate technique for training the parameters and that the model fits the mixing process well, although not perfectly. Prospects for improvement of the model are presented.Mariana S. C. AlmeidaLuís B. Almeida2005-05-20Z2011-03-11T08:56:04Zhttp://cogprints.org/id/eprint/4360This item is in the repository with the URL: http://cogprints.org/id/eprint/43602005-05-20ZSeparating a Real-Life Nonlinear Image MixtureWhen acquiring an image of a paper document, the image printed on the back page sometimes shows through. The mixture of the front- and back-page images thus obtained is markedly nonlinear, and thus constitutes a good real-life test case for nonlinear blind source separation.
This paper addresses a difficult version of this problem, corresponding to the use of "onion skin" paper, which results in a relatively strong nonlinearity of the mixture, which becomes close to singular in the lighter regions of the images. The separation is achieved through the MISEP technique, which is an extension of the well known INFOMAX method. The separation results are assessed with objective quality measures. They show an improvement over the results obtained with linear separation, but have room for further improvement.Luis B. Almeida2005-02-08Z2011-03-11T08:55:51Zhttp://cogprints.org/id/eprint/4081This item is in the repository with the URL: http://cogprints.org/id/eprint/40812005-02-08ZOn the analysis and interpretation of inhomogeneous quadratic forms as receptive fieldsIn this paper we introduce some mathematical and numerical tools to analyze and interpret inhomogeneous quadratic forms. The resulting characterization is in some aspects similar to that given by experimental studies of cortical cells, making it particularly suitable for application to second-order approximations and theoretical models of physiological receptive fields. We first discuss two ways of analyzing a quadratic form by visualizing the coefficients of its quadratic and linear term directly and by considering the eigenvectors of its quadratic term. We then present an algorithm to compute the optimal excitatory and inhibitory stimuli, i.e. the stimuli that maximize and minimize the considered quadratic form, respectively, given a fixed energy constraint. The analysis of the optimal stimuli is completed by considering their invariances, which are the transformations to which the quadratic form is most insensitive. We introduce a test to determine which of these are statistically significant. Next we propose a way to measure the relative contribution of the quadratic and linear term to the total output of the quadratic form. Furthermore, we derive simpler versions of the above techniques in the special case of a quadratic form without linear term and discuss the analysis of such functions in previous theoretical and experimental studies. In the final part of the paper we show that for each quadratic form it is possible to build an equivalent two-layer neural network, which is compatible with (but more general than) related networks used in some recent papers and with the energy model of complex cells. We show that the neural network is unique only up to an arbitrary orthogonal transformation of the excitatory and inhibitory subunits in the first layer.
Pietro BerkesLaurenz Wiskott2005-02-16Z2011-03-11T08:55:51Zhttp://cogprints.org/id/eprint/4104This item is in the repository with the URL: http://cogprints.org/id/eprint/41042005-02-16ZPattern Recognition with Slow Feature AnalysisSlow feature analysis (SFA) is a new unsupervised algorithm to learn nonlinear functions that extract slowly varying signals out of the input data. In this paper we describe its application to pattern recognition. In this context in order to be slowly varying the functions learned by SFA need to respond similarly to the patterns belonging to the same class. We prove that, given input patterns belonging to C non-overlapping classes and a large enough function space, the optimal solution consists of C-1 output signals that are constant for each individual class. As a consequence, their output provides a feature space suitable to perform classification with simple methods, such as Gaussian classifiers. We then show as an example the application of SFA to the MNIST handwritten digits database. The performance of SFA is comparable to that of other established algorithms. Finally, we suggest some possible extensions to the proposed method. Our approach is in particular attractive because for a given input signal and a fixed function space it has no parameters, it is easy to implement and apply, and it has low memory requirements and high speed during recognition. SFA finds the global solution (within the considered function space) in a single iteration without convergence issues. Moreover, the proposed method is completely problem-independent.
Pietro Berkes2005-10-20Z2011-03-11T08:56:12Zhttp://cogprints.org/id/eprint/4567This item is in the repository with the URL: http://cogprints.org/id/eprint/45672005-10-20ZAccurate and robust image superresolution by neural processing of local image representations Image superresolution involves the processing of an image sequence to generate a still image with higher resolution. Classical approaches, such as bayesian MAP methods, require iterative minimization procedures, with high computational costs. Recently, the authors proposed a method to tackle this problem, based on the use of a hybrid MLP-PNN architecture. In this paper, we present a novel superresolution method, based on an evolution of this concept, to incorporate the use of local image models. A neural processing stage receives as input the value of model coefficients on local windows. The data dimension-ality is firstly reduced by application of PCA. An MLP, trained on synthetic se-quences with various amounts of noise, estimates the high-resolution image data. The effect of varying the dimension of the network input space is exam-ined, showing a complex, structured behavior. Quantitative results are presented showing the accuracy and robustness of the proposed method.Carlos MiravetFrancisco B. Rodriguez2006-07-23Z2011-03-11T08:56:29Zhttp://cogprints.org/id/eprint/4979This item is in the repository with the URL: http://cogprints.org/id/eprint/49792006-07-23ZAutonomous learning and reproduction of complex
sequences: a multimodal architecture for
bootstraping imitation gamesThis paper introduces a control architecture
for the learning of complex sequence of gestures
applied to autonomous robots. The architecture
is designed to exploit the robot internal
sensory-motor dynamics generated by
visual, proprioceptive, and predictive informations
in order to provide intuitive behaviors
in the purpose of natural interactions
with humans.Pierre AndryPhilippe GaussierJacqueline Nadel2006-02-26Z2011-03-11T08:56:20Zhttp://cogprints.org/id/eprint/4739This item is in the repository with the URL: http://cogprints.org/id/eprint/47392006-02-26ZA broad-coverage distributed connectionist model of visual word recognitionIn this study we describe a distributed connectionist model of morphological processing, covering a realistically sized sample of the English language. The purpose of this model is to explore how effects of discrete, hierarchically structured morphological paradigms, can arise as a result of the statistical sub-regularities in the mapping between
word forms and word meanings. We present a model that learns to produce at its output a realistic semantic representation of a word, on presentation of a distributed representation of its orthography. After training, in three experiments, we compare the outputs of the model with the lexical decision latencies for large sets of English nouns and verbs. We show that the model has developed detailed representations of morphological structure, giving rise to effects analogous to those observed in visual lexical decision experiments. In addition, we show how the association between word form and word meaning also
give rise to recently reported differences between regular and irregular verbs, even in their completely regular present-tense forms. We interpret these results as underlining the key importance for lexical processing of the statistical regularities in the mappings between form and meaning.
Dr Fermin Moscoso del Prado MartinProf R. Harald Baayen2005-07-13Z2011-03-11T08:56:08Zhttp://cogprints.org/id/eprint/4468This item is in the repository with the URL: http://cogprints.org/id/eprint/44682005-07-13ZClassification of Triadic Chord Inversions Using Kohonen
Self-organizing MapsIn this paper we discuss the application of the Kohonen Selforganizing
Maps to the classification of triadic chords in inversions and root
positions. Our motivation started in the validation of Schönberg´s hypotheses of
the harmonic features of each chord inversion. We employed the Kohonen
network, which has been generally known as an optimum pattern classification
tool in several areas, including music, to verify that hypothesis. The outcomes
of our experiment refuse the Schönberg´s assumption in two aspects: structural
and perceptual/functional.Luis F. OliveiraLuis G. P. LimaAndré L. G. OliveiraRael B. G. Toffolo2006-07-23Z2011-03-11T08:56:29Zhttp://cogprints.org/id/eprint/4969This item is in the repository with the URL: http://cogprints.org/id/eprint/49692006-07-23ZDevelopmental acquisition of entrainment skills in
robot swinging using van der Pol oscillatorsIn this study we investigated the effects of different
morphological configurations on a robot swinging
task using van der Pol oscillators. The task was
examined using two separate degrees of freedom
(DoF), both in the presence and absence of neural
entrainment. Neural entrainment stabilises the
system, reduces time-to-steady state and relaxes the
requirement for a strong coupling with the
environment in order to achieve mechanical
entrainment. It was found that staged release of the
distal DoF does not have any benefits over using both
DoF from the onset of the experimentation. On the
contrary, it is less efficient, both with respect to the
time needed to reach a stable oscillatory regime and
the maximum amplitude it can achieve. The same
neural architecture is successful in achieving
neuromechanical entrainment for a robotic walking
task.Paschalis VeskosYiannis Demiris2006-10-15Z2011-03-11T08:56:39Zhttp://cogprints.org/id/eprint/5218This item is in the repository with the URL: http://cogprints.org/id/eprint/52182006-10-15ZEvolving controllers for simulated car racingThis paper describes the evolution of controllers for racing a simulated radio-controlled car around a track, modelled on a real physical track. Five different controller architectures were compared, based on neural networks, force fields and action sequences. The controllers use either egocentric (first person), Newtonian (third person) or no information about the state of the car (open-loop controller). The only controller that is able to evolve good racing behaviour is based on a neural network acting on egocentric inputs.Julian TogeliusSimon M. Lucas2005-05-05Z2011-03-11T08:55:51Zhttp://cogprints.org/id/eprint/4103This item is in the repository with the URL: http://cogprints.org/id/eprint/41032005-05-05ZForcing neurocontrollers to exploit sensory symmetry through hard-wired modularity in the game of CellzSeveral attempts have been made in the past to construct encoding schemes that allow modularity to emerge in evolving systems, but success is limited. We believe that in order to create successful and scalable encodings for emerging modularity, we first need to explore the benefits of different types of modularity by hard-wiring these into evolvable systems. In this paper we explore different ways of exploiting sensory symmetry inherent in the agent in the simple game Cellz by evolving symmetrically identical modules. It is concluded that significant increases in both speed of evolution and final fitness can be achieved relative to monolithic controllers. Furthermore, we show that a simple function approximation task that exhibits sensory symmetry can be used as a quick approximate measure of the utility of an encoding scheme for the more complex game-playing task.Julian TogeliusSimon M. Lucas2005-05-20Z2011-03-11T08:56:04Zhttp://cogprints.org/id/eprint/4361This item is in the repository with the URL: http://cogprints.org/id/eprint/43612005-05-20ZKNOWLEDGE-BASED NEURAL NETWORK FOR LINE FLOW CONTINGENCY SELECTION AND RANKINGThe Line flow Contingency Selection and Ranking (CS & R) is performed to rank the critical contingencies in order of their severity. An Artificial Neural Network based method for MW security assessment corresponding to line outage events have been reported by various authors in the literature. One way to provide an understanding of the behaviour of Neural Networks is to extract rules that can be provided to the user. The domain knowledge (fuzzy rules extracted from Multi-layer Perceptron model trained by Back Propagation algorithm) is integrated into a Neural Network for fast and accurate CS & R in an IEEE 14-bus system, for unknown load patterns and are found to be suitable for on-line applications at Energy Management Centers. The system user is provided with the capability to determine the set of conditions under which a line-outage is critical, and if critical, then how severe it is, thereby providing some degree of transparency of the ANN solution.Mr. Nitin MalikMrs. Laxmi Srivastava2006-05-25Z2011-03-11T08:56:26Zhttp://cogprints.org/id/eprint/4884This item is in the repository with the URL: http://cogprints.org/id/eprint/48842006-05-25ZLearning viewpoint invariant perceptual representations from cluttered imagesIn order to perform object recognition, it is necessary to form perceptual representations that are sufficiently specific to distinguish between objects, but that are also sufficiently flexible to generalise across changes in location, rotation and scale. A standard method for learning perceptual representations that are invariant to viewpoint is to form temporal associations across image sequences showing object transformations. However, this method requires that individual stimuli are presented in isolation and is therefore unlikely to succeed in real-world applications where multiple objects can co-occur in the visual input. This article proposes a simple modification to the learning method, that can overcome this limitation, and results in more robust learning of invariant representations.
Dr Michael Spratling2006-12-12Z2011-03-11T08:56:44Zhttp://cogprints.org/id/eprint/5298This item is in the repository with the URL: http://cogprints.org/id/eprint/52982006-12-12ZRetrospective revaluation as simple associative learningBackward blocking, unovershadowing and backward conditioned inhibition are examples of "retrospective revaluation" phenomena, that have been suggested to involve more than simple associative learning. Models of these phenomena have thus employed additional concepts, e.g. appealing to attentional effects or more elaborate learning mechanisms. I show that a suitable representation of stimuli, paired with a careful analysis of the discriminations faced by animals, leads to an account of these and other phenomena in terms of a simple "elemental" model of associative learning, with essentially the same learning mechanism as the Rescorla and Wagner (1972) model. I conclude with a discussion of some implications for theories of learning.
Stefano Ghirlanda2006-07-23Z2011-03-11T08:56:30Zhttp://cogprints.org/id/eprint/4984This item is in the repository with the URL: http://cogprints.org/id/eprint/49842006-07-23ZSymbol manipulation by internal simulation of
perception and behaviourMichel van DartelEric Postma2006-07-23Z2011-03-11T08:56:30Zhttp://cogprints.org/id/eprint/4985This item is in the repository with the URL: http://cogprints.org/id/eprint/49852006-07-23ZTowards Teaching a Robot to Count ObjectsWe present here an example of incremental
learning between two computational models
dealing with different modalities: a model allowing
to switch spatial visual attention and a
model allowing to learn the ordinal sequence
of phonetical numbers. Their merging via a
common reward signal allows anyway to produce
a cardinal counting behaviour that can
be implemented on a robot.Julien Vitay2004-07-06Z2011-03-11T08:55:37Zhttp://cogprints.org/id/eprint/3701This item is in the repository with the URL: http://cogprints.org/id/eprint/37012004-07-06ZŁukasiewicz-Topos Models of Neural Networks, Cell Genome and Interactome Nonlinear Dynamic Models
A categorical and Łukasiewicz-Topos framework for Algebraic Logic models of nonlinear dynamics in complex functional systems such as Neural Networks, Cell Genome and Interactome Networks is introduced. Łukasiewicz Algebraic Logic models of both neural and genetic networks and signaling pathways in cells are formulated in terms of nonlinear dynamic systems with n-state components that allow for the generalization of previous logical models of both genetic activities and neural networks. An algebraic formulation of variable 'next-state functions' is extended to a Łukasiewicz Topos with an n-valued Łukasiewicz Algebraic Logic subobject classifier description that represents non-random and nonlinear network activities as well as their transformations in developmental processes and carcinogenesis.
Professor I.C. Baianuicb12004-05-14Z2011-03-11T08:55:36Zhttp://cogprints.org/id/eprint/3633This item is in the repository with the URL: http://cogprints.org/id/eprint/36332004-05-14ZBehaviourally meaningful representations from normalisation and context-guided denoisingMany existing independent component analysis algorithms include a preprocessing stage where the inputs are sphered. This amounts to normalising the data such that all correlations between the variables are removed. In this work, I show that sphering allows very weak contextual modulation to steer the development of meaningful features. Context-biased competition has been proposed as a model of covert attention and I propose that sphering-like normalisation also allows weaker top-down bias to guide attention.
Harri Valpola2004-03-04Z2011-03-11T08:55:28Zhttp://cogprints.org/id/eprint/3441This item is in the repository with the URL: http://cogprints.org/id/eprint/34412004-03-04ZDenoising source separationA new algorithmic framework called denoising source separation (DSS) is introduced. The main benefit of this framework is that it allows for easy development of new source separation algorithms which are optimised for specific problems. In this framework, source separation algorithms are constucted around denoising procedures. The resulting algorithms can range from almost blind to highly specialised source separation algorithms. Both simple linear and more complex nonlinear or adaptive denoising schemes are
considered. Some existing independent component analysis algorithms are reinterpreted within DSS framework and new, robust blind source separation algorithms are suggested. Although DSS algorithms need not be explicitly based on objective functions, there is often an implicit objective function that is optimised. The exact relation between the denoising procedure and the objective function is derived and a useful approximation of the objective function is presented. In the experimental section, various DSS schemes are applied extensively to artificial data, to real magnetoencephalograms and to simulated CDMA mobile network signals. Finally, various extensions to the proposed DSS algorithms are considered. These include nonlinear observation mappings, hierarchical models and overcomplete, nonorthogonal feature spaces. With these extensions, DSS appears to have relevance to many existing models of neural information processing.
Mr Jaakko SäreläDr Harri Valpola2004-03-16Z2011-03-11T08:55:29Zhttp://cogprints.org/id/eprint/3493This item is in the repository with the URL: http://cogprints.org/id/eprint/34932004-03-16ZDenoising source separationA new algorithmic framework called denoising source separation (DSS)
is introduced. The main benefit of this framework is that it allows
for easy development of new source separation algorithms which are
optimised for specific problems. In this framework, source
separation algorithms are constucted around denoising
procedures. The resulting algorithms can range from almost blind to
highly specialised source separation algorithms. Both simple linear
and more complex nonlinear or adaptive denoising schemes are
considered. Some existing independent component analysis algorithms
are reinterpreted within DSS framework and new, robust blind source
separation algorithms are suggested. Although DSS algorithms need
not be explicitly based on objective functions, there is often an
implicit objective function that is optimised. The exact relation
between the denoising procedure and the objective function is
derived and a useful approximation of the objective function is
presented. In the experimental section, various DSS schemes are
applied extensively to artificial data, to real
magnetoencephalograms and to simulated CDMA mobile network signals.
Finally, various extensions to the proposed DSS algorithms are
considered. These include nonlinear observation mappings,
hierarchical models and overcomplete, nonorthogonal feature spaces.
With these extensions, DSS appears to have relevance to many
existing models of neural information processing.Mr Jaakko SäreläDr Harri Valpola2004-05-24Z2011-03-11T08:55:36Zhttp://cogprints.org/id/eprint/3637This item is in the repository with the URL: http://cogprints.org/id/eprint/36372004-05-24ZAccurate, fast and stable denoising source separation algorithms Denoising source separation is a recently introduced framework for
building source separation algorithms around denoising procedures.
Two developments are reported here. First, a new scheme for
accelerating and stabilising convergence by controlling step sizes
is introduced. Second, a novel signal-variance based denoising function
is proposed. Estimates of variances of different source are
whitened which actively promotes separation of sources. Experiments
with artificial data and real magnetoencephalograms demonstrate that
the developed algorithms are accurate, fast and stable.
Dr Harri Valpola4893Mr Jaakko Särelä47152005-04-14Z2011-03-11T08:55:52Zhttp://cogprints.org/id/eprint/4150This item is in the repository with the URL: http://cogprints.org/id/eprint/41502005-04-14ZBinding tactile and visual sensations via unique association by cross-anchoring between double-touching and self-occlusionBinding is one of the most fundamental cognitive functions, how to find the correspondence of sensations between different modalities such as vision and touch. Without a priori knowledge on this correspondence, binding is regarded to be a formidable issue for a robot since it often perceives multiple physical phenomena in its different modal sensors, therefore it should correctly match the foci of attention in different modalities that may have multiple correspondences each other. We suppose that learning the multimodal representation of the body should be the first step toward binding since the morphological constraints in self-body-observation would make the binding problem tractable. The multimodal sensations are expected to be constrained in perceiving own body so as to configurate the unique parts of the multiple correspondence reflecting its morphology. In this paper, we propose a method to match the foci of attention in vision and touch through the unique association by cross-anchoring different modalities. Simple experiments show the validity of the proposed method.Yuichiro YoshikawaKoh HosodaMinoru Asada2004-11-20Z2011-03-11T08:55:44Zhttp://cogprints.org/id/eprint/3943This item is in the repository with the URL: http://cogprints.org/id/eprint/39432004-11-20ZConnectionist Taxonomy LearningThe paper at hand describes an approach to automatise the creation of a class taxonomy. Information about objects, e.g. "a tank is armored and moves by track", but no prior knowledge about taxonomy structure is presented to a connectionist system which organizes itself by means of activation spreading (McClelland and Rumelhart, 1981) and weight adjustments. The resulting connectionist network has a form of a taxonomy sought-after.Miloslaw Frey2004-04-06Z2011-03-11T08:55:30Zhttp://cogprints.org/id/eprint/3542This item is in the repository with the URL: http://cogprints.org/id/eprint/35422004-04-06ZA feedback model of visual attentionFeedback connections are a prominent feature of cortical anatomy and are likely
to have significant functional role in neural information processing. We present
a neural network model of cortical feedback that successfully simulates
neurophysiological data associated with attention. In this domain our model can
be considered a more detailed, and biologically plausible, implementation of the
biased competition model of attention. However, our model is more general as it
can also explain a variety of other top-down processes in vision, such as
figure/ground segmentation and contextual cueing. This model thus suggests that
a common mechanism, involving cortical feedback pathways, is responsible for a
range of phenomena and provides a unified account of currently disparate areas
of research.
M W SpratlingM H Johnson2005-04-14Z2011-03-11T08:55:52Zhttp://cogprints.org/id/eprint/4148This item is in the repository with the URL: http://cogprints.org/id/eprint/41482005-04-14ZA Multimodal Hierarchial Approach to Robot Learning by ImitationIn this paper we propose an approach to robot learning by imitation that uses the multimodal inputs of language, vision and motor. In our approach a student robot learns from a teacher robot how to perform three separate behaviours based on these inputs. We considered two neural architectures for performing this robot learning. First, a one-step hierarchial architecture trained with two different learning approaches either based on Kohonen's self-organising map or based on the Helmholtz machine turns out to be inefficient or not capable of performing differentiated behavior. In response we produced a hierarchial architecture that combines both learning approaches to overcome these problems. In doing so the proposed robot system models specific aspects of learning using concepts of the mirror neuron system (Rizzolatti and Arbib, 1998) with regards to demonstration learning.Cornelius WeberMark ElshawAlex ZochiosStefan Wermter2004-04-06Z2011-03-11T08:55:30Zhttp://cogprints.org/id/eprint/3541This item is in the repository with the URL: http://cogprints.org/id/eprint/35412004-04-06ZNeural coding strategies and mechanisms of competitionA long running debate has concerned the question of whether neural
representations are encoded using a distributed or a local coding scheme. In
both schemes individual neurons respond to certain specific patterns of
pre-synaptic activity. Hence, rather than being dichotomous, both coding
schemes are based on the same representational mechanism. We argue that a
population of neurons needs to be capable of learning both local and distributed
representations, as appropriate to the task, and should be capable of generating
both local and distributed codes in response to different stimuli. Many neural
network algorithms, which are often employed as models of cognitive processes,
fail to meet all these requirements. In contrast, we present a neural network
architecture which enables a single algorithm to efficiently learn, and respond
using, both types of coding scheme.
Dr M W SpratlingProf M H Johnson2004-04-28Z2011-03-11T08:55:31Zhttp://cogprints.org/id/eprint/3580This item is in the repository with the URL: http://cogprints.org/id/eprint/35802004-04-28ZA Neural Model of Episodic and Semantic Spatiotemporal MemoryA neural network model is proposed that forms sparse spatiotemporal memory traces of spatiotemporal events given single occurrences of the events. The traces are distributed in that each individual cell and synapse participates in numerous traces. This sharing of representational substrate provides the basis for similarity-based generalization and thus semantic memory. Simulation results are provided demonstrating that similar spatiotemporal patterns map to similar traces. The model achieves this property by measuring the degree of match, G, between the current input pattern on each time slice and the expected input given the preceding time slices (i.e., temporal context) and then adding an amount of noise, inversely proportional to G, to the process of choosing the internal representation for the current time slice. Thus, if G is small, indicating novelty, we add much noise and the resulting internal representation of the current input pattern has low overlap with any preexisting representations of time slices. If G is large, indicating a familiar event, we add very little noise resulting in reactivation of all or most of the preexisting representation of the input pattern.Gerard J. Rinkus2005-04-14Z2011-03-11T08:55:50Zhttp://cogprints.org/id/eprint/4071This item is in the repository with the URL: http://cogprints.org/id/eprint/40712005-04-14ZProtosymbols that integrate recognition and responseWe explore two controversial hypotheses through robotic implementation: (1) Processes involved in recognition and response are tightly coupled both in their operation and epigenesis; and (2) processes involved in symbol emergence should respect the integrity of recognition and response while exploiting the periodicity of biological motion. To that end, this paper proposes a method of recognizing and generating motion patterns based on nonlinear principal component neural networks that are constrained to model both periodic and transitional movements. The method is evaluated by an examination of its ability to segment and generalize different kinds of soccer playing activity during a RoboCup match.Karl F. MacDormanRawichote ChalodhornHiroshi IshiguroMinoru Asada2005-04-14Z2011-03-11T08:55:50Zhttp://cogprints.org/id/eprint/4064This item is in the repository with the URL: http://cogprints.org/id/eprint/40642005-04-14ZSimulating development in a real robot: on the concurrent increase of sensory, motor, and neural complexityWe present a quantitative investigation on the effects of a discrete developmental progression on the acquisition of a foveation behavior by a robotic hand-arm-eyes system. Development is simulated by (a) increasing the resolution of visual and tactile systems, (b) freezing and freeing mechanical degrees of freedom, and (c) adding neuronal units to the neural control architecture. Our experimental results show that a system starting with a low-resolution sensory system, a low precision motor system, and a low complexity neural structure, learns faster that a system which is more complex at the beginning.Gabriel GomezMax LungarellaPeter Eggenberger HotzKojiro MatsushitaRolf Pfeifer2004-12-30Z2011-03-11T08:55:48Zhttp://cogprints.org/id/eprint/4012This item is in the repository with the URL: http://cogprints.org/id/eprint/40122004-12-30ZWhat is the functional role of adult neurogenesis in the hippocampus? The dentate gyrus is part of the hippocampal memory system and special in
that it generates new neurons throughout life. Here we discuss the
question of what the functional role of these new neurons might be. Our
hypothesis is that they help the dentate gyrus to avoid the problem of
catastrophic interference when adapting to new environments. We assume
that old neurons are rather stable and preserve an optimal encoding
learned for known environments while new neurons are plastic to adapt to
those features that are qualitatively new in a new environment. A simple
network simulation demonstrates that adding new plastic neurons is indeed
a successful strategy for adaptation without catastrophic interference.
Laurenz WiskottMalte J. RaschGerd Kempermann2003-12-18Z2011-03-11T08:55:24Zhttp://cogprints.org/id/eprint/3319This item is in the repository with the URL: http://cogprints.org/id/eprint/33192003-12-18ZBrain-inspired conscious computing architectureWhat type of artificial systems will claim to be conscious and will claim to experience qualia? The ability to comment upon physical states of a brain-like dynamical system coupled with its environment seems to be sufficient to make claims. The flow of internal states in such system, guided and limited by associative memory, is similar to the stream of consciousness. Minimal requirements for an artificial system that will claim to be conscious were given in form of specific architecture named articon. Nonverbal discrimination of the working memory states of the articon gives it the ability to experience different qualities of internal states. Analysis of the inner state flows of such a system during typical behavioral process shows that qualia are inseparable from perception and action. The role of consciousness in learning of skills, when conscious information processing is replaced by subconscious, is elucidated. Arguments confirming that phenomenal experience is a result of cognitive processes are presented. Possible philosophical objections based on the Chinese room and other arguments are discussed, but they are insufficient to refute claims articon’s claims. Conditions for genuine understanding that go beyond the Turing test are presented. Articons may fulfill such conditions and in principle the structure of their experiences may be arbitrarily close to human.
Prof Wlodzislaw Duch2004-05-24Z2011-03-11T08:55:36Zhttp://cogprints.org/id/eprint/3638This item is in the repository with the URL: http://cogprints.org/id/eprint/36382004-05-24Z Overlearning in marginal distribution-based ICA: analysis and solutions The present paper is written as a word of caution, with users of
independent component analysis (ICA) in mind, to overlearning
phenomena that are often observed.\\
We consider two types of overlearning, typical to high-order
statistics based ICA. These algorithms can be seen to maximise the
negentropy of the source estimates. The first kind of overlearning
results in the generation of spike-like signals, if there are not
enough samples in the data or there is a considerable amount of
noise present. It is argued that, if the data has power spectrum
characterised by $1/f$ curve, we face a more severe problem, which
cannot be solved inside the strict ICA model. This overlearning is
better characterised by bumps instead of spikes. Both overlearning
types are demonstrated in the case of artificial signals as well as
magnetoencephalograms (MEG). Several methods are suggested to
circumvent both types, either by making the estimation of the ICA
model more robust or by including further modelling of the data.
Mr Jaakko Särelä47152004-01-27Z2011-03-11T08:55:27Zhttp://cogprints.org/id/eprint/3406This item is in the repository with the URL: http://cogprints.org/id/eprint/34062004-01-27ZAs contribuições da ciência cognitiva à composição musicalThis dissertation’s goal is to construct a detailed map of the principal branches of
cognitive science and their methodological and epistemological contributions to the study
music composition. We are concerned, firstly, with the contributions to the compositional
techniques, and secondly, with their perception. The first chapter deals with the cognitivist
paradigm by means of artificial intelligence. In the second chapter we relate the artificial
intelligence with the music composition, investigating the cognitvist models of composition
by the analysis of automatic compositional systems. The third chapter brings the artificial
neural networks to the scene, within the so-called connectionist paradigm. In our fourth
chapter we established the relation between the connectionism and music composition. In this
sense, we describe implementations that model and/or simulate aspects of perception and
composition. The fifth chapter leaves the computational perspective in the study of cognition
and present alternative proposals in this sense, related to the music composition and
musicology, as the ecological approach to auditory perception and the theories of
emergentism applied to music.Luis F Oliveira2005-08-20Z2011-03-11T08:56:09Zhttp://cogprints.org/id/eprint/4506This item is in the repository with the URL: http://cogprints.org/id/eprint/45062005-08-20ZAs contribuições da ciência cognitiva à composição musicalThis dissertation’s goal is to construct a detailed map of the principal branches of
cognitive science and their methodological and epistemological contributions to the study
music composition. We are concerned, firstly, with the contributions to the compositional
techniques, and secondly, with their perception. The first chapter deals with the cognitivist
paradigm by means of artificial intelligence. In the second chapter we relate the artificial
intelligence with the music composition, investigating the cognitvist models of composition
by the analysis of automatic compositional systems. The third chapter brings the artificial
neural networks to the scene, within the so-called connectionist paradigm. In our fourth
chapter we established the relation between the connectionism and music composition. In this
sense, we describe implementations that model and/or simulate aspects of perception and
composition. The fifth chapter leaves the computational perspective in the study of cognition
and present alternative proposals in this sense, related to the music composition and
musicology, as the ecological approach to auditory perception and the theories of
emergentism applied to music.Luis F Oliveira2003-12-13Z2011-03-11T08:55:24Zhttp://cogprints.org/id/eprint/3306This item is in the repository with the URL: http://cogprints.org/id/eprint/33062003-12-13ZEvolution of the layers in a subsumption architecture robot controllerAn approach to robotics called layered evolution and merging features from the subsumption architecture into evolutionary robotics is presented, its advantages and its relevance for science and engineering are discussed. This approach is used to construct a layered controller for a simulated robot that learns which light source to approach in an environment with obstacles. The evolvability and performance of layered evolution on this task is compared to (standard) monolithic evolution, incremental and modularised evolution. To test the optimality of the evolved solutions the evolved controller is merged back into a single network. On the grounds of the test results, it is argued that layered evolution provides a superior approach for many tasks, and future research projects involving this approach are suggested.Mr Julian Togelius45352003-09-19Z2011-03-11T08:55:20Zhttp://cogprints.org/id/eprint/3157This item is in the repository with the URL: http://cogprints.org/id/eprint/31572003-09-19ZSynchronization in model networks of class I neuronsWe study a modification of the Hoppensteadt-Izhikevich canonical model for networks of class I neurons, in which the 'pulse' emitted by a neuron is smooth rather than a delta-function. We prove two types of results about synchronization and desynchronization of such networks, the first type pertaining to 'pulse' functions which are symmetric, and the other type in the regime in which each neuron is connected to many other neurons.
Dr Guy Katriel2003-10-06Z2011-03-11T08:55:21Zhttp://cogprints.org/id/eprint/3181This item is in the repository with the URL: http://cogprints.org/id/eprint/31812003-10-06ZSynchronization in model networks of class I neuronsWe study a modification of the Hoppensteadt-Izhikevich canonical model for networks of class I neurons, in which the 'pulse' emitted by a neuron is smooth rather than a delta-function. We prove two types of results about synchronization and desynchronization of such networks, the first type pertaining to 'pulse' functions which are symmetric, and the other type in the regime in which each neuron is connected to many other neurons.
Guy Katriel2003-09-19Z2011-03-11T08:55:20Zhttp://cogprints.org/id/eprint/3152This item is in the repository with the URL: http://cogprints.org/id/eprint/31522003-09-19ZOrder-disorder transition in the Chialvo-Bak `minibrain' controlled by network geometryWe examine a simple biologically-motivated neural network, the three-layer version of the Chialvo-Bak `minibrain' [Neurosci. 90 (1999) 1137], and present numerical results which indicate that a non-equilibrium phase transition between ordered and disordered phases occurs subject to the tuning of a control parameter. Scale-free behaviour is observed at the critical point. Notably, the transition here is due solely to network geometry and not any noise factor. The phase of the network is thus a design parameter which can be tuned. The phases are determined by differing levels of interference between active paths in the network and the consequent accidental destruction of good paths.Joseph WakelingJWakeling2004-02-12Z2011-03-11T08:55:26Zhttp://cogprints.org/id/eprint/3350This item is in the repository with the URL: http://cogprints.org/id/eprint/33502004-02-12ZA biologically inspired computational model of the Block Copying TaskWe present in this paper a biologically inspired model of the Basal Ganglia which deals with Block Copying as a sequence learning task. By breaking a relatively complex task into simpler operations with well-defined skills, an approach which is termed as a skill-based machine design is used in the device of computational models to complete such tasks. Basal Ganglia are critically involved in sensorimotor control. From the learning aspects, Actor-Critic architectures have been proposed to model the Basal Ganglia and Temporal difference has been proposed as a learning algorithm. The model is implemented and simulation results are presented to show the capability of our model to successfully complete the task.Tian LanMichael ArnoldTerrence SejnowskiMarwan Jabri2004-02-12Z2011-03-11T08:55:25Zhttp://cogprints.org/id/eprint/3336This item is in the repository with the URL: http://cogprints.org/id/eprint/33362004-02-12ZCollaboration Development through Interactive Learning between Human and RobotIn this paper, we investigated interactive learning between human subjects and robot experimentally, and its essential characteristics are examined using the dynamical systems approach. Our research concentrated on the navigation system of a specially developed humanoid robot called Robovie and seven human subjects whose eyes were covered, making them dependent on the robot for directions. We compared the usual feed-forward neural network (FFNN) without recursive connections and the recurrent neural network (RNN). Although the performances obtained with both the RNN and the FFNN improved in the early stages of learning, as the subject changed the operation by learning on its own, all performances gradually became unstable and failed. Results of a questionnaire given to the subjects confirmed that the FFNN gives better mental impressions, especially from the aspect of operability. When the robot used a consolidation-learning algorithm using the rehearsal outputs of the RNN, the performance improved even when interactive learning continued for a long time. The questionnaire results then also confirmed that the subject's mental impressions of the RNN improved significantly. The dynamical systems analysis of RNNs support these differences and also showed that the collaboration scheme was developed dynamically along with succeeding phase transitions.Tetsuya OgataNoritaka MasagoShigeki SuganoJun Tani2004-04-06Z2011-03-11T08:55:17Zhttp://cogprints.org/id/eprint/3001This item is in the repository with the URL: http://cogprints.org/id/eprint/30012004-04-06ZControlling chaos in a chaotic neural networkThe chaotic neural network constructed with chaotic neuron shows the associative memory function, but its memory searching process cannot be stabilized in a stored state because of the chaotic motion of the network. In this paper, a pinning control method focused on the chaotic neural network is proposed. The computer simulation proves that the chaos in the chaotic neural network can be controlled with this method and the states of the network can converge in one of its stored patterns if the control strength and the pinning density are chosen suitable. It is found that in general the threshold of the control strength of a controlled network is smaller at higher pinned density and the chaos of the chaotic neural network can be controlled more easily if the pinning control is added to the variant neurons between the initial pattern and the target pattern.Dr G. HeggheProf. Z. CaoProf. P. ZhuProf. H. Ogura2004-02-12Z2011-03-11T08:55:26Zhttp://cogprints.org/id/eprint/3343This item is in the repository with the URL: http://cogprints.org/id/eprint/33432004-02-12ZA Developmental Approach for low-level ImitationsHistorically, a lot of authors in psychology and in
robotics tend to separate "true imitation" and its
related high-level mechanisms which seem to be exclusive to human adult, from low-level imitations or
"mimicries" observed on babies or primates. Closely,
classical researches suppose that an imitative artificial system must be able to build a model of
the demonstrator's geometry, in order to reproduce finely the movements on each joints. Conversely, we
will advocate that if imitation is viewed as a part of a
developmental course, then (1) an artificial developing system does not need to build any internal model
of the other, to perform real-time and low-level imitations of human movements despite the related correspondence problem between man and robot and,
(2) a simple sensory-motor loop could be at the basis
of multiples heterogeneous imitative behaviors often
explained in the literature by different models.Pierre AndryPhilippe GaussierJacqueline NadelMichele Courant2006-12-08Z2011-03-11T08:56:43Zhttp://cogprints.org/id/eprint/5280This item is in the repository with the URL: http://cogprints.org/id/eprint/52802006-12-08ZThe evolution of signal form: Effects of learned versus inherited recognitionOrganisms can learn by individual experience to recognize relevant stimuli
in the environment or they can genetically inherit this ability from their
parents. Here, we ask how these two modes of acquisition affect signal evolution, focusing in particular on the exaggeration and cost of signals. We argue first, that faster learning by individual receivers cannot be a driving force for the evolution of exaggerated and costly signals unless signal senders are related or the same receiver and sender meet repeatedly. We argue instead that biases in receivers’ recognition mechanisms can promote the evolution of costly exaggeration in signals. We provide support for this hypothesis by simulating coevolution between senders and receivers, using artificial neural networks as a model of receivers’ recognition mechanisms. We analyse the joint effects of receiver biases, signal cost and mode of acquisition, investigating the circumstances under which learned recognition gives rise to more exaggerated signals than inherited recognition. We conclude the paper by discussing the relevance of our results to a number of biological scenarios.Masashi KamoStefano GhirlandaMagnus Enquist2004-04-06Z2011-03-11T08:55:30Zhttp://cogprints.org/id/eprint/3543This item is in the repository with the URL: http://cogprints.org/id/eprint/35432004-04-06ZExploring the functional significance of dendritic inhibition in cortical pyramidal cellsInhibitory synapses contacting the soma and axon initial segment are commonly
presumed to participate in shaping the response properties of cortical pyramidal
cells. Such an inhibitory mechanism has been explored in numerous computational
models. However, the majority of inhibitory synapses target the dendrites of
pyramidal cells, and recent physiological data suggests that this dendritic
inhibition affects tuning properties. We describe a model that can be used to
investigate the role of dendritic inhibition in the competition between
neurons. With this model we demonstrate that dendritic inhibition significantly
enhances the computational and representational properties of neural networks.
M W SpratlingM H Johnson2003-03-30Z2011-03-11T08:55:14Zhttp://cogprints.org/id/eprint/2854This item is in the repository with the URL: http://cogprints.org/id/eprint/28542003-03-30ZFaster Training in Nonlinear ICA using MISEPMISEP has been proposed as a generalization of the INFOMAX method in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs, making the method suitable to the separation of components with a wide range of statistical distributions. In all implementations up to now, MISEP had used multilayer perceptrons (MLPs) to perform the nonlinear ICA operation. Use of MLPs sometimes leads to a relatively slow training. This has been attributed, at least in part, to the non-local character of the MLP's units. This paper investigates the possibility of using a network of radial basis function (RBF) units for performing the nonlinear ICA operation. It shows that the local character of the RBF network's units allows a significant speedup in the training of the system. The paper gives a brief introduction to the basics of the MISEP method, and presents experimental results showing the speed advantage of using an RBF-based network to perform the ICA operation.Luis B. Almeida2004-02-12Z2011-03-11T08:55:26Zhttp://cogprints.org/id/eprint/3348This item is in the repository with the URL: http://cogprints.org/id/eprint/33482004-02-12ZModelling cortico basal-ganglionic loops and the development of sequential information encodingA connectionist model consisting of thirty cortico-basal ganglionic loops was implemented. This model encodes temporal information into a spatial pattern of neuronal activations in the prefrontal cortex using neurophysiologically plausible activation functions and circuitry without learning. This neural architecture was used to model experiments with infants. Initial results suggest that the cortical basal ganglionic circuitry has an inherent ability to differentiate sequential information.Ryuta FukudaMicheal SpratlingDenis MareschalMark Johnson2003-10-09Z2011-03-11T08:55:21Zhttp://cogprints.org/id/eprint/3196This item is in the repository with the URL: http://cogprints.org/id/eprint/31962003-10-09ZNeural blackboard architectures of combinatorial structures in cognitionHuman cognition is unique in the way in which it relies on combinatorial (or compositional) structures. Language provides ample evidence for the existence of combinatorial structures, but they can also be found in visual cognition. To understand the neural basis of human cognition, it is therefore essential to understand how combinatorial structures can be instantiated in neural terms. In his recent book on the foundations of language, Jackendoff described four fundamental problems for a neural instantiation of combinatorial structures: the massiveness of the binding problem, the problem of 2, the problem of variables and the transformation of combinatorial structures from working memory to long-term memory. This paper aims to show that these problems can be solved by means of neural ‘blackboard’ architectures. For this purpose, a neural blackboard architecture for sentence structure is presented. In this architecture, neural structures that encode for words are temporarily bound in a manner that preserves the structure of the sentence. It is shown that the architecture solves the four problems presented by Jackendoff. The ability of the architecture to instantiate sentence structures is illustrated with examples of sentence complexity observed in human language performance. Similarities exist between the architecture for sentence structure and blackboard architectures for combinatorial structures in visual cognition, derived from the structure of the visual cortex. These architectures are briefly discussed, together with an example of a combinatorial structure in which the blackboard architectures for language and vision are combined. In this way, the architecture for language is grounded in perception. Dr. Frank van der Velde2003-05-25Z2011-03-11T08:55:17Zhttp://cogprints.org/id/eprint/2972This item is in the repository with the URL: http://cogprints.org/id/eprint/29722003-05-25ZProcessing of analogy in the thalamocortical circuit The corticothalamic feedback and the thalamic reticular nucleus have gained
much attention lately because of their integrative and modulatory functions.
A previous study by the author suggested that
this circuitry can process analogies (i.e., the {\em analogy hypothesis}).
In this paper, the proposed model was implemented as a network of leaky
integrate-and-fire neurons to test the {\em analogy hypothesis}.
The previous proposal required specific delay and
temporal dynamics, and the implemented network tuned
accordingly functioned as predicted. Furthermore, these specific
conditions turn out to be consistent with experimental data, suggesting
that a further investigation of the thalamocortical circuit within the {\em
analogical framework} may be worthwhile.
Yoonsuck Choechoe2003-06-03Z2011-03-11T08:55:17Zhttp://cogprints.org/id/eprint/2997This item is in the repository with the URL: http://cogprints.org/id/eprint/29972003-06-03ZProcessing of analogy in the thalamocortical circuit The corticothalamic feedback and the thalamic reticular nucleus have gained
much attention lately because of their integrative and modulatory functions.
A previous study by the author suggested that
this circuitry can process analogies (i.e., the {\em analogy hypothesis}).
In this paper, the proposed model was implemented as a network of leaky
integrate-and-fire neurons to test the {\em analogy hypothesis}.
The previous proposal required specific delay and
temporal dynamics, and the implemented network tuned
accordingly functioned as predicted. Furthermore, these specific
conditions turn out to be consistent with experimental data, suggesting
that a further investigation of the thalamocortical circuit within the {\em
analogical framework} may be worthwhile.
Yoonsuck Choechoe2004-02-12Z2011-03-11T08:55:25Zhttp://cogprints.org/id/eprint/3340This item is in the repository with the URL: http://cogprints.org/id/eprint/33402004-02-12ZSparse visual models for biologically inspired sensorimotor controlGiven the importance of using resources efficiently in the competition for survival, it is reasonable to think that natural evolution has discovered efficient cortical coding strategies for representing natural visual information. Sparse representations have intrinsic advantages in terms of fault-tolerance and low-power consumption potential, and can therefore be attractive for robot sensorimotor control with powerful dispositions for decision-making. Inspired by the mammalian brain and its visual ventral pathway, we present in this paper a hierarchical sparse coding network architecture that extracts visual features for use in sensorimotor control. Testing with natural images demonstrates that this sparse coding facilitates processing and learning in subsequent layers. Previous studies have shown how the responses of complex cells could be sparsely represented by a higher-order neural layer. Here we extend sparse coding in each network layer, showing that detailed modeling of earlier stages in the visual pathway enhances the characteristics of the receptive fields developed in subsequent stages. The yield network is more dynamic with richer and more biologically plausible input and output representation.Li YangMarwan Jabri2004-02-12Z2011-03-11T08:55:25Zhttp://cogprints.org/id/eprint/3328This item is in the repository with the URL: http://cogprints.org/id/eprint/33282004-02-12ZSpeech Development by ImitationThe Double Cone Model (DCM) is a model
of how the brain transforms sensory input to
motor commands through successive stages of
data compression and expansion. We have
tested a subset of the DCM on speech recognition, production and imitation. The experiments show that the DCM is a good candidate
for an artificial speech processing system that
can develop autonomously. We show that the
DCM can learn a repertoire of speech sounds
by listening to speech input. It is also able to
link the individual elements of speech to sequences that can be recognized or reproduced,
thus allowing the system to imitate spoken
language.Bjorn BreidegardChristian Balkenius2008-06-13T00:08:41Z2011-03-11T08:57:07Zhttp://cogprints.org/id/eprint/6085This item is in the repository with the URL: http://cogprints.org/id/eprint/60852008-06-13T00:08:41ZTomographic Image Reconstruction of Fan-Beam Projections with Equidistant Detectors using Partially Connected Neural NetworksWe present a neural network approach for tomographic imaging problem using interpolation methods and fan-beam projections. This approach uses a partially connected neural network especially assembled for solving tomographic
reconstruction with no need of training. We extended the calculations to perform reconstruction with interpolation and to allow tomography of fan-beam geometry. The main goal is to aggregate speed while maintaining or improving the quality of the tomographic reconstruction process.Luciano Frontino de Medeiroslfm@egc.ufsc.brHamilton Pereira da Silvahsilva@facinter.brEduardo Parente Ribeiroedu@eletrica.ufpr.br2004-02-12Z2011-03-11T08:55:26Zhttp://cogprints.org/id/eprint/3351This item is in the repository with the URL: http://cogprints.org/id/eprint/33512004-02-12ZA Unified Model For Developmental RoboticsWe present the architecture and distributed
algorithms of an implemented system called
NeuSter, that unifies learning, perception and action
for autonomous robot control. NeuSter comprises
several sub-systems that provide online
learning for networks of million neurons on machine
clusters. It extracts information from sensors,
builds its own representations of the environment
in order to learn non-predefined goals.Williams PaquierNicolas Do HuuRaja Chatila2004-02-12Z2011-03-11T08:55:26Zhttp://cogprints.org/id/eprint/3353This item is in the repository with the URL: http://cogprints.org/id/eprint/33532004-02-12ZVisual binding, reentry, and neuronal synchrony in
a physically situated brain-based deviceBy constructing and analyzing a physically
situated brain-based device (i.e. a device
with sensors and actuators whose behavior
is guided by a simulated nervous system),
we show that reentrant connectivity and dynamic
synchronization can provide an effective
mechanism for binding the visual features
of objects.Anil K. SethJeffrey L. McKinstryGerald M. EdelmanJeffrey L. Krichmar2002-12-04Z2011-03-11T08:54:37Zhttp://cogprints.org/id/eprint/1456This item is in the repository with the URL: http://cogprints.org/id/eprint/14562002-12-04ZExplaining the nervous system in terms of computer programming and the object-class abstractionIt is argued that the key to understanding the brain is to view it as a device making extensive use of methodologies developed in computer programming, the idea of compiling source code written in a high-level language providing a mechanism for conceptually linking the two domains. Following the argument through, one arrives at a clarification of what the nervous system in its complexity is all about; it consists of a collection of devices for implementing specific kinds of competence, in ways in principle indicated in detail by application of the object-oriented programming paradigm to the various kinds of processes featuring in cognitive life. Brian D. Josephson2003-01-05Z2011-03-11T08:55:07Zhttp://cogprints.org/id/eprint/2687This item is in the repository with the URL: http://cogprints.org/id/eprint/26872003-01-05ZMISEP - Linear and Nonlinear ICA Based on Mutual InformationLinear Independent Components Analysis (ICA) has become an important signal processing and data analysis technique, the typical application being blind source separation in a wide range of signals, such as biomedical, acoustical and astrophysical ones. Nonlinear ICA is less developed, but has the potential to become at least as powerful.
This paper presents MISEP, an ICA technique for linear and nonlinear mixtures, which is based on the minimization of the mutual information of the estimated components. MISEP is a generalization of the popular INFOMAX technique, which is extended in two ways: (1) to deal with nonlinear mixtures, and (2) to be able to adapt to the actual statistical distributions of the sources, by dynamically estimating the nonlinearities to be used at the outputs. The resulting MISEP method optimizes a network with a specialized architecture, with a single objective function: the output entropy. Examples of both linear and nonlinear ICA performed by MISEP are presented in the paper.
Luis B. Almeida2002-09-17Z2011-03-11T08:55:00Zhttp://cogprints.org/id/eprint/2465This item is in the repository with the URL: http://cogprints.org/id/eprint/24652002-09-17ZSorting Methods in Self-Organization of Models and Clusterizations (Review of New Basic Ideas) - Iterative (Multirow) Polynomial GMDH AlgorithmsReview of the Group Method of Data Handling approachA.G. Ivakhnenko2003-10-04Z2011-03-11T08:55:03Zhttp://cogprints.org/id/eprint/2516This item is in the repository with the URL: http://cogprints.org/id/eprint/25162003-10-04ZAdaptivity through Physical ImmaturityGiven a neural control structure, what would be the impact of body growth on control performance? This question, which addresses the issue of the interaction between innate structure, ongoing developing structure and experience, is very relevant to the field of epigenetic robotics. Much of the early social interaction is done as the body develops and the interplay cannot be ignored. We hypothesize that starting with fewer degrees of freedom enables a more efficient exploration of the sensorimotor space, that results in multiple directions of stability. While not necessarily corresponding to optimal task performance, they will guide the coordination of additional degrees of freedom. These additional degrees of freedom then allow for optimal task performance as well as for more tolerance and adaptation to environmental interaction. We propose a simple case-study to validate our hypothesis and describe experiments with a small humanoid robot.Max LungarellaLuc Berthouze2002-10-22Z2011-03-11T08:55:05Zhttp://cogprints.org/id/eprint/2546This item is in the repository with the URL: http://cogprints.org/id/eprint/25462002-10-22ZCognitive mechanisms underlying the creative processThis paper proposes an explanation of the cognitive change that occurs as the creative process proceeds. During the initial, intuitive phase, each thought activates, and potentially retrieves information from, a large region containing many memory locations. Because of the distributed, content-addressable structure of memory, the diverse contents of these many locations merge to generate the next thought. Novel associations often result. As one focuses on an idea, the region searched and retrieved from narrows, such that the next thought is the product of fewer memory locations. This enables a shift from association-based to causation-based thinking, which facilitates the fine-tuning and manifestation of the creative work.
Liane Gabora2002-08-08Z2011-03-11T08:54:58Zhttp://cogprints.org/id/eprint/2381This item is in the repository with the URL: http://cogprints.org/id/eprint/23812002-08-08ZCortical region interactions and the functional role of apical dendritesThe basal and distal apical dendrites of pyramidal cells occupy distinct
cortical layers and are targeted by axons originating in different cortical
regions. Hence, apical and basal dendrites receive information from distinct
sources. Physiological evidence suggests that this anatomically observed
segregation of input sources may have functional significance. This possibility
has been explored in various connectionist models that employ neurons with
functionally distinct apical and basal compartments. A neuron in which separate
sets of inputs can be integrated independently has the potential to operate in a
variety of ways which are not possible for the conventional model of a neuron in
which all inputs are treated equally. This article thus considers how
functionally distinct apical and basal dendrites can contribute to the
information processing capacities of single neurons and, in particular, how
information from different cortical regions could have disparate affects on
neural activity and learning.
M. W. Spratling2003-01-07Z2011-03-11T08:55:08Zhttp://cogprints.org/id/eprint/2691This item is in the repository with the URL: http://cogprints.org/id/eprint/26912003-01-07ZFaster Training in Nonlinear ICA using MISEPMISEP has been proposed as a generalization of the INFOMAX method in two directions: (1) handling of nonlinear mixtures, and (2) learning the nonlinearities to be used at the outputs, making the method suitable to the separation of components with a wide range of statistical distributions. In all implementations up to now, MISEP had used multilayer perceptrons (MLPs) to perform the nonlinear ICA operation. Use of MLPs sometimes leads to a relatively slow training. This has been attributed, at least in part, to the non-local character of the MLP's units. This paper investigates the possibility of using a network of radial basis function (RBF) units for performing the nonlinear ICA operation. It shows that the local character of the RBF network's units allows a significant speedup in the training of the system. The paper gives a brief introduction to the basics of the MISEP method, and presents experimental results showing the speed advantage of using an RBF-based network to perform the ICA operation.Luis B. Almeida2003-09-26Z2011-03-11T08:55:03Zhttp://cogprints.org/id/eprint/2500This item is in the repository with the URL: http://cogprints.org/id/eprint/25002003-09-26ZFrom Visuo-Motor Development to Low-level ImitationWe present the first stages of the developmental course of a robot using vision and a 5 degree of freedom robotic arm. During an exploratory behavior, the robot learns visuo-motor control of its mechanical arm. We show how a simple neural network architecture, combining elementary vision, a self-organized algorithm, and dynamical Neural Fields is able to learn and use proper associations between vision and arm movements, even if the problem is ill posed (2-D toward 3-D mapping and also mechanical redundancy between different joints). Highlighting the generic aspect of such an architecture, we show as a robotic result that it is used as a basis for simple gestural imitations of humans. Finally we show how the imitative mechanism carries on the developmental course, allowing the acquisition of more and more complex behavioral capabilities.Pierre AndryPhilippe GaussierJacqueline Nadel2002-07-18Z2011-03-11T08:54:57Zhttp://cogprints.org/id/eprint/2331This item is in the repository with the URL: http://cogprints.org/id/eprint/23312002-07-18ZAn improved 2D optical flow sensor for motion segmentation A functional focal-plane implementation of a 2D optical flow system is presented that detects an
preserves motion discontinuities. The system is composed of two different network layers of analog
computational units arranged in a retinotopical order. The units in the first layer (the optical
flow network) estimate the local optical flow field in two visual dimensions, where the strength
of their nearest-neighbor connections determines the amount of motion integration. Whereas in an
earlier implementation \cite{Stocker_Douglas99} the connection strength was set constant in the
complete image space, it is now \emph{dynamically and locally} controlled by the second network
layer (the motion discontinuities network) that is recurrently connected to the optical flow
network. The connection strengths in the optical flow network are modulated such that visual
motion integration is ideally only facilitated within image areas that are likely to represent
common
motion sources.
Results of an experimental aVLSI chip illustrate the potential of the approach and its
functionality under real-world conditions.alan stocker2006-12-03Z2011-03-11T08:56:43Zhttp://cogprints.org/id/eprint/5273This item is in the repository with the URL: http://cogprints.org/id/eprint/52732006-12-03ZIntensity generalisation: physiology and modelling of a neglected topicI briefly review empirical data about the generalisation of acquired behaviour to novel stimuli, showing that variations in stimulus intensity affect behaviour differently from variations in characteristics such as, for instance, visual shape or sound frequency. I argue that such differences can be seen already in how the sense organs react to changes in intensity compared to changes in other stimulus characteristics. I then evaluate a number
of models of generalisation with respect to their ability to reproduce intensity generalisation. I reach three main conclusions. First, realistic stimulus representations, based on knowledge of the sense organs, are necessary to
account for intensity effects. Models employing stimulus representations too remote from the sense organs are unable to reproduce the data. Second, the intuitive notion that generalisation is based on similarities between stimuli, possibly modelled as distances in an appropriate representation space, is difficult to reconcile with data about intensity generalisation. Third, several simple models, in conjunction with realistic stimulus representations, can account for a wide array of generalisation phenomena along both intensity and non-intensity stimulus dimensions. The paper also introduces concepts which may be generally useful to evaluate and compare different models of behaviour.
Stefano Ghirlanda2002-08-27Z2011-03-11T08:54:59Zhttp://cogprints.org/id/eprint/2431This item is in the repository with the URL: http://cogprints.org/id/eprint/24312002-08-27ZModeling Directional Selectivity Using Self-Organizing Delay-Aadaptation MapsUsing a delay adaptation learning rule, we model the activity-dependent development of directionally selective cells in the primary visual cortex. Based on input stimuli, a learning rule shifts delays to create synchronous arrival of spikes at cortical cells. As a result, delays become tuned creating a smooth cortical map of direction selectivity. This result demonstrates how delay adaption can serve as a powerful abstraction for modeling temporal learning in the brain.
Tal TverskytaltverskyDr. Risto Miikkulainen2002-08-23Z2011-03-11T08:54:59Zhttp://cogprints.org/id/eprint/2422This item is in the repository with the URL: http://cogprints.org/id/eprint/24222002-08-23ZModeling Directional Selectivity Using Self-Organizing Delay-Aadaptation MapsUsing a delay adaptation learning rule, we model the activity-dependent development of directionally selective cells in the primary visual cortex. Based on input stimuli, a learning rule shifts delays to create synchronous arrival of spikes at cortical cells. As a result, delays become tuned creating a smooth cortical map of direction selectivity. This result demonstrates how delay adaption can serve as a powerful abstraction for modeling temporal learning in the brain.
Mr. Tal TverskytaltverskyDr. Risto Miikkulainen2002-06-20Z2011-03-11T08:54:56Zhttp://cogprints.org/id/eprint/2287This item is in the repository with the URL: http://cogprints.org/id/eprint/22872002-06-20ZA Neural Model of Corticocerebellar Interactions During Attentive Imitation And Predictive Learning Of Sequential Handwriting Movements
A NEURAL MODEL OF CORTICOCEREBELLAR INTERACTIONS DURING
ATTENTIVE IMITATION AND PREDICTIVE LEARNING OF SEQUENTIAL
HANDWRITING MOVEMENTS
RAINER WALTER PAINE
Boston University Graduate School of Arts and Sciences, 2002
Major Professor: Stephen Grossberg, Wang Professor of Cognitive and Neural Systems
ABSTRACT
Much sensory-motor behavior develops through imitation, as during the learning of
handwriting by children. Such complex sequential acts are broken down into distinct
motor control synergies, or muscle groups, whose activities overlap in time to generate
continuous, curved movements that obey an inverse relation between curvature and speed.
How are such complex movements learned through attentive imitation? Novel movements
may be made as a series of distinct segments, but a practiced movement can be made
smoothly, with a continuous, often bell-shaped, velocity profile. How does learning of
complex movements transform reactive imitation into predictive, automatic performance?
A neural model is developed which suggests how parietal and motor cortical mechanisms,
such as difference vector encoding, interact with adaptively-timed, predictive cerebellar
learning during movement imitation and predictive performance. To initiate
movement, visual attention shifts along the shape to be imitated and generates vector
movement using motor cortical cells. During such an imitative movement, cerebellar
Purkinje cells with a spectrum of delayed response profiles sample and learn the changing
directional information and, in turn, send that learned information back to the cortex and
eventually to the muscle synergies involved. If the imitative movement deviates from an
attentional focus around a shape to be imitated, the visual system shifts attention, and may
saccade, back to the shape, thereby providing corrective directional information to the arm
movement system. This imitative movement cycle repeats until the corticocerebellar system
can accurately drive the movement based on memory alone.
A cortical working memory buffer transiently stores the cerebellar output and releases
it at a variable rate, allowing speed scaling of learned movements which is limited by the
rate of cerebellar memory readout. Movements can be learned at variable speeds if the
density of the spectrum of delayed cellular responses in the cerebellum varies with speed.
Learning at slower speeds facilitates learning at faster speeds. Size can be varied after
learning while keeping the movement duration constant. Context effects arise from the
overlap of cerebellar memory outputs. The model is used to simulate key psychophysical
and neural data about learning to make curved movements.
Rainer W. Paine2003-03-12Z2011-03-11T08:55:07Zhttp://cogprints.org/id/eprint/2658This item is in the repository with the URL: http://cogprints.org/id/eprint/26582003-03-12ZPhonemic Coding Might Result From
Sensory-Motor Coupling DynamicsHuman sound systems are invariably phonemically coded. Furthermore,
phoneme inventories follow very particular tendancies. To explain
these phenomena, there existed so far three kinds of approaches :
``Chomskyan''/cognitive innatism, morpho-perceptual innatism
and the more recent approach of ``language as a complex cultural system
which adapts under the pressure of efficient communication''.
The two first approaches are clearly not satisfying, while
the third, even if much more convincing,
makes a lot of speculative assumptions and did not
really bring answers to the question of phonemic coding. We propose
here a new hypothesis based on a low-level model of
sensory-motor interactions. We show that certain very
simple and non language-specific neural devices
allow a population of agents to build signalling systems
without any functional pressure. Moreover, these systems
are phonemically coded. Using a realistic vowel articulatory
synthesizer, we show that the inventories of vowels
have striking similarities with human vowel systems.Pierre-Yves Oudeyer2002-08-08Z2011-03-11T08:54:58Zhttp://cogprints.org/id/eprint/2380This item is in the repository with the URL: http://cogprints.org/id/eprint/23802002-08-08ZPre-integration lateral inhibition enhances unsupervised learningA large and influential class of neural network architectures use
post-integration lateral inhibition as a mechanism for competition. We argue
that these algorithms are computationally deficient in that they fail to
generate, or learn, appropriate perceptual representations under certain
circumstances. An alternative neural network architecture is presented in which
nodes compete for the right to receive inputs rather than for the right to
generate outputs. This form of competition, implemented through pre-integration
lateral inhibition, does provide appropriate coding properties and can be used
to efficiently learn such representations. Furthermore, this architecture is
consistent with both neuro-anatomical and neuro-physiological data. We thus
argue that pre-integration lateral inhibition has computational advantages over
conventional neural network architectures while remaining equally biologically
plausible.M. W. SpratlingM. H. Johnson2003-02-09Z2011-03-11T08:54:56Zhttp://cogprints.org/id/eprint/2281This item is in the repository with the URL: http://cogprints.org/id/eprint/22812003-02-09ZSecond order isomorphism: A reinterpretation and its implications in brain and cognitive sciencesShepard and Chipman's second order isomorphism describes how
the brain may represent the relations in the world.
However, a common interpretation of the theory can cause difficulties.
The problem originates from the static nature
of representations. In an alternative interpretation, I propose that
we assign an active role to the internal representations and
relations. It turns out that a collection of such active units can
perform analogical tasks. The new interpretation is supported
by the existence of neural circuits that may be implementing such a function.
Within this framework, perception, cognition, and motor function
can be understood under a unifying principle of analogy.
Yoonsuck Choe2006-12-08Z2011-03-11T08:56:43Zhttp://cogprints.org/id/eprint/5277This item is in the repository with the URL: http://cogprints.org/id/eprint/52772006-12-08ZSpectacular pehnomena and limits to rationality in genetic and cultural evolutionIn studies of both animal and human behaviour, game theory is used as a tool for understanding strategies that appear in interactions between individuals. Game theory focuses on adaptive behaviour, which can be attained only at evolutionary equilibrium. Here we suggest that behaviour appearing during interactions is often outside the scope of such analysis. In many types of interaction, conflicts of interest exist between players, fueling the evolution of manipulative strategies. Such strategies evolve out of equilibrium, commonly appearing as spectacular morphology or behaviour with obscure meaning, to which other players may react in non-adaptive, irrational way approach, and outline the conditions in which evolutionary equilibria cannot be maintained. Evidence from studies of biological interactions seems to support the view that behaviour is often not at equilibrium. This also appears to be the case for many human cultural traits, which have spread rapidly despite the fact that they have a negative influence on reproduction.Magnus EnquistAnthony ArakStefano GhirlandaCarl-Adam Wachtmeister2002-02-18Z2011-03-11T08:54:53Zhttp://cogprints.org/id/eprint/2089This item is in the repository with the URL: http://cogprints.org/id/eprint/20892002-02-18ZTowards Incremental Parsing of Natural Language using Recursive Neural NetworksIn this paper we develop novel algorithmic ideas for building a natural language
parser grounded upon the hypothesis of incrementality. Although widely accepted
and experimentally supported under a cognitive perspective as a model of the human
parser, the incrementality assumption has never been exploited for building automatic
parsers of unconstrained real texts. The essentials of the hypothesis are that words are
processed in a left-to-right fashion, and the syntactic structure is kept totally connected
at each step.
Our proposal relies on a machine learning technique for predicting the correctness of
partial syntactic structures that are built during the parsing process. A recursive neural
network architecture is employed for computing predictions after a training phase on
examples drawn from a corpus of parsed sentences, the Penn Treebank. Our results
indicate the viability of the approach andlay out the premises for a novel generation of
algorithms for natural language processing which more closely model human parsing.
These algorithms may prove very useful in the development of eÆcient parsers.Fabrizio CostaPaolo FrasconiVincenzo LombardoGiovanni Soda2003-09-19Z2011-03-11T08:55:20Zhttp://cogprints.org/id/eprint/3151This item is in the repository with the URL: http://cogprints.org/id/eprint/31512003-09-19ZIntelligent systems in the context of surrounding environmentWe investigate the behavioral patterns of a population of agents, each controlled by a simple biologically motivated neural network model, when they are set in competition against each other in the Minority Model of Challet and Zhang. We explore the effects of changing agent characteristics, demonstrating that crowding behavior takes place among agents of similar memory, and show how this allows unique `rogue' agents with higher memory values to take advantage of a majority population. We also show that agents' analytic capability is largely determined by the size of the intermediary layer of neurons.
In the context of these results, we discuss the general nature of natural and artificial intelligence systems, and suggest intelligence only exists in the context of the surrounding environment (embodiment).
Source code for the programs used can be found at http://neuro.webdrake.net/.Joseph WakelingJWakelingPer Bak2005-02-01Z2011-03-11T08:55:49Zhttp://cogprints.org/id/eprint/4043This item is in the repository with the URL: http://cogprints.org/id/eprint/40432005-02-01ZIntelligent systems in the context of surrounding environmentWe investigate the behavioral patterns of a population of agents, each controlled by a simple biologically motivated neural network model, when they are set in competition against each other in the Minority Model of Challet and Zhang. We explore the effects of changing agent characteristics, demonstrating that crowding behavior takes place among agents of similar memory, and show how this allows unique `rogue' agents with higher memory values to take advantage of a majority population. We also show that agents' analytic capability is largely determined by the size of the intermediary layer of neurons.
In the context of these results, we discuss the general nature of natural and artificial intelligence systems, and suggest intelligence only exists in the context of the surrounding environment (embodiment).
Source code for the programs used can be found at http://neuro.webdrake.net/.Joseph WakelingJWakelingPer Bak2001-11-19Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1904This item is in the repository with the URL: http://cogprints.org/id/eprint/19042001-11-19ZRepresentation and Extrapolation in Multi-Layer PerceptronsTo give an adequate explanation of cognition and perform certain practical tasks connectionist systems must be able to extrapolate. This work has explored the relationship between input representation and extrapolation, using simulations of multi-layer perceptrons trained to model the identity function. It has been discovered that representation has a marked effect on extrapolation. Antony Browne2012-11-09T19:34:57Z2012-11-09T19:34:57Zhttp://cogprints.org/id/eprint/8082This item is in the repository with the URL: http://cogprints.org/id/eprint/80822012-11-09T19:34:57ZDopaminergic Regulation of Neuronal Circuits in Prefrontal CortexNeuromodulators, like dopamine, have considerable influence on the
processing capabilities of neural networks.
This has for instance been shown in the working memory functions
of prefrontal cortex, which may be regulated by altering the
dopamine level. Experimental work provides evidence on the biochemical
and electrophysiological actions of dopamine receptors, but there are few
theories concerning their significance for computational properties
(ServanPrintzCohen90,Hasselmo94).
We point to experimental data on neuromodulatory regulation of
temporal properties of excitatory neurons and depolarization of inhibitory
neurons, and suggest computational models employing these effects.
Changes in membrane potential may be modelled by the firing threshold,
and temporal properties by a parameterization of neuronal responsiveness
according to the preceding spike interval.
We apply these concepts to two examples using spiking neural networks.
In the first case, there is a change in the input synchronization of
neuronal groups, which leads to
changes in the formation of synchronized neuronal ensembles.
In the second case, the threshold
of interneurons influences lateral inhibition, and the switch from a
winner-take-all network to a parallel feedforward mode of processing.
Both concepts are interesting for the modeling of cognitive functions and may
have explanatory power for behavioral changes associated with dopamine
regulation.Gabriele Schelergscheler@gmail.com2003-07-25Z2011-03-11T08:55:19Zhttp://cogprints.org/id/eprint/3082This item is in the repository with the URL: http://cogprints.org/id/eprint/30822003-07-25ZVisualization of Data by Method of Elastic Maps and Its Applications in Genomics, Economics and Sociology Technology of data visualization and data modeling is suggested. The basic of the technology is original idea of elastic net and methods of its construction and application. A short review of relevant methods has been made. The methods proposed are illustrated by applying them to the real economical, sociological and biological datasets and to some model data distributions.
The basic of the technology is original idea of elastic net - regular point approximation of some manifold that is put into the multidimensional space and has in a certain sense minimal energy. This manifold is an analogue of principal surface and serves as non-linear screen on what multidimensional data are projected.
Remarkable feature of the technology is its ability to work with and to fill gaps in data tables. Gaps are unknown or unreliable values of some features. It gives a possibility to predict plausibly values of unknown features by values of other ones. So it provides technology of constructing different prognosis systems and non-linear regressions.
The technology can be used by specialists in different fields. There are several examples of applying the method presented in the end of this paper.
Prof. Alexander. N. GorbanDr. Andrei Yu. Zinovyev2003-08-08Z2011-03-11T08:55:19Zhttp://cogprints.org/id/eprint/3088This item is in the repository with the URL: http://cogprints.org/id/eprint/30882003-08-08ZVisualization of Data by Method of Elastic Maps and Its Applications in Genomics, Economics and Sociology Technology of data visualization and data modeling is suggested. The basic of the technology is original idea of elastic net and methods of its construction and application. A short review of relevant methods has been made. The methods proposed are illustrated by applying them to the real economical, sociological and biological datasets and to some model data distributions.
The basic of the technology is original idea of elastic net - regular point approximation of some manifold that is put into the multidimensional space and has in a certain sense minimal energy. This manifold is an analogue of principal surface and serves as non-linear screen on what multidimensional data are projected.
Remarkable feature of the technology is its ability to work with and to fill gaps in data tables. Gaps are unknown or unreliable values of some features. It gives a possibility to predict plausibly values of unknown features by values of other ones. So it provides technology of constructing different prognosis systems and non-linear regressions.
The technology can be used by specialists in different fields. There are several examples of applying the method presented in the end of this paper.
Prof. Alexander. N. GorbanDr. Andrei Yu. Zinovyev2001-04-10Z2011-03-11T08:54:37Zhttp://cogprints.org/id/eprint/1441This item is in the repository with the URL: http://cogprints.org/id/eprint/14412001-04-10ZAssociative Neural NetworkAn associative neural network (ASNN) is a combination of an ensemble of the feed-forward neural networks and the K-nearest neighbor technique. The introduced network uses correlation between ensemble responses as a measure of distance amid the analyzed cases for the nearest neighbor technique and provides an improved prediction by the bias correction of the neural network ensemble. An associative neural network has a memory that can coincide with the training set. If new data become available, the network further improves its predicting ability and can often provide a reasonable approximation of the unknown function without a need to retrain the neural network ensemble.Igor Tetko2001-01-16Z2011-03-11T08:54:29Zhttp://cogprints.org/id/eprint/1240This item is in the repository with the URL: http://cogprints.org/id/eprint/12402001-01-16ZBinding and Normalization of Binary Sparse Distributed Representations by Context-Dependent ThinningDistributed representations were often criticized as inappropriate for encoding of data with a complex structure. However Plate's Holographic Reduced Representations and Kanerva's Binary Spatter Codes are recent schemes that allow on-the-fly encoding of nested compositional structures by real-valued or dense binary vectors of fixed dimensionality.
In this paper we consider procedures of the Context-Dependent Thinning which were developed for representation of complex hierarchical items in the architecture of Associative-Projective Neural Networks. These procedures provide binding of items represented by sparse binary codevectors (with low probability of 1s). Such an encoding is biologically plausible and allows a high storage capacity of distributed associative memory where the codevectors may be stored.
In contrast to known binding procedures, Context-Dependent Thinning preserves the same low density (or sparseness) of the bound codevector for varied number of component codevectors. Besides, a bound codevector is not only similar to another one with similar component codevectors (as in other schemes), but it is also similar to the component codevectors themselves. This allows the similarity of structures to be estimated just by the overlap of their codevectors, without retrieval of the component codevectors. This also allows an easy retrieval of the component codevectors.
Examples of algorithmic and neural-network implementations of the thinning procedures are considered. We also present representation examples for various types of nested structured data (propositions using role-filler and predicate-arguments representation schemes, trees, directed acyclic graphs) using sparse codevectors of fixed dimension. Such representations may provide a fruitful alternative to the symbolic representations of traditional AI, as well as to the localist and microfeature-based connectionist representations.
Dmitri A. RachkovskijErnst M. Kussul2002-01-16Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2036This item is in the repository with the URL: http://cogprints.org/id/eprint/20362002-01-16ZThe adaptive advantage of symbolic theft over sensorimotor toil: Grounding language in perceptual categoriesUsing neural nets to simulate learning and the genetic algorithm to simulate evolution in a toy world of mushrooms and mushroom-foragers, we place two ways of acquiring categories into direct competition with one another: In (1) "sensorimotor toil,” new categories are acquired through real-time, feedback-corrected, trial and error experience in sorting them. In (2) "symbolic theft,” new categories are acquired by hearsay from propositions – boolean combinations of symbols describing them. In competition, symbolic theft always beats sensorimotor toil. We hypothesize that this is the basis of the adaptive advantage of language. Entry-level categories must still be learned by toil, however, to avoid an infinite regress (the “symbol grounding problem”). Changes in the internal representations of categories must take place during the course of learning by toil. These changes can be analyzed in terms of the compression of within-category similarities and the expansion of between-category differences. These allow regions of similarity space to be separated, bounded and named, and then the names can be combined and recombined to describe new categories, grounded recursively in the old ones. Such compression/expansion effects, called "categorical perception" (CP), have previously been reported with categories acquired by sensorimotor toil; we show that they can also arise from symbolic theft alone. The picture of natural language and its origins that emerges from this analysis is that of a powerful hybrid symbolic/sensorimotor capacity, infinitely superior to its purely sensorimotor precursors, but still grounded in and dependent on them. It can spare us from untold time and effort learning things the hard way, through direct experience, but it remain anchored in and translatable into the language of experience.Angelo CangelosiStevan Harnad2002-06-10Z2011-03-11T08:54:56Zhttp://cogprints.org/id/eprint/2249This item is in the repository with the URL: http://cogprints.org/id/eprint/22492002-06-10ZAttentional and Semantic AnticipationsWhy are attentional processes important in the driving of anticipations? Anticipatory processes are fundamental cognitive abilities of living systems, in order to rapidly and accurately perceive new events in the environment, and to trigger adapted behaviors to the newly perceived events. To process anticipations adapted to sequences of various events in complex environments, the cognitive system must be able to run specific anticipations on the basis of selected relevant events. Then more attention must be given to events potentially relevant for the living system, compared to less important events.
What are useful attentional factors in anticipatory processes? The relevance of events in the environment depend on the effects they can have on the survival of the living system. The cognitive system must then be able to detect relevant events to drive anticipations and to trigger adapted behaviors. The attention given to an event depends on i) its external physical relevance in the environment, such as time duration and visual quality, and ii) on its internal semantic relevance in memory, such as knowledge about the event (semantic field in memory) and anticipatory power (associative strength to anticipated associates).
How can we model interactions between attentional and semantic anticipations? Specific types of distributed recurrent neural networks are able to code temporal sequences of events as associated attractors in memory. Particular learning protocol and spike rate transmission through synaptic associations allow the model presented to vary attentionally the amount of activation of anticipations (by activation or inhibition processes) as a function of the external and internal relevance of the perceived events. This type of model offers a unique opportunity to account for both anticipations and attention in unified terms of neural dynamics in a recurrent network.
Frédéric LavigneSylvain Denis2004-10-08Z2011-03-11T08:55:42Zhttp://cogprints.org/id/eprint/3865This item is in the repository with the URL: http://cogprints.org/id/eprint/38652004-10-08ZConnecting adaptive behaviour and expectations in models of innovation: The Potential Role of Artificial Neural NetworksIn this methodological work I explore the possibility of explicitly modelling expectations conditioning the R&D decisions of firms. In order to isolate this problem from the controversies of cognitive science, I propose a black box strategy through the concept of “internal model”. The last part of the article uses artificial neural networks to model the expectations of firms in a model of industry dynamics based on Nelson & Winter (1982).Murat Yildizogluyildi2001-09-04Z2011-03-11T08:54:47Zhttp://cogprints.org/id/eprint/1788This item is in the repository with the URL: http://cogprints.org/id/eprint/17882001-09-04ZConnectionist Inference ModelsThe performance of symbolic inference tasks has long been a challenge to connectionists. In this paper, we present an extended survey of this area. Existing connectionist inference systems are reviewed, with particular reference to how they perform variable binding and rule-based reasoning, and whether they involve distributed or localist representations. The benefits and disadvantages of different representations and systems are outlined, and conclusions drawn regarding the capabilities of connectionist inference systems when compared with symbolic inference systems or when used for cognitive modeling.
Antony BrowneRon Sun2002-08-08Z2011-03-11T08:54:58Zhttp://cogprints.org/id/eprint/2379This item is in the repository with the URL: http://cogprints.org/id/eprint/23792002-08-08ZDendritic inhibition enhances neural coding properties.The presence of a large number of inhibitory contacts at the soma and axon
initial segment of cortical pyramidal cells has inspired a large and influential
class of neural network model which use post-integration lateral inhibition as a
mechanism for competition between nodes. However, inhibitory synapses also
target the dendrites of pyramidal cells. The role of this dendritic inhibition
in competition between neurons has not previously been addressed. We
demonstrate, using a simple computational model, that such pre-integration
lateral inhibition provides networks of neurons with useful representational and
computational properties which are not provided by post-integration
inhibition.
M. W SpratlingM. H. Johnson2002-01-11Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2016This item is in the repository with the URL: http://cogprints.org/id/eprint/20162002-01-11ZEvolution of communication and language using signals, symbols and wordsThis paper describes different types of models for the evolution of communication and language. It uses the distinction between signals, symbols, and words for the analysis of evolutionary models of language. In particular, it show how evolutionary computation techniques, such as artificial life, can be used to study the emergence of syntax and symbols from simple communication signals. Initially, a computational model that evolves repertoires of isolated signals is presented. This study has simulated the emergence of signals for naming foods in a population of foragers. This type of model studies communication systems based on simple signal-object associations. Subsequently, models that study the emergence of grounded symbols are discussed in general, including a detailed description of a work on the evolution of simple syntactic rules. This model focuses on the emergence of symbol-symbol relationships in evolved languages. Finally, computational models of syntax acquisition and evolution are discussed. These different types of computational models provide an operational definition of the signal/symbol/word distinction. The simulation and analysis of these types of models will help to understand the role of symbols and symbol acquisition in the origin of language.Angelo Cangelosi2001-02-10Z2011-03-11T08:54:30Zhttp://cogprints.org/id/eprint/1298This item is in the repository with the URL: http://cogprints.org/id/eprint/12982001-02-10ZEvolving modular architectures for neural networksNeural networks that learn the What and Where task perform better if they possess a modular architecture for separately processing the identity and spatial location of objects. In previous simulations the modular architecture either was hardwired or it developed during an individual's life based on a preference for short connections given a set of hardwired unit locations. We present two sets of simulations in which the network architecture is genetically inherited and it evolves in a population of neural networks in two different conditions: (1) both the architecture and the connection weights evolve; (2) the network architecture is inherited and it evolves but the connection weights are learned during life. The best results are obtained in condition (2). Condition (1) gives unsatisfactory results because (a) adapted sets of weights can suddenly become maladaptive if the architecture changes, (b) evolution fails to properly assign computational resources (hidden units) to the two tasks, (c) genetic linkage between sets of weights for different modules can result in a favourable mutation in one set of weights being accompanied by an unfavourable mutation in another set of weights.
Andrea Di FerdinandoRaffaele CalabrettaDomenico Parisi2002-01-11Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2020This item is in the repository with the URL: http://cogprints.org/id/eprint/20202002-01-11ZHow nouns and verbs differentially affect the behavior of artificial organismsThis paper presents an Artificial Life and Neural Network (ALNN) model for the evolution of syntax. The simulation methodology provides a unifying approach for the study of the evolution of language and its interaction with other behavioral and neural factors. The model uses an object manipulation task to simulate the evolution of language based on a simple verb-noun rule. The analyses of results focus on the interaction between language and other non-linguistic abilities, and on the neural control of linguistic abilities. The model shows that the beneficial effects of language on non-linguistic behavior are explained by the emergence of distinct internal representation patterns for the processing of verbs and nouns.Angelo CangelosiDomenico Parisi2002-01-11Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2019This item is in the repository with the URL: http://cogprints.org/id/eprint/20192002-01-11ZA Hybrid Neural Network and Virtual Reality System for Spatial Language ProcessingThis paper describes a neural network model for the study of spatial language. It deals with both geometric and functional variables, which have been shown to play an important role in the comprehension of spatial prepositions. The network is integrated with a virtual reality interface for the direct manipulation of geometric and functional factors. The training uses experimental stimuli and data. Results show that the networks reach low training and generalization errors. Cluster analyses of hidden activation show that stimuli primarily group according to extra-geometrical variables.Guillermina MartinezAngelo CangelosiKenny Coventry2001-06-19Z2011-03-11T08:54:42Zhttp://cogprints.org/id/eprint/1624This item is in the repository with the URL: http://cogprints.org/id/eprint/16242001-06-19ZNo Easy Way OutThe mind/body problem is the feeling/function problem: How and why do
feeling systems feel? The problem is not just "hard" but insoluble (unless one
is ready to resort to telekinetic dualism). Fortunately, the "easy" problems of
cognitive science (such as the how and why of categorization and language)
are not insoluble. Five books (by Damasio, Edelman/Tononi, McGinn,
Tomasello and Fodor) are reviewed in this context.Stevan Harnad2003-04-24Z2011-03-11T08:55:15Zhttp://cogprints.org/id/eprint/2906This item is in the repository with the URL: http://cogprints.org/id/eprint/29062003-04-24ZScientific Models, Connectionist Networks, and Cognitive ScienceThe employment of a particular class of computer programs known as "connectionist networks" to model mental processes is a widespread approach to research in
cognitive science these days. Little has been written, however, on the precise connection that is thought to hold between such programs and actual in vivo cognitive
processes such that the former can be said to "model" the latter in a scientific sense. What is more, this relation can be shown to be problematic. In this paper I give
a brief overview of the use of connectionist models in cognitive science, and then explore some of the statements connectionists have made about the nature of the
"modeling relation" thought to hold between them and cognitive processes. Finally I show that these accounts are inadequate and that more work is necessary if
connectionist networks are to be seriously regarded as scientific models of cognitive processesChristopher D. Green2001-11-23Z2011-03-11T08:54:50Zhttp://cogprints.org/id/eprint/1914This item is in the repository with the URL: http://cogprints.org/id/eprint/19142001-11-23ZA self-organizing neural network model of the acquisition of word meaningIn this paper we present a self-organizing connectionist model of the acquisition of word meaning. Our model consists of two neural networks and builds on the basic concepts of Hebbian learning and self-organization. One network learns to approximate word transition probabilities, which are used for lexical representation, and the other network, a self-organizing map, is trained on these representations, projecting them onto a 2D space. The model relies on lexical co-occurrence information to represent word meanings in the lexicon. The results show that our model is able to acquire semantic representations from both artificial data and real corpus of language use. In addition, the model demonstrates the ability to develop rather accurate word representations even with a sparse training set.
Igor FarkasPing Li2001-11-22Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1910This item is in the repository with the URL: http://cogprints.org/id/eprint/19102001-11-22ZSemantic Effect on Episodic AssociationsWe examined the influence of the pre-existing organiza-tion of the semantic memory on forming new episodic associations between words. Testing human subjects' performance we found that a semantic relationship be-tween words facilitates forming episodic associations be-tween them. Furthermore, the amount of facilitation in-creases linearly as a function of the number of co-occurrence of the words, up to a ceiling. Constrained by these empirical findings we developed a computational model, based on the theory of spreading activation over semantic networks. The model uses self-organizing maps to represent semantic relatedness, and lateral connections to represent the episodic associations. When two words are presented to the model, the interaction of the two ac-tivation waves is summed and added to the direct lateral connection between them. The main result is that the model is capable of replicating the empirical results. The model also makes several testable predictions: First, it should be easier to form an association from a word with few semantic neighbors to a word with many se-mantic neighbors than vice-versa. Second, after associat-ing an unrelated word pair it should be easier to associate another two words each related to one of the words in the first pair. Third, a less focused activation wave, which may be the cause of schizophrenic thought disor-der, should decrease the advantage in learning rate of re-lated over unrelated pairs.Yaron SilbermanRisto MiikkulainenShlomo Bentin2002-07-16Z2011-03-11T08:54:57Zhttp://cogprints.org/id/eprint/2324This item is in the repository with the URL: http://cogprints.org/id/eprint/23242002-07-16ZWhat does it take to evolve behaviorally complex organisms?What genotypic features explain the evolvability of organisms that have to accomplish many different tasks? The genotype of behaviorally complex organisms may be more likely to encode modular neural architectures because neural modules dedicated to distinct tasks avoid neural interference, i.e., the arrival of conflicting messages for changing the value of connection weights during learning. However, if the connection weights for the various modules are genetically inherited, this raises the problem of genetic linkage: favorable mutations may fall on one portion of the genotype encoding one neural module and unfavorable mutations on another portion encoding another module. We show that this can prevent the genotype from reaching an adaptive optimum. This effect is different from other linkage effects described in the literature and we argue that it represents a new class of genetic constraints. Using simulations we show that sexual reproduction can alleviate the problem of genetic linkage by recombining separate modules all of which incorporate either favorable or unfavorable mutations. We speculate that this effect may contribute to the taxonomic prevalence of sexual reproduction among higher organisms. In addition to sexual recombination, the problem of genetic linkage for behaviorally complex organisms may be mitigated by entrusting evolution with the task of finding appropriate modular architectures and learning with the task of finding the appropriate connection weights for these architectures.Raffaele CalabrettaAndrea Di FerdinandoGünter P. WagnerDomenico Parisi2001-02-07Z2011-03-11T08:54:29Zhttp://cogprints.org/id/eprint/1287This item is in the repository with the URL: http://cogprints.org/id/eprint/12872001-02-07ZBuilding large-scale hierarchical models of the world with binary sparse distributed representationsMany researchers agree on the basic architecture of the "world model" where knowledge about the world required for organization of agent's intelligent behavior is represented. However, most proposals on possible implementation of such a model are far from being plausible both from computational and neurobiological points of view.
Implementation ideas based on distributed connectionist representations offer a huge information capacity, flexibility of similarity representation, and possibility to use a distributed neural network memory. However, for a long time distributed representations suffered from the "superposition catastrophe". Local representations are vivid, pictorial and easily interpretable, allow for an easy manual construction of hierarchical structures and an economical computer simulation of toy tasks. The problems of local representations show up with scaling to the real world models, and it is unclear how to solve them under reasonable requirements imposed on memory size and speed.
We discuss the architecture of Associative-Projective Neural Networks (APNNs) that is based on binary sparse distributed representations of fixed dimensionality for items of various complexity and generality, and provides a promise for scaling up to the full-sized model of the real world. An on-the-fly binding procedure proposed for APNNs overcomes the superposition catastrophe, permitting representation of the order and grouping of structure components. These representations allow a simple estimation of structures' similarity, as well as finding various kinds of associations based on their context-dependent similarity. Structured distributed auto-associative neural network is used as long-term memory, wherein representations of items organized into part-whole (compositional) and concept (generalization) hierarchies are built. Examples of schematic APNN architectures and processes for recognition, prediction, reaction, analogical reasoning, and other tasks required for functioning of an intelligent system, as well as APNN implementations, are considered.
Dmitri A. RachkovskijErnst M. Kussul2001-11-23Z2011-03-11T08:54:50Zhttp://cogprints.org/id/eprint/1915This item is in the repository with the URL: http://cogprints.org/id/eprint/19152001-11-23ZTilt Aftereffects in a Self-Organizing Model of the Primary Visual CortexRF-LISSOM, a self-organizing model of laterally connected orientation maps in the primary visual cortex, was used to study the psychological phenomenon known as the tilt aftereffect. The same self-organizing processes that are responsible for the long-term development of the map are shown to result in tilt aftereffects over short time scales in the adult. The model permits simultaneous observation of large numbers of neurons and connections, making it possible to relate high-level phenomena to low-level events, which is difficult to do experimentally. The results give detailed computational support for the long-standing conjecture that the direct tilt aftereffect arises from adaptive lateral interactions between feature detectors. They also make a new prediction that the indirect effect results from the normalization of synaptic efficacies during this process. The model thus provides a unified computational explanation of self-organization and both the direct and indirect tilt aftereffect in the primary visual cortex.James A. BednarRisto Miikkulainen2002-06-10Z2011-03-11T08:54:56Zhttp://cogprints.org/id/eprint/2248This item is in the repository with the URL: http://cogprints.org/id/eprint/22482002-06-10ZAnticipatory Semantic ProcessesWhy anticipatory processes correspond to cognitive abilities of living systems? To be adapted to an environment, behaviors need at least i) internal representations of events occurring in the external environment; and ii) internal anticipations of possible events to occur in the external environment. Interactions of these two opposite but complementary cognitive properties lead to various patterns of experimental data on semantic processing.
How to investigate dynamic semantic processes? Experimental studies in cognitive psychology offer several interests such as: i) the control of the semantic environment such as words embedded in sentences; ii) the methodological tools allowing the observation of anticipations and adapted oculomotor behavior during reading; and iii) the analyze of different anticipatory processes within the theoretical framework of semantic processing.
What are the different types of semantic anticipations? Experimental data show that semantic anticipatory processes involve i) the coding in memory of sequences of words occurring in textual environments; ii) the anticipation of possible future words from currently perceived words; and iii) the selection of anticipated words as a function of the sequences of perceived words, achieved by anticipatory activations and inhibitory selection processes.
How to modelize anticipatory semantic processes? Localist or distributed neural networks models can account for some types of semantic processes, anticipatory or not. Attractor neural networks coding temporal sequences are presented as good candidate for modeling anticipatory semantic processes, according to specific properties of the human brain such as i) auto-associative memory; ii) learning and memorization of sequences of patterns; and iii) anticipation of memorized patterns from previously perceived patterns.
Frédéric LavignePascal Lavigne2000-07-04Z2011-03-11T08:53:42Zhttp://cogprints.org/id/eprint/150This item is in the repository with the URL: http://cogprints.org/id/eprint/1502000-07-04ZConstructional Tools as the Origin of Cognitive CapacitiesIt is argued that cognitive capacities can be understood as the outcome of the collective action of a set of agents created by tools that explore possible behaviours and train the agents to behave in such appropriate ways as may be discovered. The coherence of the whole system is assured by a combination of vetting the performance of new agents and dealing appropriately with any faults that the whole system may develop. This picture is shown to account for a range of cognitive capacities, including language.Brian D. Josephson2001-02-12Z2011-03-11T08:54:29Zhttp://cogprints.org/id/eprint/1296This item is in the repository with the URL: http://cogprints.org/id/eprint/12962001-02-12ZDuplication of modules facilitates the evolution of functional specializationThe evolution of simulated robots with three different architectures is studied. We compared a non-modular feed forward network, a hardwired modular and a duplication-based modular motor control network. We conclude that both modular architectures outperform the non-modular architecture, both in terms of rate of adaptation as well as the level of adaptation achieved. The main difference between the hardwired and duplication-based modular architectures is that in the latter the modules reached a much higher degree of functional specialization of their motor control units with regard to high level behavioral functions. The hardwired architectures reach the same level of performance, but have a more distributed assignment of functional tasks to the motor control units. We conclude that the mechanism through which functional specialization is achieved is similar to the mechanism proposed for the evolution of duplicated genes. It is found that the duplication of multifunctional modules first leads to a change in the regulation of the module, leading to a differentiation of the functional context in which the module is used. Then the module adapts to the new functional context. After this second step the system is locked into a functionally specialized state. We suggest that functional specialization may be an evolutionary absorption state.Raffaele Calabretta2001-02-12Z2011-03-11T08:54:32Zhttp://cogprints.org/id/eprint/1304This item is in the repository with the URL: http://cogprints.org/id/eprint/13042001-02-12ZDuplication of modules facilitates the evolution of functional specializationThe evolution of simulated robots with three different architectures is studied. We compared a non-modular feed forward network, a hardwired modular and a duplication-based modular motor control network. We conclude that both modular architectures outperform the non-modular architecture, both in terms of rate of adaptation as well as the level of adaptation achieved. The main difference between the hardwired and duplication-based modular architectures is that in the latter the modules reached a much higher degree of functional specialization of their motor control units with regard to high level behavioral functions. The hardwired architectures reach the same level of performance, but have a more distributed assignment of functional tasks to the motor control units. We conclude that the mechanism through which functional specialization is achieved is similar to the mechanism proposed for the evolution of duplicated genes. It is found that the duplication of multifunctional modules first leads to a change in the regulation of the module, leading to a differentiation of the functional context in which the module is used. Then the module adapts to the new functional context. After this second step the system is locked into a functionally specialized state. We suggest that functional specialization may be an evolutionary absorption state.Raffaele CalabrettaStefano NolfiDomenico ParisiGunter P. Wagner2002-01-11Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2023This item is in the repository with the URL: http://cogprints.org/id/eprint/20232002-01-11ZEvolution of Symbolisation in Chimpanzees and Neural Netsfrom Introduction: Animal communication systems and human languages can be characterised by the type of cognitive abilities that are required. If we consider the main semiotic distinction between communication using icons, signals, or symbols (Peirce, 1955; Harnad, 1990; Deacon, 1997) we can identify different cognitive loads for each type of reference. The use and understanding of icons require instinctive behaviour (e.g. emotions) or simple perceptual processes (e.g. visual similarities between an icon and its meaning). Communication systems that use signals are characterised by referential associations between objects and visual or auditory signals. They require the cognitive ability to learn stimulus associations, such as in conditional learning. Symbols have double associations. Initially, symbolic systems require the establishment of associations between signals and objects. Secondly, other types of relationships are learned between the signals themselves. The use of rule for the logical combination of symbols is an example of symbolic relationship. Symbolisation is the ability to acquire and handle symbols and symbolic relationships.
Angelo Cangelosi2001-06-26Z2011-03-11T08:54:43Zhttp://cogprints.org/id/eprint/1647This item is in the repository with the URL: http://cogprints.org/id/eprint/16472001-06-26ZFrom Robotic Toil to Symbolic Theft: Grounding Transfer from Entry-Level to Higher-Level CategoriesNeural network models of categorical perception (compression of within-category similarity
and dilation of between-category differences) are applied to the symbol-grounding problem
(of how to connect symbols with meanings) by connecting analog sensorimotor projections to
arbitrary symbolic representations via learned category-invariance detectors in a hybrid
symbolic/nonsymbolic system. Our nets are trained to categorize and name 50x50 pixel
images (e.g., circles, ellipses, squares and rectangles) projected onto the receptive field of a
7x7 retina. They first learn to do prototype matching and then entry-level naming for the four
kinds of stimuli, grounding their names directly in the input patterns via hidden-unit
representations ("sensorimotor toil"). We show that a higher-level categorization (e.g.,
"symmetric" vs. "asymmetric") can learned in two very different ways: either (1) directly
from the input, just as with the entry-level categories (i.e., by toil), or (2) indirectly, from
boolean combinations of the grounded category names in the form of propositions describing
the higher-order category ("symbolic theft"). We analyze the architectures and input
conditions that allow grounding (in the form of compression/separation in internal similarity
space) to be "transferred" in this second way from directly grounded entry-level category
names to higher-order category names. Such hybrid models have implications for the
evolution and learning of language.Angelo CangelosiAlberto GrecoStevan Harnad2001-11-18Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1892This item is in the repository with the URL: http://cogprints.org/id/eprint/18922001-11-18ZHebbian Learning and
Temporary Storage in the Convergence-Zone Model of Episodic MemoryThe Convergence-Zone model shows how sparse, random memory patterns can lead to one-shot storage and high capacity in the hippocampal component of the episodic memory system. This paper presents a biologically more realistic version of the model, with continuously-weighted connections and storage through Hebbian learning and normalization. In contrast to the gradual weight adaptation in many neural network models, episodic memory turns out to require high learning rates. Normalization allows earlier patterns to be overwritten,
introducing time-dependent forgetting similar to the hippocampus.Michael HoweRisto Miikkulainen2000-11-15Z2011-03-11T08:54:27Zhttp://cogprints.org/id/eprint/1109This item is in the repository with the URL: http://cogprints.org/id/eprint/11092000-11-15ZLearning synaptic clusters for non-linear dendritic processingNonlinear dendritic processing appears to be a feature of biological neurons and would also be of use in many applications of artificial neural networks. This paper presents a model of an initially standard linear unit which uses unsupervised learning to find clusters of inputs within which inactivity at one synapse can occlude the activity at the other synapses.Michael SpratlingGillian Hayes2001-06-19Z2011-03-11T08:54:42Zhttp://cogprints.org/id/eprint/1620This item is in the repository with the URL: http://cogprints.org/id/eprint/16202001-06-19ZNeural Network Models of Categorical PerceptionStudies of the categorical perception (CP) of sensory continua have a long and rich history in
psychophysics. In 1977, Macmillan et al. introduced the use of signal detection theory to CP
studies. Anderson et al. simultaneously proposed the first neural model for CP, yet this line
of research has been less well explored. In this paper, we assess the ability of neural-network
models of CP to predict the psychophysical performance of real observers with speech sounds
and artificial/novel stimuli. We show that a variety of neural mechanisms is capable of gen-erating
the characteristics of categorical perception. Hence, CP may not be a special mode of
perception but an emergent property of any sufficiently powerful general learning system.R.I. DamperS.R. Harnad2000-10-18Z2011-03-11T08:54:25Zhttp://cogprints.org/id/eprint/1038This item is in the repository with the URL: http://cogprints.org/id/eprint/10382000-10-18ZQueueing Network Modelling with Distributed Neural Networks for Service Quality Estimation in B-ISDN NetworksWe discuss an original scheme based on distributed feedforward neural networks (NN), aimed at modelling several queueing systems in cascade fed with bursty traffic. For each queueing system, a neural network is trained to anticipate the average number of waiting packets, the packet loss rate and the coefficient of variation of the packet inter-departure time, given the mean rate, the peak rate and the coefficient of variation of the packet inter-arrival time. The latter serves for the calculation of the coefficient of variation of the cell inter-arrival time of the aggregated traffic which is fed as input to the next NN along the path. The potential of this method was sucessfully illustrated on several single server FIFO queues in (Aussem99). We now apply this technique to model a small queueing network made up from a combination of queues in tandem and in parallel fed by a superimposition of OnOff sources. Our long-term goal is the design of preventive control strategy in a multiservice communication network.
Alex AussemAntoine MahulRaymond Marie2001-02-25Z2011-03-11T08:54:33Zhttp://cogprints.org/id/eprint/1314This item is in the repository with the URL: http://cogprints.org/id/eprint/13142001-02-25ZRatCog: A GUI maze simulation tool with plugin "rat brains."We have implemented RatCog, a Graphical User Interface (GUI) radial-maze simulation tool providing various computational models of rats. Rat models are loaded as runtime plugin files, and an Application Programming Interface (API) enables additional plugins to be created. One implemented plugin is a back-propagation trained connectionist model. GUI features include maze graphics and performance statistics. The GUI makes it easier to use these computional models, while the plugins make the models widely available.C. G. PrinceJ. TaltonI. S. N. BerkeleyC. Gunay2001-11-18Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1899This item is in the repository with the URL: http://cogprints.org/id/eprint/18992001-11-18ZSelf-Organization of Innate Face Preferences: Could Genetics Be Expressed Through Learning?Self-organizing models develop realistic cortical structures when given approximations of the visual environment as input, and are an effective way to model the development of face recognition abilities. However, environment-driven self-organization alone cannot account for the fact that newborn human infants will preferentially attend to face-like stimuli even immediately after birth. Recently it has been proposed that internally generated input patterns, such as those found in the developing retina and in PGO waves during REM sleep, may have the same effect on self-organization as does the external environment. Internal pattern generators constitute an efficient way to specify, develop, and maintain functionally appropriate perceptual organization. They may help express complex structures from minimal genetic information, and retain this genetic structure within a highly plastic system. Simulations with the RF-LISSOM model show that such preorganization can account for newborn face preferences, providing a computational framework for examining how genetic influences interact with experience to construct a complex system.
James A. BednarRisto Miikkulainen2000-10-18Z2011-03-11T08:54:25Zhttp://cogprints.org/id/eprint/1039This item is in the repository with the URL: http://cogprints.org/id/eprint/10392000-10-18ZSufficient Conditions for Error Back Flow Convergence in Dynamical Recurrent Neural NetworksThis paper extends previous analysis of the gradient decay to a class of discrete-time fully recurrent networks, called Dynamical Recurrent Neural Networks (DRNN), obtained by modelling synapses as Finite Impulse Response (FIR) filters instead of multiplicative scalars. Using elementary matrix manipulations, we provide an upper bound on the norm of the weight matrix ensuring that the gradient vector, when propagated in a reverse manner in time through the error-propagation network, decays exponentially to zero. This bounds apply to all FIR architecture proposals as well as fixed point recurrent networks, regardless of delay and connectivity. In addition, we show that the computational overhead of the learning algorithm can be reduced drastically by taking advantage of the exponential decay of the gradient.Alex Aussem2000-07-18Z2011-03-11T08:54:21Zhttp://cogprints.org/id/eprint/877This item is in the repository with the URL: http://cogprints.org/id/eprint/8772000-07-18ZThe Geometry of Stimulus ControlMany studies, both in ethology and comparative psychology, have shown that animals react to modifications of familiar stimuli. This phenomenon is often referred to as generalisation. Most modifications lead to a decrease in responding, but to certain new stimuli an increase in responding is observed. This holds for both innate and learned behaviour. Here we propose a heuristic approach to stimulus control, or stimulus selection, with the aim of explaining these phenomena. The model has two key elements. First, we choose the receptor level as the fundamental stimulus space. Each stimulus is represented as the pattern of activation it induces in sense organs. Second, in this space we introduce a simple measure of `similarity' between stimuli by calculating how activation patterns overlap. The main advantage we recognise in this approach is that the generalisation of acquired responses emerges from a few simple principles which are grounded in the recognition of how animals actually perceive stimuli. Many traditional problems that face theories of stimulus control (e.g. the Spence-Hull theory of gradient interaction or ethological theories of stimulus summation) do not arise in the present framework. These problems include the amount of generalisation along different dimensions, peak-shift phenomena (with respect to both positive and negative shifts), intensity generalisation, and generalisation after conditioning on two positive stimuli.Stefano GhirlandaMagnus Enquist1999-10-11Z2011-03-11T08:53:43Zhttp://cogprints.org/id/eprint/184This item is in the repository with the URL: http://cogprints.org/id/eprint/1841999-10-11ZThe Geometry of Stimulus ControlMany studies, both in ethology and comparative psychology, have shown that animals react to modifications of familiar stimuli. This phenomenon is often referred to as generalisation. Most modifications lead to a decrease in responding, but to certain new stimuli an increase in responding is observed. This holds for both innate and learned behaviour. Here we propose a heuristic approach to stimulus control, or stimulus selection, with the aim of explaining these phenomena. The model has two key elements. First, we choose the receptor level as the fundamental stimulus space. Each stimulus is represented as the pattern of activation it induces in sense organs. Second, in this space we introduce a simple measure of `similarity' between stimuli by calculating how activation patterns overlap. The main advantage we recognise in this approach is that the generalisation of acquired responses emerges from a few simple principles which are grounded in the recognition of how animals actually perceive stimuli. Many traditional problems that face theories of stimulus control (e.g. the Spence-Hull theory of gradient interaction or ethological theories of stimulus summation) do not arise in the present framework. These problems include the amount of generalisation along different dimensions, peak-shift phenomena (with respect to both positive and negative shifts), intensity generalisation, and generalisation after conditioning on two positive stimuliStefano GhirlandaMagnus Enquist2000-08-07Z2011-03-11T08:54:22Zhttp://cogprints.org/id/eprint/921This item is in the repository with the URL: http://cogprints.org/id/eprint/9212000-08-07ZMemory Evolutive SystemsNatural autonomous systems, such as biological, neural, social or cultural systems, are open, self-organized systems with a more or less large hierarchy of interacting complexity levels; they are able to memorize their experiences and to adapt to various conditions through a change of behavior. These last fifteen years, the Authors have developed a mathematical model for these systems, based on Category Theory. The aim of the paper is to give an overview of this model, called Memory Evolutive Systems.Andree EhresmannJean-Paul Vanbremeersch2001-05-09Z2011-03-11T08:54:38Zhttp://cogprints.org/id/eprint/1488This item is in the repository with the URL: http://cogprints.org/id/eprint/14882001-05-09ZThe What and Why of Binding: The Modeler's PerspectiveIn attempts to formulate a computational understanding of brain function,
one of the fundamental concerns is the data structure by which the brain
represents information. For many decades, a conceptual framework has
dominated the thinking of both brain modelers and neurobiologists. That
framework is referred to here as "classical neural networks." It is well
supported by experimental data, although it may be incomplete. A
characterization of this framework will be offered in the next section.
Difficulties in modeling important functional aspects of the brain on the
basis of classical neural networks alone have led to the recognition that
another, general mechanism must be invoked to explain brain function. That
mechanism I call "binding." Binding by neural signal synchrony had been
mentioned several times in the liter ature (Lege´ndy, 1970; Milner, 1974)
before it was fully formulated as a general phenomenon (von der Malsburg,
1981). Although experimental evidence for neural syn chrony was soon found,
the idea was largely ignored for many years. Only recently has it become a
topic of animated discussion. In what follows, I will summarize the nature
and the roots of the idea of binding, especially of temporal binding, and
will discuss some of the objec tions raised against it.
Christoph von der Malsburg1999-07-01Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/546This item is in the repository with the URL: http://cogprints.org/id/eprint/5461999-07-01ZRepresentation and processing of structures with binary sparse distributed codesThe schemes for compositional distributed representations include those allowing on-the-fly construction of fixed dimensionality codevectors to encode structures of various complexity. Similarity of such codevectors takes into account both structural and semantic similarity of represented structures. In this paper we provide a comparative description of sparse binary distributed representation developed in the frames of the Associative-Projective Neural Network architecture and more well-known Holographic Reduced Representations of Plate and Binary Spatter Codes of Kanerva. The key procedure in Associative-Projective Neural Networks is Context-Dependent Thinning which binds codevectors and maintains their sparseness. The codevectors are stored in structured memory array which can be realized as distributed auto-associative memory. Examples of distributed representation of structured data are given. Fast estimation of similarity of analogical episodes by the overlap of their codevectors is used in modeling of analogical reasoning for retrieval of analogs from memory and for analogical mapping.Dmitri A. Rachkovskij1999-05-30Z2011-03-11T08:53:52Zhttp://cogprints.org/id/eprint/382This item is in the repository with the URL: http://cogprints.org/id/eprint/3821999-05-30ZTip-of-the-Tongue Phenomena: An Introductory Phenomenological AnalysisThe issue of meaningful yet unexpressed background - to language, to our experiences of the body - is one whose exploration is still in its infancy. There are various aspects of "invisible," implicit, or background experiences which have been investigated from the viewpoints of phenomenology, cognitive psychology, and linguistics. I will claim that James, as explicated by Gurwitsch and others, has analyzed the phenomenon of fringes in such a way as to provide a structural framework from which to investigate and better understand those ideas or concepts that are unexpressed, particularly those experienced in the sense of being sought-after. I will consider Johnsons conception of the image-schematic gestalt (ISG) as a way of bridging the descriptive gap between phenomenology and cognitive psychology. Starting from an analysis of the fringes, I will turn to a consideration of the of tip-of-tongue (TOT) state, as a kind of feeling-of-knowing (FOK) state, from a variety of approaches, focusing mainly on cognitive psychology and phenomenology. I will then integrate a phenomenological analysis of these experiences, from the James/Gurwitsch structural viewpoint, with a cognitive/phenomenological analysis in terms of ISGs; and further integrate that with a cognitive/functional analysis of consciousness. I will employ this synthesis of three viewpoints to explore the thesis that the TOT state and similar experiences may relate to the gestalt nature of schemas as well as to particular cues, and may thus be experienced as an aspect of the continuum to the general background to all our conscious experiences.Steven R. Brown1999-04-28Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/536This item is in the repository with the URL: http://cogprints.org/id/eprint/5361999-04-28ZField Computation in Natural and Artificial IntelligenceWe review the concepts of field computation, a model of computation that processes information represented as spatially continuous arrangements of continuous data. We show that many processes in the brain are described usefully as field computation. Throughout we stress the connections between field computation and quantum mechanics, especially including the important role of information fields, which represent by virtue of their form rather than their magnitude. We also show that field computation permits simultaneous nonlinear computation in linear superposition.Bruce J. MacLennan2001-09-05Z2011-03-11T08:54:47Zhttp://cogprints.org/id/eprint/1790This item is in the repository with the URL: http://cogprints.org/id/eprint/17902001-09-05ZHow to model consciousness in Memory Evolutive Systems?Memory Evolutive Systems (MES) represent a mathematical model, based on Category Theory, to study natural open autonomous systems such as biological, neural or social systems. It has been progressively developed by the authors in a series of papers since 1986. In this model the dynamics is modulated by the competitive interactions between a net of internal more or less complex organs of regulation, called CoRegulators (CR), with a differential access to a central hierarchical Memory. This article attempts to model the notions of Semantics and Consciousness in such a MES
A Semantics will emerge through the detection of specific invariances by the CRs that leads to classify objects according to their main attributes, and record the invariance classes. The model explains how it relies on a hierarchical 2 steps process: first a pragmatically 'acted' classification at the level of specific attributes (such as colors), then this classification is 'reflected' and analyzed at a higher level, and a new formal unit, called a 'concept', is formed to represent the invariance class (e.g., the color 'blue').
The introduction of more and more abstract concepts gives more flexibility to the comportment. It is essential for the development of some kind of 'consciousness'. A 'conscious' CR is characterized by the capacity to respond to a new event or to a fracture by an increase in awareness, which permits: (i) to extend its actual 'landscape' (formed by the information it can gather) retrospectively to past lower levels; (ii) to operate an abduction process in this extended landscape to find possible causes of the fracture; (iii) and finally to planify a strategy for several steps ahead, through the formation of internal 'virtual' landscapes in which strategies can be tried without energy costs. Thus consciousness would amount to an internalization of Semantics and Time, giving a selective advantage.
In the second Part of the paper, a MES modeling a neural system is explicitly described and it is shown how the various processes described above are in agreement with present neurophysiological knowledge.
Finally the general ideas are illustrated on a concrete example.
Andrée C. EhresmannJean-Paul Vanbremeersch1999-04-30Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/537This item is in the repository with the URL: http://cogprints.org/id/eprint/5371999-04-30ZBinding and Normalization of Binary Sparse Distributed Representations by Context-Dependent ThinningDistributed representations were often criticized as inappropriate for encoding of data with a complex structure. However Plate's Holographic Reduced Representations and Kanerva's Binary Spatter Codes are recent schemes that allow on-the-fly encoding of nested compositional structures by real-valued or dense binary vectors of fixed dimensionality. In this paper we consider procedures of the Context-Dependent Thinning which were developed for representation of complex hierarchical items in the architecture of Associative-Projective Neural Networks. These procedures provide binding of items represented by sparse binary codevectors (with low probability of 1s). Such an encoding is biologically plausible and allows to reach high information capacity of distributed associative memory where the codevectors may be stored. In distinction to known binding procedures, Context-Dependent Thinning allows to support the same low density (or sparseness) of the bound codevector for varied number of constituent codevectors. Besides, a bound codevector is not only similar to another one with similar constituent codevectors (as in other schemes), but it is also similar to the constituent codevectors themselves. This allows to estimate a structure similarity just by the overlap of codevectors, without the retrieval of the constituent codevectors. This also allows an easy retrieval of the constituent codevectors. Examples of algorithmic and neural network implementations of the thinning procedures are considered. We also present representation examples of various types of nested structured data (propositions using role-filler and predicate-arguments representation, trees, directed acyclic graphs) using sparse codevectors of fixed dimension. Such representations may provide a fruitful alternative to the symbolic representations of traditional AI, as well as to the localist and microfeature-based connectionist representations.Dmitri A. RachkovskijErnst M. Kussul1999-04-09Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/535This item is in the repository with the URL: http://cogprints.org/id/eprint/5351999-04-09ZDRAMA, a connectionist architecture for control and learning in autonomous robotsThis work proposes a connectionist architecture, DRAMA, for dynamic control and learning of autonomous robots. DRAMA stands for dynamical recurrent associative memory architecture. It is a time-delay recurrent neural network, using Hebbian update rules. It allows learning of spatio-temporal regularities and time series in discrete sequences of inputs, in the face of an important amount of noise. The first part of this paper gives the mathematical description of the architecture and analyses theoretically and through numerical simulations its performance. The second part of this paper reports on the implementation of DRAMA in simulated and physical robotic experiments. Training and rehearsal of the DRAMA architecture is computationally fast and inexpensive, which makes the model particularly suitable for controlling `computationally-challenged' robots. In the experiments, we use a basic hardware system with very limited computational capability and show that our robot can carry out real time computation and on-line learning of relatively complex cognitive tasks. In these experiments, two autonomous robots wander randomly in a fixed environment, collecting information about its elements. By mutually associating information of their sensors and actuators, they learn about physical regularities underlying their experience of varying stimuli. The agents learn also from their mutual interactions. We use a teacher-learner scenario, based on mutual following of the two agents, to enable transmission of a vocabulary from one robot to the other.Aude BillardGillian Hayes1999-10-06Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/547This item is in the repository with the URL: http://cogprints.org/id/eprint/5471999-10-06ZDynamical recurrent neural networks towards prediction and modeling of dynamical systemsThis paper addresses the use of Dynamical Recurrent Neural Networks (DRNN) for time series prediction and modeling of small dynamical systems. Since the recurrent synapses are represented by Finite Impulse Response (FIR) filters, DRNN are state-based connectionist models in which all hidden units act as state variables of a dynamical system. The model is trained with Temporal Recurrent Backprop (TRBP), an efficient backward recurrent training procedure with minimal computational burden which benefits from the exponential decay of gradient reversely in time. The gradient decay is first illustrated on intensive experiments on the standard sunspot series. The ability of the model to internally encode useful information on the underlying process is then illustrated by several experiments on well known chaotic processes. Parsimonious DRNN models are able to find an appropriate internal representation of various chaotic processes from the observation of a subset of the state variables.A. Aussem2002-01-16Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2021This item is in the repository with the URL: http://cogprints.org/id/eprint/20212002-01-16ZHeterochrony and adaptation in developing neural networksThis paper discusses the simulation results of a model of biological development for neural networks based on a regulatory genome. The models results are analyzed using the framework of Heterochrony theory (McKinney and McNamara, 1991). The network development is controlled by genes that produce elements regulating the activation, inhibition, and delay of neurogenetic events. The genome can also regulate the gene expression mechanisms. An ecological task of foraging behavior is used to test the model with an evolving population of artificial organisms. Organisms evolve an optimal foraging behavior and the ability to adapt to changing environments. The adaptive strategy consists in changes of network architecture that are determined by the regulatory rearrangment of neurogenetic events. Results show how heterochronic changes play an adaptive role in the evolution of neural networks.Angelo Cangelosi2002-01-11Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2022This item is in the repository with the URL: http://cogprints.org/id/eprint/20222002-01-11ZModeling the evolution of communication: From stimulus associations to grounded symbolic associationsThis paper describes a model for the evolution of communication systems using simple syntactic rules, such as word combinations. It also focuses on the distinction between simple word-object associations and symbolic relationships. The simulation method combines the use of neural networks and genetic algorithms. The behavioral task is influenced by Savage-Rumbaugh & Rumbaughs (1978) ape language experiments. The results show that languages that use combination of words (e.g. verb-object rule) can emerge by auto-organization and cultural transmission. Neural networks are tested to see if evolved languages are based on symbol acquisition. The implications of this model for Deacons (1997) hypothesis on the role of symbolic acquisition for the origin of language are discussed.Angelo Cangelosi2001-11-23Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1913This item is in the repository with the URL: http://cogprints.org/id/eprint/19132001-11-23ZModeling the self-organization of directional selectivity in the primary visual cortexA model is proposed to demonstrate how neurons in the primary visual cortex could self-organize to represent the direction of motion. The model is based on a temporal extension of the Self-Organizing Map where neurons act as leaky integrators. The map is trained with moving Gaussian inputs, and it develops a retinotopic map with orientation columns that divide into areas of opposite direction selectivity, as found in the visual cortex.
Igor FarkasRisto Miikkulainen1999-10-08Z2011-03-11T08:54:03Zhttp://cogprints.org/id/eprint/550This item is in the repository with the URL: http://cogprints.org/id/eprint/5501999-10-08ZNeural-based Queueing System Modelling for Service Quality Estimation in B-ISDN NetworksThis paper addresses an original scheme based on feedforward neural networks, aimed at modelling queueing systems fed with bursty traffic. A neural network is trained to anticipate the average number of waiting cells, the cell loss rate and the coefficient of variation of the cell inter-departure time, given the mean rate, the peak rate and the coefficient of variation of the cell inter-arrival time. Our long-term goal is the design of a preventive control strategy in B-ISDN networks based on distributed neural networks modelling each queueing system located at the input and output ports of the switching facilities. To illustrate the potential of neural networks for modelling queueing systems, a neural network is successfully trained to model OnOff/D/1/c, OnOff/OnOff/1/c and multi-OnOff/D/1/c queueing systems.Alex AussemSebastien RouxelRaymond Marie2000-11-15Z2011-03-11T08:54:27Zhttp://cogprints.org/id/eprint/1108This item is in the repository with the URL: http://cogprints.org/id/eprint/11082000-11-15ZPresynaptic lateral inhibition provides a better architecture for self-organising neural networksUnsupervised learning is an important property of the brain and of many artificial neural networks. A large variety of unsupervised learning algorithms have been proposed. This paper takes a different approach in considering the architecture of the neural network rather than the learning algorithm. It is shown that a self-organising neural network architecture using pre-synaptic lateral inhibition enables a single learning algorithm to find distributed, local, and topological representations as appropriate to the structure of the input data received. It is argued that such an architecture not only has computational advantages but is a better model of cortical self-organisation.Michael Spratling2002-12-11Z2011-03-11T08:55:07Zhttp://cogprints.org/id/eprint/2646This item is in the repository with the URL: http://cogprints.org/id/eprint/26462002-12-11ZSARDSRN: A NEURAL NETWORK SHIFT-REDUCE PARSERSimple Recurrent Networks (SRNs) have been widely used in natural language tasks. SARDSRN extends the SRN by
explicitly representing the input sequence in a SARDNET self-organizing map. The distributed SRN component leads to good generalization and robust cognitive properties, whereas the SARDNET map provides exact representations of the sentence constituents. This combination allows SARDSRN to learn to parse sentences with more complicated structure than can the SRN alone, and suggests that the approach could scale up to realistic natural language. Marshall R. MayberryRisto Miikkulainen2000-02-09Z2011-03-11T08:53:41Zhttp://cogprints.org/id/eprint/139This item is in the repository with the URL: http://cogprints.org/id/eprint/1392000-02-09ZThe theory of the organism-environment system: III. Role of efferent influences on receptors in the formation of knowledge.The present article is an attempt to give - in the frame of the theory of the organism-environment system (Jarvilehto 1998a) - a new interpretation to the role of efferent influences on receptor activity and to the functions of senses in the formation of knowledge. It is argued, on the basis of experimental evidence and theoretical considerations, that the senses are not transmitters of environmental information, but they create a direct connection between the organism and the environment, which makes the development of a dynamic living system, the organism-environment system, possible. In this connection process the efferent influences on receptor activity are of particular significance, because with their help the receptors may be adjusted in relation to the parts of the environment which are most important in the achievement of behavioral results. Perception is the process of joining of new parts of the environment to the organism-environment system; thus, the formation of knowledge by perception is based on reorganization (widening and differentiation) of the organism-environment system, and not on transmission of information from the environment. With the help of the efferent influences on receptors each organism creates its own peculiar world which is simultaneously subjective and objective. The present considerations have far reaching influences as well on experimental work in neurophysiology and psychology of perception as on philosophical considerations of knowledge formation.Timo Jarvilehto1999-10-08Z2011-03-11T08:54:03Zhttp://cogprints.org/id/eprint/549This item is in the repository with the URL: http://cogprints.org/id/eprint/5491999-10-08ZWedding connectionist and algorithmic modelling towards forecasting Caulerpa taxifolia development in the north-western Mediterranean seaWe discuss the use of supervised neural networks as a metamodelling technique for discrete event stochastic simulation in order to reduce significantly the computational burden involved by discrete simulations. A sophisticated computer model, coupling a Geographical Information System with a stochastic discrete event simulator, has been developed to anticipate the propagation of the green alga {\em Caulerpa taxifolia} in the North-Western Mediterranean sea. The simulation model provides reliable predictions, a couple of years in advance, of: i) the local expansion patterns of the alga, ii) the increase of {\em C. taxifolia} biomass and iii), the covered surfaces. However because the algorithmic model accounts for spatial interactions and anthropic dispersion/activities such as eradication, introduction of specific predators etc., simulations are extremely time and memory consuming. Therefore, to reduce the computational burden, a neural network was successfully trained on artificially generated data provided by the simulation runs to provide accurate forecasts 12 years in advance along with associated confidence intervals. The ability of the neural networks to capture the underlying physics of the phenomena is clearly illustrated by several preliminary experiments on a large coastal area. The neural network is able to construct, on this site, estimates of the {\em Caulerpa taxifolia} expansion 12 years in advance in good agreement with the simulation trajectories.Alex AussemDavid. Hill1998-10-22Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/518This item is in the repository with the URL: http://cogprints.org/id/eprint/5181998-10-22ZCerebellar Control of Robot ArmsDecades of research into the structure and function of the cerebellum have led to a clear understanding of many of its cells, as well as how learning takes place. Furthermore, there are many theories on what signals the cerebellum operates on, and how it works in concert with other parts of the nervous system. Nevertheless, the application of computational cerebellar models to the control of robot dynamics remains in its infant state. To date, a few applications have been realized, yet limited to the control of traditional robot structures which, strictly speaking, do not require adaptive control for the tasks that are performed since their dynamic structures are relatively simple. The currently emerging family of light-weight robots poses a new challenge to robot control: due to their complex dynamics traditional methods, depending on a full analysis of the dynamics of the system, are no longer applicable since the joints influence each other dynamics during movement. Can artificial cerebellar models compete here? In this overview paper we present a succinct introduction of the cerebellum, and discuss where it could be applied to tackle problems in robotics. Without conclusively answering the above question, an overview of several applications of cerebellar models to robot control is given.P. van der Smagt2001-05-10Z2011-03-11T08:54:38Zhttp://cogprints.org/id/eprint/1495This item is in the repository with the URL: http://cogprints.org/id/eprint/14952001-05-10ZNeural Learning of Embodied Interaction DynamicsThis paper presents our approach towards realizing a robot which can bootstrap itself towards higher complexity through embodied interaction dynamics with the environment including other agents. First, the elements of interaction dynamics are extracted from conceptual analysis of embodied interaction and its emergence, especially of behavioral imitation. Then three case studies are made, presenting our neural architecture and the robotic experiments on some of the important elements discussed above: self exploration and entrainment, emergent coordination, and categorizing self behavior. Finally we propose that integrating all these elements will be the important step towards realizing the bootstrapping agent envisaged above.
Yasuo KuniyoshiLuc Berthouze1998-06-18Z2011-03-11T08:54:12Zhttp://cogprints.org/id/eprint/690This item is in the repository with the URL: http://cogprints.org/id/eprint/6901998-06-18ZFacial beauty and fractal geometryWhat is it that makes a face beautiful? Average faces obtained by photographic (Galton 1878) or digital (Langlois & Roggman 1990) blending are judged attractive but not optimally attractive (Alley & Cunningham 1991) --- digital exaggerations of deviations from average face blends can lead to higher attractiveness ratings (Perrett, May, & Yoshikawa 1994). My novel approach to face design does not involve blending at all. Instead, the image of a female face with high ratings is composed from a fractal geometry based on rotated squares and powers of two. The corresponding geometric rules are more specific than those previously used by artists such as Leonardo and Duerer. They yield a short algorithmic description of all facial characteristics, many of which are compactly encodable with the help of simple feature detectors similar to those found in mammalian brains. This suggests that a face's beauty correlates with simplicity relative to the subjective observer's way of encoding it.Juergen Schmidhuber1998-06-22Z2011-03-11T08:54:12Zhttp://cogprints.org/id/eprint/700This item is in the repository with the URL: http://cogprints.org/id/eprint/7001998-06-22ZWhat sort of architecture is required for a human-like agent?This paper is about how to give human-like powers to complete agents. For this the most important design choice concerns the overall architecture. Questions regarding detailed mechanisms, forms of representations, inference capabilities, knowledge etc. are best addressed in the context of a global architecture in which different design decisions need to be linked. Such a design would assemble various kinds of functionality into a complete coherent working system, in which there are many concurrent, partly independent, partly mutually supportive, partly potentially incompatible processes, addressing a multitude of issues on different time scales, including asynchronous, concurrent, motive generators. Designing human like agents is part of the more general problem of understanding design space, niche space and their interrelations, for, in the abstract, there is no one optimal design, as biological diversity on earth shows.Aaron Sloman1998-06-22Z2011-03-11T08:54:12Zhttp://cogprints.org/id/eprint/695This item is in the repository with the URL: http://cogprints.org/id/eprint/6951998-06-22ZThe evolution of what?There is now a huge amount of interest in consciousness among scientists as well as philosophers, yet there is so much confusion and ambiguity in the claims and counter-claims that it is hard to tell whether any progress is being made. This ``position paper'' suggests that we can make progress by temporarily putting to one side questions about what consciousness is or which animals or machines have it or how it evolved. Instead we should focus on questions about the sorts of architectures that are possible for behaving systems and ask what sorts of capabilities, states and processes, might be supported by different sorts of architectures. We can then ask which organisms and machines have which sorts of architectures. This combines the standpoint of philosopher, biologist and engineer. If we can find a general theory of the variety of possible architectures (a characterisation of ``design space'') and the variety of environments, tasks and roles to which such architectures are well suited (a characterisation of ``niche space'') we may be able to use such a theory as a basis for formulating new more precisely defined concepts with which to articulate less ambiguous questions about the space of possible minds. For instance our initially ill-defined concept (``consciousness'') might split into a collection of more precisely defined concepts which can be used to ask unambiguous questions with definite answers. As a first step this paper explores a collection of conjectures regarding architectures and their evolution. In particular we explore architectures involving a combination of coexisting architectural levels including: (a) reactive mechanisms which evolved very early, (b) deliberative mechanisms which evolved later in response to pressures on information processing resources and (c) meta-management mechanisms that can explicitly inspect evaluate and modify some of the contents of various internal information structures. It is conjectured that in response to the needs of these layers, perceptual and action subsystems also developed layers, and also that an ``alarm'' system which initially existed only within the reactive layer may have become increasingly sophisticated and extensive as its inputs and outputs were linked to the newer layers. Processes involving the meta-management layer in the architecture could explain the origin of the notion of ``qualia''. Processes involving the ``alarm'' mechanism and mechanisms concerned with resource limits in the second and third layers gives us an explanation of three main forms of emotion, helping to account for some of the ambiguities which have bedevilled the study of emotion. Further theoretical and practical benefits may come from further work based on this design-based approach to consciousness. A deeper longer term implication is the possibility of a new science investigating laws governing possible trajectories in design space and niche space, as these form parts of high order feedback loops in the biosphere.Aaron Sloman1998-10-20Z2011-03-11T08:53:51Zhttp://cogprints.org/id/eprint/365This item is in the repository with the URL: http://cogprints.org/id/eprint/3651998-10-20ZTHE THEORY OF THE ORGANISM-ENVIRONMENT SYSTEM: II. SIGNIFICANCE OF NERVOUS ACTIVITY IN THE ORGANISM-ENVIRONMENT SYSTEMThe relation between mental processes and brain activity is studied from the point of view of the theory of the organism-environment system. It is argued that the systemic point of view leads to a new kind of definition of the primary tasks of neurophysiology and to a new understanding of the traditional neurophysiological concepts. Neurophysiology is restored to its place as a part of biology: its task is the study of neurons as living units, not as computer chips. Neurons are living units which are organised as metabolic systems in connection with other neurons; they are not units which would carry out some psychological functions or maintain states which are typical only of the whole organism-environment system. Psychological processes, on the other hand, are processes always comprising the whole organism-environment system.Timo Jarvilehto1998-04-28Z2011-03-11T08:53:55Zhttp://cogprints.org/id/eprint/436This item is in the repository with the URL: http://cogprints.org/id/eprint/4361998-04-28ZNeural model of transfer-of-binding in visual relative motion perception.A new way of measuring generalization in unsupervised learning is presented. The measure is based on an exclusive allocation, or credit assignment, criterion. In a classifier that satisfies the criterion, input patterns are parsed so that the credit for each input feature is assigned exclusively to one of multiple, possibly overlapping, output categories. Such a classifier achieves context-sensitive, global representations of pattern data. Two additional constraints, sequence masking and uncertainty multiplexing, are described; these can be used to refine the measure of generalization. The generalization performance of EXIN networks, winner-take-all competitive learning networks, linear decorrelator networks, and Nigrin's SONNET-2 network is compared.J.A. MarshallC.P. SchmittG.J. KalarickalR.K. Alley1998-12-10Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/520This item is in the repository with the URL: http://cogprints.org/id/eprint/5201998-12-10ZAre feedforward and recurrent networks systematic? Analysis and implications for a connectionist cognitive architectureHuman cognition is said to be systematic: cognitive ability generalizes to structurally related behaviours. The connectionist approach to cognitive theorizing has been strongly criticized for its failure to explain systematicity. Demonstrations of generalization notwithstanding, I show that two widely used networks (feedforward and recurrent) do not support systematicity under the condition of local input/output representations. For a connectionist explanation of systematicity, these results leave two choices, either: (1) develop models capable of systematicity under local input/output representations; or (2) justify the choice of similarity-based (nonlocal) component representations sufficient for systematicity.S. Phillips2006-12-03Z2011-03-11T08:56:43Zhttp://cogprints.org/id/eprint/5271This item is in the repository with the URL: http://cogprints.org/id/eprint/52712006-12-03ZArtificial neural networks as models of stimulus controlWe evaluate the ability of artificial neural network models (multi-layer perceptrons) to predict stimulus-response relationships. A variety of empirical results are considered, such as generalization, peak-shift (supernormality) and stimulus intensity effects. The networks were trained on the same tasks as the animals in the considered experiments. The subsequent generalization tests on the networks showed that the model replicates correctly the empirical results. It is concluded that these models are valuable tools in the study of animal behaviour.
Stefano GhirlandaMagnus Enquist1998-03-26Z2011-03-11T08:54:07Zhttp://cogprints.org/id/eprint/618This item is in the repository with the URL: http://cogprints.org/id/eprint/6181998-03-26ZCategories and Word Meanings in Adaptive OrganismsCategories and word meanings tend to be interpreted as single entities that are activated in the mind when an individual categorizes the world or uses language. We argue that this view is mistaken by using models based on neural networks and adaptive behavior simulations. Connectionist simulations of populations of organisms that live and reproduce in an environment show that categories and word meanings are open and changing collections of activation patterns in the network's internal units rather than single activation patterns.D ParisiD DenaroA Cangelosi2003-06-06Z2011-03-11T08:54:51Zhttp://cogprints.org/id/eprint/1975This item is in the repository with the URL: http://cogprints.org/id/eprint/19752003-06-06ZConnectionism Reconsidered: Minds, Machines and ModelsIn this paper the issue of drawing inferences about biological cognitive systems on the basis of connectionist simulations is addressed. In particular, the justification of inferences based on connectionist models trained using the backpropagation learning algorithm is examined. First it is noted that a justification commonly found in the philosophical literature is inapplicable. Then some general issues are raised about the relationships between models and biological systems. A way of conceiving the role of hidden units in connectionist networks is then introduced. This, in combination with an assumption about the way evolution goes about solving problems, is then used to suggest a means of justifying inferences about biological systems based on connectionist research.Istvan Berkeley1998-03-31Z2011-03-11T08:54:07Zhttp://cogprints.org/id/eprint/622This item is in the repository with the URL: http://cogprints.org/id/eprint/6221998-03-31ZA connectionist model for categorical perception and symbol groundingNeural network models of categorical perception can help solve the symbol-grounding problem [Harnad, 1990; 1993] by connecting analog sensory projections to symbolic representations through learned category-invariance detectors in a hybrid symbolic/nonsymbolic system. Our nets learn to categorize and name 50x50 pixel images of circles, ellipses, squares and rectangles projected onto the receptive field of a 7x7 retina. The nets first learn to do prototype matching and then entry-level naming for the four kinds of stimuli, grounding their names directly in the input patterns via hidden-unit representations. Next, a higher-order categorization (symmetric vs. asymmetric) is learned, either directly from the input, as with the entry- level categories, or from combinations of the grounded category names (symbols). We analyze the architectures and input conditions that allow grounding to be "transferred" from directly grounded entry-level category names to higher- order category names. Implications of such hybrid models for the evolution and learning of language are discussed.Alberto GrecoAngelo CangelosiStevan Harnad2001-11-07Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/501This item is in the repository with the URL: http://cogprints.org/id/eprint/5012001-11-07ZConnections, Binding, Unification & Analogical PromiscuityThis paper claims that higher cognition implemented by a connectionist system will be essentially analogical, with analogical mapping by continuous systematic substitution as the core cognitive process. The centrality of analogy is argued to be necessary in order for a connectionist system to use representations that are effectively symbolic. In turn, these representations are argued to be a necessary consequence of a sequence of broad design decisions needed to address technical problems in adapting a connectionist system for higher cognition. The design decisions are driven by the demands of a paradigmatic cognitive task and the desire to remain faithful to the constraints of connectionist components. Thus, the argument explains the origin of symbolic representations and analogy as necessary consequences of task demands and connectionist processing capabilities.Ross W. GaylerRoger Wales1998-07-29Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/500This item is in the repository with the URL: http://cogprints.org/id/eprint/5001998-07-29ZConnections, Binding, Unification and Analogical PromiscuityThis paper claims that higher cognition implemented by a connectionist system will be essentially analogical, with analogical mapping by continuous systematic substitution as the core cognitive process. The centrality of analogy is argued to be necessary in order for a connectionist system to use representations that are effectively symbolic. In turn, these representations are argued to be a necessary consequence of a sequence of broad design decisions needed to address technical problems in adapting a connectionist system for higher cognition. The design decisions are driven by the demands of a paradigmatic cognitive task and the desire to remain faithful to the constraints of connectionist components. Thus, the argument explains the origin of symbolic representations and analogy as necessary consequences of task demands and connectionist processing capabilities.Ross W. GaylerRoger Wales1999-01-03Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/522This item is in the repository with the URL: http://cogprints.org/id/eprint/5221999-01-03ZDisambiguation and Grammar as Emergent Soft ConstraintsWhen reading a sentence such as "The diplomat threw the ball in the ballpark for the princess" our interpretation changes from a dance event to baseball and back to dance. Such on-line disambiguation happens automatically and appears to be based on dynamically combining the strengths of association between the keywords and the two senses. Subsymbolic neural networks are very good at modeling such behavior. They learn word meanings as soft constraints on interpretation, and dynamically combine these constraints to form the most likely interpretation. On the other hand, it is very difficult to show how systematic language structures such as relative clauses could be processed in such a system. The network would only learn to associate them to specific contexts and would not be able to process new combinations of them. A closer look at understanding embedded clauses shows that humans are not very systematic in processing grammatical structures either. For example, "The girl who the boy who the girl who lived next door blamed hit cried" is very difficult to understand, whereas "The car that the man who the dog that had rabies bit drives is in the garage" is not. This difference emerges from the same semantic constraints that are at work in the disambiguation task. In this chapter we will show how the subsymbolic parser can be combined with high-level control that allows the system to process novel combinations of relative clauses systematically, while still being sensitive to the semantic constraints.Risto MiikkulainenMarshall R. III Mayberry1998-11-10Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/519This item is in the repository with the URL: http://cogprints.org/id/eprint/5191998-11-10ZDouble Loops Flows and Bidirectional Hebb's Law in Neural NetworkThis paper presents the double loop feedback model, which is used for structure and data flow modelling through reinforcement learning in an artificial neural network. We first consider physiological arguments suggesting that loops and double loops are widely spread in the exchange flows of the central nervous system. We then demonstrate that the double loop pattern, named a mental object, works as a functional memory unit and we describe the main properties of a double loop resonator built with the classical Hebb's law learning principle in a feedforward basis. In this model, we show how some mental objects aggregate themselves in building blocks, then what are the properties of such blocks. We propose the mental objects block as the representing structure of a concept in a neural network. We show how the local application of Hebb's law at the cell level leads to the concept of functional organization cost at the network level (upward effect), which explains spontaneous reorganization of mental blocks (downward effect). In this model, the simple hebbian learning paradigm appears to have emergent effects in both upward and downward directions.Christophe Lecerf1998-03-27Z2011-03-11T08:54:07Zhttp://cogprints.org/id/eprint/621This item is in the repository with the URL: http://cogprints.org/id/eprint/6211998-03-27ZThe emergence of a "language" in an evolving population of neural networksThe evolution of language implies the parallel evolution of an ability to respond appropriately to signals (language understanding) and an ability to produce the appropriate signals in the appropriate circumstances (language production). When linguistic signals are produced to inform other individuals, individuals that respond appropriately to these signals may increase their reproductive chances but it is less clear what is the reproductive advantage for the languages producers. We present simulations in which populations of neural networks living in an environment evolve a simple language with an informative function. Signals are produced to help other individuals to categorize edible and poisonous mushrooms in order to decide whether to approach or avoid encountered mushrooms. Language production, while not under direct evolutionary pressure, evolves as a by-product of the independently evolving perceptual ability to categorize mushrooms.A CangelosiD Parisi2002-09-21Z2011-03-11T08:55:00Zhttp://cogprints.org/id/eprint/2469This item is in the repository with the URL: http://cogprints.org/id/eprint/24692002-09-21ZIntelligent Diagnosis SystemsThis paper examines and compares several different approaches to design of intelligent systems for diagnosis applications. These include expert systems (or knowledge based systems), truth (or reason) maintenance systems, case based reasoning systems, and inductive approaches like decision trees, neural networks (or connectionist systems), and statistical pattern classification systems. Each of these approaches is demonstrated through the design of a system for a simple automobile fault diagnosis task. The paper also discusses the domain characteristics that influence the choice of a specific technique (or combination of techniques) for a given application.Karthik BalakrishnanVasant Honavar1998-06-24Z2011-03-11T08:53:59Zhttp://cogprints.org/id/eprint/477This item is in the repository with the URL: http://cogprints.org/id/eprint/4771998-06-24ZInteractive coordination processes: How the brain accomplishes what we take for granted in computer languagesAn example of sending two messages in an e-mail program reveals a fundamental sequence-construction mechanism by which perceptual categories and motor schema are automatically generalized. By this mechanism, the human brain accomplishes more flexibly what we take for granted in stored-program computers-ordered steps (a sequence of operators in a problem space), variable bindings, conditional statements, and subgoaling.W J. Clancey2000-11-15Z2011-03-11T08:54:26Zhttp://cogprints.org/id/eprint/1106This item is in the repository with the URL: http://cogprints.org/id/eprint/11062000-11-15ZLearning sensory-motor cortical mappings without trainingThis paper shows how the relationship between two arrays of artificial neurons, representing different cortical regions, can be learned. The algorithm enables each neural network to self-organise into a topological map of the domain it represents at the same time as the relationship between these maps is found. Unlike previous methods learning is achieved without a separate training phase; the algorithm which learns the mapping is also that which performs the mapping.Michael SpratlingGillian Hayes2001-04-16Z2011-03-11T08:54:37Zhttp://cogprints.org/id/eprint/1451This item is in the repository with the URL: http://cogprints.org/id/eprint/14512001-04-16ZThe Mind: Embodied, Embedded, but not ExtendedThis commentry focuses on the one major ecumenical theme propounded in Andy Clark's Being There that I find
difficult to accept; this is Clarks advocacy, especially in the third and final part of the book, of
the extended nature of the embedded, embodied mind.Gerard O'Brien1999-05-08Z2011-03-11T08:53:39Zhttp://cogprints.org/id/eprint/84This item is in the repository with the URL: http://cogprints.org/id/eprint/841999-05-08ZMixing Memory and Desire: Want and Will in Neural ModelingValues are critical for intelligent behavior, since values determine interests, and interests determine relevance. Therefore we address relevance and its role in intelligent behavior in animals and machines. Animals avoid exhaustive enumeration of possibilities by focusing on relevant aspects of the environment, which emerge into the (cognitive) foreground, while suppressing irrelevant aspects, which submerge into the background. Nevertheless, the background is not invisible, and aspects of it can pop into the foreground if background processing deems them potentially relevant. Essential to these ideas are questions of how contexts are switched, which defines cognitive/behavioral episodes, and how new contexts are created, which allows the efficiency of foreground/background processing to be extended to new behaviors and cognitive domains. Next we consider mathematical characterizations of the foreground/background distinction, which we treat as a dynamic separation of the concrete space into (approximately) orthogonal subspaces, which are processed differently. Background processing is characterized by large receptive fields which project into a space of relatively low dimension to accomplish rough categorization of a novel stimulus and its approximate location. Such background processing is partly innate and partly learned, and we discuss possible correlational (Hebbian) learning mechanisms. Foreground processing is characterized by small receptive fields which project into a space of comparatively high dimension to accomplish precise categorization and localization of the stimuli relevant to the context. We also consider mathematical models of valences and affordances, which are an aspect of the foreground. Cells processing foregound information have no fixed meaning (i.e., their meaning is contextual), so it is necessary to explain how the processing accomplished by foreground neurons can be made relative to the context. Thus we consider the properties of several simple mathematical models of how the contextual representation controls foreground processing. We show how simple correlational processes accomplish the contextual separation of foreground from background on the basis of differential reinforcement. That is, these processes account for the contextual separation of the concrete space into disjoint subspaces corresponding to the foreground and background. Since an episode may comprise the activation of several contexts (at varying levels of activity) we consider models, suggested by quantum mechanics, of foreground processing in superposition. That is, the contextual state may be a weighted superposition of several pure contexts, with a corresponding superposition of the foreground representations and the processes operating on them. This leads us to a consideration of the nature and origin of contexts. Although some contexts are innate, many are learned. We discuss a mathematical model of contexts which allows a context to split into several contexts, agglutinate from several contexts, or to constellate out of relatively acontextual processing. Finally, we consider the acontextual processing which occurs when the current context is no longer relevant, and may trigger the switch to another context or the formation of a new context. We relate this to the situation known as "breakdown" in phenomenology.Bruce J. MacLennan1998-06-09Z2011-03-11T08:54:11Zhttp://cogprints.org/id/eprint/679This item is in the repository with the URL: http://cogprints.org/id/eprint/6791998-06-09ZModelling Attentional Biases in the Perception of Geometric FormsAttentional biases are observed in all modalities of human perception. For example, experimental participants respond more quickly to particular dimensions of stimuli and to stimuli located in particular positions of the visual and auditory fields. This paper reports robust top-right attentional bias in perception of simple geometric forms and an explanation of this bias in terms of our long experience at reading English text from left to right coupled with the need to adjust attention upwards as we locomote through visual space. Attempts to explain the attentional bias in terms of simple neural networks given training at recognising features scrolling from right to left and top to bottom in the visual field are reported.C LatimerS Gazzard1998-07-30Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/502This item is in the repository with the URL: http://cogprints.org/id/eprint/5021998-07-30ZMultiplicative Binding, Representation Operators & Analogy (Workshop Poster)Analogical inference depends on systematic substitution of the components of compositional structures. Simple systematic substitution has been achieved in a number of connectionist systems that support binding (the ability to create connectionist representations of the combination of component representations). These systems have used various implementations of two generic composition operators: bind() and bundle(). This paper introduces a novel implementation of the bind() operator that is simple, can be efficiently implemented, and highlights the relationship between retrieval queries and analogical mapping. A frame of role/filler bindings can easily be represented using bind() and bundle(). However, typical binding systems are unable to adequately represent multiple frames and arbitrary nested compositional structures. A novel family of representational operators (called braid()) is introduced to address these problems. Other binding systems make the strong assumption that the roles and fillers are disjoint in order to avoid ambiguities inherent in their representational idioms. The braid() operator can be used to avoid this assumption. The new representational idiom suggests how the cognitive processes of bottom-up and top-down object recognition might be implemented. These processes depend on analogical mapping to integrate disjoint representations and drive perceptual search.Ross W. Gayler2000-08-02Z2011-03-11T08:54:22Zhttp://cogprints.org/id/eprint/915This item is in the repository with the URL: http://cogprints.org/id/eprint/9152000-08-02ZNeural implementation of psychological spacesPsychological spaces give natural framework for construction of mental representations. Neural model of psychological spaces provides a link between neuroscience and psychology. Categorization performed in high-dimensional spaces by dynamical associative memory models is approximated with low-dimensional feedforward neural models calculating probability density functions in psychological spaces. Applications to the human categorization experiments are discussed.
Wlodzislaw Duch1998-08-05Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/507This item is in the repository with the URL: http://cogprints.org/id/eprint/5071998-08-05ZNeural networks as a model for visual perception: what is lacking?A central mystery of visual perception is the classical problem of invariant object recognition: Different appearances of an object can be perceived as ``the same'', despite, e.g., changes in position or illumination, distortions, or partial occlusion by other objects. This article reports on a recent email discussion over the question whether a neural network can learn the simplest of these invariances, i.e. generalize over the position of a pattern on the input layer, including the author's view on what ``learning shift-invariance'' could mean. That definition leaves the problem unsolved. A similar problem is the one of learning to detect symmetries present in an input pattern. It has been solved by a standard neural network requiring some 70000 input examples. Both leave some doubt if backpropagation learning is a realistic model for perceptual processes. Abandoning the view that a stimulus-response system showing the desired behavior must be learned from scratch, yields a radically different solution. Perception can be seen as an active process that rapidly converges from some initial state to an ordered state, which in itself codes for a percept. As an example, I will present a solution to the visual correspondence problem, which greatly alleviates both problems mentioned above.Rolf P. Würtz2000-02-22Z2011-03-11T08:53:41Zhttp://cogprints.org/id/eprint/140This item is in the repository with the URL: http://cogprints.org/id/eprint/1402000-02-22ZPattern-Generator-Driven Development in Self-Organizing ModelsSelf-organizing models develop realistic cortical structures when given approximations of the visual environment as input. Recently it has been proposed that internally generated input patterns, such as those found in the developing retina and in PGO waves during REM sleep, may have the same effect. Internal pattern generators would constitute an efficient way to specify, develop, and maintain functionally appropriate perceptual organization. They may help express complex structures from minimal genetic information, and retain this genetic structure within a highly plastic system. Simulations with the RF-LISSOM orientation map model indicate that such preorganization is possible, providing a computational framework for examining how genetic influences interact with visual experience.James A. BednarRisto Miikkulainen2000-08-10Z2011-03-11T08:54:22Zhttp://cogprints.org/id/eprint/913This item is in the repository with the URL: http://cogprints.org/id/eprint/9132000-08-10ZPlatonic model of mind as an approximation to neurodynamicsHierarchy of approximations involved in simplification of microscopic theories, from sub-cellural to the whole brain level, is presented. A new approximation to neural dynamics is described, leading to a Platonic-like model of mind based on psychological spaces. Objects and events in these spaces correspond to quasi-stable states of brain dynamics and may be interpreted from psychological point of view. Platonic model bridges the gap between neurosciences and psychological sciences. Static and dynamic versions of this model are outlined and Feature Space Mapping, a neurofuzzy realization of the static version of Platonic model, described. Categorization experiments with human subjects are analyzed from the neurodynamical and Platonic model points of view. Wlodzislaw Duch2000-11-15Z2011-03-11T08:54:26Zhttp://cogprints.org/id/eprint/1107This item is in the repository with the URL: http://cogprints.org/id/eprint/11072000-11-15ZA self-organising neural network for modelling cortical developmentThis paper presents a novel self-organising neural network. It has been developed for use as a simplified model of cortical development. Unlike many other models of topological map formation all synaptic weights start at
zero strength (so that synaptogenesis might be modelled). In addition, the algorithm works with the same format of encoding for both inputs to and outputs from the network (so that the transfer and recoding of information between cortical regions might be modelled).Michael SpratlingGillian Hayes1998-06-22Z2011-03-11T08:54:12Zhttp://cogprints.org/id/eprint/694This item is in the repository with the URL: http://cogprints.org/id/eprint/6941998-06-22ZThe ``Semantics'' of Evolution: Trajectories and Trade-offs in Design Space and Niche SpaceThis paper attempts to characterise a unifying overview of the practice of software engineers, AI designers, developers of evolutionary forms of computation, designers of adaptive systems, etc. The topic overlaps with theoretical biology, developmental psychology and perhaps some aspects of social theory. Just as much of theoretical computer science follows the lead of engineering intuitions and tries to formalise them, there are also some important emerging high level cross disciplinary ideas about natural information processing architectures and evolutionary mechanisms and that can perhaps be unified and formalised in the future. There is some speculation about the evolution of human cognitive architectures and consciousness.Aaron Sloman1998-07-09Z2011-03-11T08:54:13Zhttp://cogprints.org/id/eprint/716This item is in the repository with the URL: http://cogprints.org/id/eprint/7161998-07-09ZThe ``Semantics'' of Evolution: Trajectories and Trade-offs in Design Space and Niche Space.This paper attempts to characterise a unifying overview of the practice of software engineers, AI designers, developers of evolutionary forms of computation, designers of adaptive systems, etc. The topic overlaps with theoretical biology, developmental psychology and perhaps some aspects of social theory. Just as much of theoretical computer science follows the lead of engineering intuitions and tries to formalise them, there are also some important emerging high level cross disciplinary ideas about natural information processing architectures and evolutionary mechanisms and that can perhaps be unified and formalised in the future. There is some speculation about the evolution of human cognitive architectures and consciousness.A. Sloman1999-01-03Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/524This item is in the repository with the URL: http://cogprints.org/id/eprint/5241999-01-03ZText and Discourse Understanding: The DISCERN SystemThe subsymbolic approach to natural language processing (NLP) captures a number of intriguing properties of human-like information processing such as learning from examples, context sensitivity, generalization, robustness of behavior, and intuitive reasoning. Within this new paradigm, the central issues are quite different from (even incompatible with) the traditional issues in symbolic NLP, and the research has proceeded without much in common with the past. However, the ultimate goal is still the same: to understand how humans process language. Even if NLP is being built on a new foundation, as can be argued, many of the results obtained through symbolic research are still valid, and could be used as a guide for developing subsymbolic models of natural language processing. This is where DISCERN (DIstributed SCript processing and Episodic memoRy Network (Miikkulainen 1993), a subsymbolic neural network model of script-based story understanding, fits in. DISCERN is purely a subsymbolic model, but at the high level it consists of modules and information structures similar to those of symbolic systems, such as scripts, lexicon, and episodic memory. At the highest level of natural language processing such as text and discourse understanding, the symbolic and subsymbolic paradigms have to address the same basic issues. Outlining a subsymbolic approach to those issues is the purpose of DISCERN. In more specific terms, DISCERN aims: (1) to demonstrate that distributed artificial neural networks can be used to build a large-scale natural language processing system that performs approximately at the level of symbolic models; (2) to show that several cognitive phenomena can be explained at the subsymbolic level using the special properties of these networks; and (3) to identify central issues in subsymbolic NLP and to develop well-motivated techniques to deal with them. To the extent that DISCERN is successful in these areas, it constitutes a first step towards building text and discourse understanding systems within the subsymbolic paradigm.Risto Miikkulainen1999-01-07Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/529This item is in the repository with the URL: http://cogprints.org/id/eprint/5291999-01-07ZUne leçon de piano, ou la double boucle de l'apprentissage cognitifThis book is intended as a course support for french DEA/DESS students. Based upon the double loop learning model, the mental object concept is described with its dynamic properties.Lecerf Christophe1998-12-10Z2011-03-11T08:54:16Zhttp://cogprints.org/id/eprint/765This item is in the repository with the URL: http://cogprints.org/id/eprint/7651998-12-10ZWhat changes in children's drawing procedures? Relational complexity as a constraint on representational redescriptionChildren's ability to modify their drawing procedures changes in their first decade. Young children make size/shape changes and end-of-sequence insertions/deletions of drawing elements. Older children also make middle-of-sequence insertions/deletions and position/orientation changes in drawing elements. Why do modifications occur in this order? We argue that older children's modifications require processing ternary relations, which according to a relational complexity theory, is beyond the working memory capacity of young children.S. PhillipsG. S. HalfordW. H. Wilson1998-07-03Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/495This item is in the repository with the URL: http://cogprints.org/id/eprint/4951998-07-03ZWhy feed-forward networks are in a bad shapeIt has often been noted that the learning problem in feed-forward neural networks is very badly conditioned. Although, generally, the special form of the transfer function is usually taken to be the cause of this condition, we show that it is caused by the manner in which neurons are connected. By analyzing the expected values of the Hessian in a feed-forward network it is shown that, even in a network where all the learning samples are well chosen and the transfer function is not in its saturated state, the system has a non-optimal condition. We subsequently propose a change in the feed-forward network structure which alleviates this problem. We finally demonstrate the positive influence of this approach.P. van der SmagtG. Hirzinger1998-07-03Z2011-03-11T08:53:38Zhttp://cogprints.org/id/eprint/52This item is in the repository with the URL: http://cogprints.org/id/eprint/521998-07-03ZCan artificial cerebellar models compete to control robots?Contains extended abstracts of the NIPS*97 workshop "Can Artificial Models Compete to Control Robots?"P. van der SmagtD. Bullock1998-03-27Z2011-03-11T08:53:36Zhttp://cogprints.org/id/eprint/9This item is in the repository with the URL: http://cogprints.org/id/eprint/91998-03-27ZIn search of common foundations for cortical computationThis research concerns forms of coding, processing and learning that are common to many different cortical regions and cognitive functions. Local cortical processors may coordinate their activity by maximizing the transmission of information that is coherently related to the context in which it occurs, thereby forming synchronized population codes. In this coordination, contextual field (CF) connections link processors within and between cortical regions. The effects of CF connections are distinct from those mediating receptive field (RF) input. CFs can guide both learning and processing without becoming confused with RF information. Simulations explore the capabilities of networks built from local processors with both RF and CF connections. Physiological evidence for CFs, synchronization, and plasticity in RF and CF connections is described. Coordination via CFs is related to perceptual grouping, the effects of context on contrast sensitivity, amblyopia, implicit influences of color in achromotopsia, object and word perception, and the discovery of distal environmental variables and their interactions through self-organization. In cortical computation there may occur a flexible evaluation of relations between input signals by locally specialized but adaptive processors whose activity is dynamically associated and coordinated within and between regions through specialized contextual connections.W A PhillipsW Singer1999-01-27Z2011-03-11T08:54:17Zhttp://cogprints.org/id/eprint/794This item is in the repository with the URL: http://cogprints.org/id/eprint/7941999-01-27ZThe Origin and Evolution of Culture and CreativityLike the information patterns that evolve through biological processes, mental representations, or memes, evolve through adaptive exploration and transformation of an information space through variation, selection, and transmission. Since unlike genes, memes do not come packaged with instructions for their replication, our brains do it for them, strategically, guided by a fitness landscape that reflects both internal drives and a worldview that is continually updated through meme assimilation. This paper presents a model for how an individual becomes a meme-evolving agent via the emergence of an autocatalytic network of sparse, distributed memories, and discusses implications for complex, creative thought processes and why they are unique to humans. Memetics can do more than account for the spread of catchy tunes; it can pave the way for the kind of overarching framework for the humanities that the first form of evolution has provided for the biological sciences.L. Gabora1998-04-28Z2011-03-11T08:53:56Zhttp://cogprints.org/id/eprint/437This item is in the repository with the URL: http://cogprints.org/id/eprint/4371998-04-28ZModeling dynamic receptive field changes produced by intracortical microstimulationIntracortical microstimulation (ICMS) of a localized site in the somatosensory cortex of rats and monkeys for 2-6 hours produces a large increase in the cortical representation of the skin region represented by the ICMS-site neurons before ICMS, with very little effect on the ICMS-site neuron's RF location, RF size, and responsiveness (Recanzone et al., 1992). The "EXIN" (afferent excitatory and lateral inhibitory) learning rules (Marshall, 1995) are used to model RF changes during ICMS. The EXIN model produces reorganization of RF topography similar to that observed experimentally. The possible role of inhibitory learning in producing the effects of ICMS is studied by simulating the EXIN model with only lateral inhibitory learning. The model also produces an increase in the cortical representation of the skin region represented by the ICMS-site RF. ICMS is compared to artificial scotoma conditioning (Pettet & Gilbert, 1992) and retinal lesions (Darian-Smith & Gilbert, 1995), and it is suggested that lateral inhibitory learning may be a general principle of cortical plasticity.G.J. KalarickalJ.A. Marshall2003-02-10Z2011-03-11T08:55:09Zhttp://cogprints.org/id/eprint/2764This item is in the repository with the URL: http://cogprints.org/id/eprint/27642003-02-10ZTowards a unified model of cortical computation II: From control architecture to a model of consciousnessThe recently introduced Static and Dynamic State (SDS)
Feedback control scheme together with its modified form, the Data Compression and Reconstruction (DCR) architecture that performs pseudoinverse computation, suggests a unified model of cortical processing including consciousness. The constraints of the model are outlined were and the features of the cortical architecture that are suggested and sometimes dictated by these constraints are listed. Constraints are imposed on cortical layers, e.g., (1) the model prescribes a connectivity substructure that is shown to fit the main properties of the `basic neural circuit' of the cerebral cortex (Shepherd and Koch, 1990, Douglas and Martin 1990). In: The synaptic organization of the brain, Oxford University Press, 1990), and (2) the stability requirements of the pseudoinverse method offer an explanation for the columnar organization of the cortex. Constraints are also imposed on the hierarchy of cortical areas, e.g., the proposed control architecture requires computations of the control variables belonging to both the `desired' and the experienced' moves as well as a `sign-proper' separation of feedback channels that fit known properties of the basal ganglia -- thalamocortical loops (Lorincz, 1997). An outline is given as to how the DCR scheme can be extended towards a model for consciousness that can deal with the `homunculus fallacy' by resolving the fallacy and saving he homunculus as an inherited and learnt partially ordered list of preferences.Andras Lorinczandras.lorincz1998-06-15Z2011-03-11T08:53:58Zhttp://cogprints.org/id/eprint/458This item is in the repository with the URL: http://cogprints.org/id/eprint/4581998-06-15ZBootstrapping knowledge representations: from entailment meshes via semantic nets to learning websThe symbol-based, correspondence epistemology used in AI is contrasted with the constructivist, coherence epistemology promoted by cybernetics. The latter leads to bootstrapping knowledge representations, in which different parts of the cognitive system mutually support each other. Gordon Pask's entailment meshes and their implementation in the ThoughtSticker program are reviewed as a basic application of this methodology. Entailment meshes are then extended to entailment nets: directed graph representations governed by the "bootstrapping axiom", determining which concepts are to be distinguished or merged. This allows a constant restructuring and elicitation of the conceptual network. Semantic networks and frame-like representations with inheritance can be expressed in this very general scheme by introducing a basic ontology of node and link types. Entailment nets are then generalized to associative nets characterized by weighted links. Learning algorithms are presented which can adapt the link strengths, based on the frequency with which links are selected by hypertext browsers. It is argued that these different bootstrapping methods could be applied to make the World-Wide Web more intelligent, by allowing it to self-organize and support inferences through spreading activation.Francis Heylighen1999-10-08Z2011-03-11T08:54:03Zhttp://cogprints.org/id/eprint/551This item is in the repository with the URL: http://cogprints.org/id/eprint/5511999-10-08ZCombining Neural Network Forecasts on Wavelet-Transformed Time SeriesWe discuss a simple strategy aimed at improving neural network prediction accuracy, based on the combination of predictions at varying resolution levels of the domain under investigation (here: time series). First, a wavelet transform is used to decompose the time series into varying scales of temporal resolution. The latter provide a sensible decomposition of the data so that the underlying temporal structures of the original time series become more tractable. Then, a Dynamical Recurrent Neural Network (DRNN) is trained on each resolution scale with the temporal-recurrent backpropagation (TRBP) algorithm. By virtue of its internal dynamic, this general class of dynamic connectionist network approximates the underlying law governing each resolution level by a system of nonlinear difference equations. The individual wavelet scale forecasts are afterwards recombined to form the current estimate. The predictive ability of this strategy is assessed with the sunspot series.Alex AussemFionn Murtagh1999-01-03Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/527This item is in the repository with the URL: http://cogprints.org/id/eprint/5271999-01-03ZConvergence-Zone Episodic Memory: Analysis and SimulationsHuman episodic memory provides a seemingly unlimited storage for everyday experiences, and a retrieval system that allows us to access the experiences with partial activation of their components. The system is believed to consist of a fast, temporary storage in the hippocampus, and a slow, long-term storage within the neocortex. This paper presents a neural network model of the hippocampal episodic memory inspired by Damasio's idea of Convergence Zones. The model consists of a layer of perceptual feature maps and a binding layer. A perceptual feature pattern is coarse coded in the binding layer, and stored on the weights between layers. A partial activation of the stored features activates the binding pattern, which in turn reactivates the entire stored pattern. For many configurations of the model, a theoretical lower bound for the memory capacity can be derived, and it can be an order of magnitude or higher than the number of all units in the model, and several orders of magnitude higher than the number of binding-layer units. Computational simulations further indicate that the average capacity is an order of magnitude larger than the theoretical lower bound, and making the connectivity between layers sparser causes an even further increase in capacity. Simulations also show that if more descriptive binding patterns are used, the errors tend to be more plausible (patterns are confused with other similar patterns), with a slight cost in capacity. The convergence-zone episodic memory therefore accounts for the immediate storage and associative retrieval capability and large capacity of the hippocampal memory, and shows why the memory encoding areas can be much smaller than the perceptual maps, consist of rather coarse computational units, and be only sparsely connected to the perceptual maps.Mark MollRisto Miikkulainen1999-01-03Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/523This item is in the repository with the URL: http://cogprints.org/id/eprint/5231999-01-03ZDyslexic and Category-Specific Impairments in a Self-Organizing Feature Map Model of the LexiconDISLEX is an artificial neural network model of the mental lexicon. It was built to test computationally whether the lexicon could consist of separate feature maps for the different lexical modalities and the lexical semantics, connected with ordered pathways. In the model, the orthographic, phonological, and semantic feature maps and the associations between them are formed in an unsupervised process, based on cooccurrence of the lexical symbol and its meaning. After the model is organized, various damage to the lexical system can be simulated, resulting in dyslexic and category-specific aphasic impairments similar to those observed in human patients.Risto Miikkulainen2011-12-16T00:11:43Z2011-12-16T00:11:43Zhttp://cogprints.org/id/eprint/7709This item is in the repository with the URL: http://cogprints.org/id/eprint/77092011-12-16T00:11:43ZThe Many Functions of Discourse Particles: A Computational Model of Pragmatic InterpretationWe present a connectionist model for the interpretation of discourse
particles in real dialogues that is based on neuronal
principles of categorization (categorical perception, prototype
formation, contextual interpretation). It can be shown that
discourse particles operate just like other morphological and
lexical items with respect to interpretation processes. The description
proposed locates discourse particles in an elaborate
model of communication which incorporates many different
aspects of the communicative situation. We therefore also
attempt to explore the content of the category discourse particle.
We present a detailed analysis of the meaning assignment
problem and show that 80%– 90% correctness for unseen discourse
particles can be reached with the feature analysis provided.
Furthermore, we show that ‘analogical transfer’ from
one discourse particle to another is facilitated if prototypes
are computed and used as the basis for generalization. We
conclude that the interpretation processes which are a part of
the human cognitive system are very similar with respect to
different linguistic items. However, the analysis of discourse
particles shows clearly that any explanatory theory of language
needs to incorporate a theory of communication processes.Gabriele Schelergscheler@gmail.comKerstin Fischer1998-04-28Z2011-03-11T08:53:37Zhttp://cogprints.org/id/eprint/24This item is in the repository with the URL: http://cogprints.org/id/eprint/241998-04-28ZModeling dynamic receptive field changes in primary visual cortex using inhibitory learningThe position, size, and shape of the visual receptive field (RF) of some primary visual cortical neurons change dynamically, in response to artificial scotoma conditioning in cats (Pettet & Gilbert, 1992) and to retinal lesions in cats and monkeys (Darian-Smith & Gilbert, 1995). The "EXIN" learning rules (Marshall, 1995) are used to model dynamic RF changes. The EXIN model is compared with an adaptation model (Xing & Gerstein, 1994) and the LISSOM model (Sirosh & Miikkulainen, 1994; Sirosh et al., 1996). To emphasize the role of the lateral inhibitory learning rules, the EXIN and the LISSOM simulations were done with only lateral inhibitory learning. During scotoma conditioning, the EXIN model without feedforward learning produces centrifugal expansion of RFs initially inside the scotoma region, accompanied by increased responsiveness, without changes in spontaneous activation. The EXIN model without feedforward learning is more consistent with the neurophysiological data than are the adaptation model and the LISSOM model. The comparison between the EXIN and the LISSOM models suggests experiments to determine the role of feedforward excitatory and lateral inhibitory learning in producing dynamic RF changes during scotoma conditioning.J.A. MarshallG.J. Kalarickal1997-12-05Z2011-03-11T08:54:05Zhttp://cogprints.org/id/eprint/586This item is in the repository with the URL: http://cogprints.org/id/eprint/5861997-12-05ZThe Psychophysics of Synthetic Categorical PerceptionStudies of the categorical perception (CP) of sensory continua have a long and rich history in psychophysics. A major development was Macmillan et al.'s application in 1977 of signal detection theory to analyze several experimental paradigms, in particular explicating the relation between the psychometric labeling function and discrimination measures. Simultaneously, Anderson et al. proposed a neural model for what we will term synthetic CP, yet this line of research has been less well explored. In this paper, we assess neural-network models of CP with particular reference to their ability to predict the psychophysical behavior of real observers -- including the relation between labeling and discrimination. Synthetic categorization of a variety of stimuli, including speech sounds and artificial/novel dimensions, is reviewed and discussed in terms of both classical theories of CP and more recent developments. A variety of neural mechanisms is capable of replicating the essentials of categorical perception, indicating that CP is not a special mode of perception but an emergent property of any sufficiently-powerful general learning system. However, the most convincing replication is from a simulation whose output is continuous rather than discrete.R I DamperStevan Harnad1999-01-03Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/525This item is in the repository with the URL: http://cogprints.org/id/eprint/5251999-01-03ZSelf-Organization, Plasticity, and Low-level Visual Phenomena in a Laterally Connected Map Model of the Primary Visual CortexBased on a Hebbian adaptation process, the afferent and lateral connections in the RF-LISSOM model organize simultaneously and cooperatively, and form structures such as those observed in the primary visual cortex. The neurons in the model develop local receptive fields that are organized into orientation, ocular dominance, and size selectivity columns. At the same time, patterned lateral connections form between neurons that follow the receptive field organization. This structure is in a continuously-adapting dynamic equilibrium with the external and intrinsic input, and can account for reorganization of the adult cortex following retinal and cortical lesions. The same learning processes may be responsible for a number of low-level functional phenomena such as tilt aftereffects, and combined with the leaky integrator model of the spiking neuron, for segmentation and binding. The model can also be used to verify quantitatively the hypothesis that the visual cortex forms a sparse, redundancy-reduced encoding of the input, which allows it to process massive amounts of visual information efficiently.Risto MiikkulainenJames A. BednarYoonsuck ChoeJoseph Sirosh1998-12-09Z2011-03-11T08:54:16Zhttp://cogprints.org/id/eprint/763This item is in the repository with the URL: http://cogprints.org/id/eprint/7631998-12-09ZSystematicity: Psychological evidence with connectionist implicationsAt root, the systematicity debate over classical versus connectionist explanations for cognitive architecture turns on quantifying the degree to which human cognition is systematic. We introduce into the debate recent psychological data that provides strong support for the purely structure-based generalizations claimed by Fodor and Pylyshyn (1988). We then show, via simulation, that two widely used connectionist models (feedforward and simple recurrent networks) do not capture the same degree of generalization as human subjects. However, we show that this limitation is overcome by tensor networks that support relational processing.S. PhillipsG. S. Halford1998-07-03Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/494This item is in the repository with the URL: http://cogprints.org/id/eprint/4941998-07-03ZTeaching a robot to see how it movesThe positioning of a robot hand in order to grasp an object is a problem fundamental to robotics. The task we want to perform can be described as follows: given a visual scene the robot arm must reach an indicated point in that visual scene. This marked point indicates the observed object that has to be grasped. In order to accomplish this task, a mapping from the visual scene to the corresponding robot joint values must be available. The task set out in this chapter is to design a self-learning controller that constructs that mapping without knowledge of the geometry of the camera-robot system.P. van der Smagt1998-07-03Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/493This item is in the repository with the URL: http://cogprints.org/id/eprint/4931998-07-03ZVisual feedback in motionIn this chapter we introduce a method for model-free monocular visual guidance of a robot arm. The robot arm, with a single camera in its end effector, should be positioned above a stationary target. It is shown that a trajectory can be planned in visual space by using components of the optic flow, and this trajector can be translated to joint torques by a self-learning neural network. No model of the robot, camera, or environment is used. The method reaches a high grasping accuracy after only a few trials.P. van der SmagtF. Groen1999-01-03Z2011-03-11T08:54:01Zhttp://cogprints.org/id/eprint/526This item is in the repository with the URL: http://cogprints.org/id/eprint/5261999-01-03ZVisual Schemas in Neural Networks for Object Recognition and Scene AnalysisVISOR is a large connectionist system that shows how visual schemas can be learned, represented, and used through mechanisms natural to neural networks. Processing in VISOR is based on cooperation, competition, and parallel bottom-up and top-down activation of schema representations. Simulations show that VISOR is robust against noise and variations in the inputs and parameters. It can indicate the confidence of its analysis, pay attention to important minor differences, and use context to recognize ambiguous objects. Experiments also suggest that the representation and learning are stable, and its behavior is consistent with human processes such as priming, perceptual reversal, and circular reaction in learning. The schema mechanisms of VISOR can serve as a starting point for building robust high-level vision systems, and perhaps for schema-based motor control and natural language processing systems as well. </blockquote>Wee Kheng LeowRisto Miikkulainen2001-06-19Z2011-03-11T08:54:42Zhttp://cogprints.org/id/eprint/1608This item is in the repository with the URL: http://cogprints.org/id/eprint/16082001-06-19ZWarping Similarity Space in Category Learning by Backprop NetsWe report simulations with backpropagation networks trained to discriminate and then categorize a set
of stimuli. The findings suggest a possible mechanism for categorical perception based on altering
interstimulus similarity. A. TijsselingStevan Harnad1998-04-28Z2011-03-11T08:53:37Zhttp://cogprints.org/id/eprint/25This item is in the repository with the URL: http://cogprints.org/id/eprint/251998-04-28ZNeural model of visual stereomatching: Slant, transparency, and cloudsStereomatching of oblique and transparent surfaces is described using a model of cortical binocular "tuned" neurons selective for disparities of individual visual features and neurons selective for the position, depth, and 3-D orientation of local surface patches. The model is based on a simple set of learning rules. In the model, monocular neurons project excitatory connection pathways to binocular neurons at appropriate disparities. Binocular neurons project excitatory connection pathways to appropriately tuned "surface patch" neurons. The surface patch neurons project reciprocal excitatory connection pathways to the binocular neurons. Anisotropic intralayer inhibitory connection pathways project between neurons with overlapping receptive fields. The model's responses to simulated stereo image pairs depicting a variety of oblique surfaces and transparently overlaid surfaces are presented. For all the surfaces, the model (1) assigns disparity matches and surface patch representations based on global surface coherence and uniqueness, (2) permits coactivation of neurons representing multiple disparities within the same image location, (3) represents oblique slanted and tilted surfaces directly, rather than approximating them with a series of frontoparallel steps, (4) assigns disparities to a cloud of points at random depths, like human observers and unlike Prazdny's (1985) method, and (5) causes globally consistent matches to override greedy local matches. The model represents transparency, unlike the model of Marr and Poggio (1976), and it assigns unique disparities, unlike the model of Prazdny (1985).J.A. MarshallG.J. KalarickalGraves E.B.1998-07-03Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/492This item is in the repository with the URL: http://cogprints.org/id/eprint/4921998-07-03ZAnalysis and control of a rubbertuator armThe control of light-weight compliant robot arms is cumbersome due to the fact that their Coriolis forces are large, and the forces exerted by the relatively weak actuators may change in time due to external (e.g., temperature) influences. We describe and analyse the behaviour of a light-weight robot arm, the SoftArm robot. It is found that the hysteretic force-position relationship of the arm can be explained from its structure. This knowledge is used in the construction of a neural-network based controller. Experiments show that the network is able to accurately control the robot arm after a training session of only a few minutes.P. van der SmagtF. GroenK. Schulten1998-06-22Z2011-03-11T08:54:12Zhttp://cogprints.org/id/eprint/704This item is in the repository with the URL: http://cogprints.org/id/eprint/7041998-06-22ZBeyond Turing EquivalenceWhat is the relation between intelligence and computation? Although the difficulty of defining `intelligence' is widely recognized, many are unaware that it is hard to give a satisfactory definition of `computational' if computation is supposed to provide a non-circular explanation for intelligent abilities. The only well-defined notion of `computation' is what can be generated by a Turing machine or a formally equivalent mechanism. This is not adequate for the key role in explaining the nature of mental processes, because it is too general, as many computations involve nothing mental, nor even processes: they are simply abstract structures. We need to combine the notion of `computation' with that of `machine'. This may still be too restrictive, if some non-computational mechanisms prove to be useful for intelligence. We need a theory-based taxonomy of {\em architectures} and {\em mechanisms} and corresponding process types. Computational machines my turn out to be a sub-class of the machines available for implementing intelligent agents. The more general analysis starts with the notion of a system with independently variable, causally interacting sub-states that have different causal roles, including both `belief-like' and `desire-like' sub-states, and many others. There are many significantly different such architectures. For certain architectures (including simple computers), some sub-states have a semantic interpretation for the system. The relevant concept of semantics is defined partly in terms of a kind of Tarski-like structural correspondence (not to be confused with isomorphism). This always leaves some semantic indeterminacy, which can be reduced by causal loops involving the environment. But the causal links are complex, can share causal pathways, and always leave mental states to some extent semantically indeterminate.Aaron Sloman2000-08-02Z2011-03-11T08:54:22Zhttp://cogprints.org/id/eprint/914This item is in the repository with the URL: http://cogprints.org/id/eprint/9142000-08-02ZComputational physics of the mindIn the XIX century and earlier such physicists as Newton, Mayer, Hooke, Helmholtz and Mach were actively engaged in the research on psychophysics, trying to relate psychological sensations to intensities of physical stimuli. Computational physics allows to simulate complex neural processes giving a chance to answer not only the original psychophysical questions but also to create models of mind. In this paper several approaches relevant to modeling of mind are outlined. Since direct modeling of the brain functions is rather limited due to the complexity of such models a number of approximations is introduced. The path from the brain, or computational neurosciences, to the mind, or cognitive sciences, is sketched, with emphasis on higher cognitive functions such as memory and consciousness. No fundamental problems in understanding of the mind seem to arise. From computational point of view realistic models require massively parallel architectures. Wlodzislaw Duch1998-04-28Z2011-03-11T08:53:56Zhttp://cogprints.org/id/eprint/438This item is in the repository with the URL: http://cogprints.org/id/eprint/4381998-04-28ZLearning to predict visibility and invisibility from occlusion eventsVisual occlusion events constitute a major source of depth information. This paper presents a self-organizing neural network that learns to detect, represent, and predict the visibility and invisibility relationships that arise during occlusion events, after a period of exposure to motion sequences containing occlusion and disocclusion events. The network develops two parallel opponent channels or "chains" of lateral excitatory connections for every resolvable motion trajectory. One channel, the "On" chain or "visible" chain, is activated when a moving stimulus is visible. The other channel, the "Off" chain or "invisible" chain, carries a persistent, amodal representation that predicts the motion of a formerly visible stimulus that becomes invisible due to occlusion. The learning rule uses disinhibition from the On chain to trigger learning in the Off chain. The On and Off chain neurons can learn separate associations with object depth ordering. The results are closely related to the recent discovery (Assad & Maunsell, 1995) of neurons in macaque monkey posterior parietal cortex that respond selectively to inferred motion of invisible stimuli.J A MarshallR K AlleyR S Hubbard1998-04-29Z2011-03-11T08:54:10Zhttp://cogprints.org/id/eprint/654This item is in the repository with the URL: http://cogprints.org/id/eprint/6541998-04-29ZOrthographic Processing in Visual Word Recognition: A Multiple Read-Out ModelA model of orthographic processing is described that postulates read-out from different information dimensions, determined by variable response criteria set on these dimensions. Performance in a perceptual identification task is simulated as the percentage of trials on which a noisy criterion set on the dimension of single word detector activity is reached. Two additional criteria set on the dimensions of total lexical activity and time from stimulus onset are hypothesized to be operational in the lexical decision task. These additional criteria flexibly adjust to changes in stimulus material and task demands, thus accounting for strategic influences on performance in this task. The model unifies results obtained in response-limited and data-limited paradigms, and helps resolve a number of inconsistencies in the experimental literature that cannot be accommodated by other current models of visual word recognition.Jonathan GraingerArthur M. Jacobs1998-06-22Z2011-03-11T08:53:58Zhttp://cogprints.org/id/eprint/470This item is in the repository with the URL: http://cogprints.org/id/eprint/4701998-06-22ZRetrieval properties of attractor neural networks that obey Dale's law using a self-consistent signal-to-noise analysisThe recently proposed self-consistent signal-to-noise analysis is applied to a current--rate dynamics attractor network of excitatory neurons with a Hebbian synaptic matrix. The effect of inhibitory interneurons is included by a term modeling their effective inhibition that depends upon both the level of activity of the excitatory neurons and the stored patterns. The low rate attractor structure is identified, and at low loading the network retrieves single patterns with uniform low rates without errors, and is stable to the admixture of additional patterns. The self-consistent signal-to-noise method enables the analysis of the network properties with an extensive number of patterns, and the results are compared with simulations. The method allows the identification of the fixed point structure of networks for which there is no Lyapanov function, and hence for which mean-field techniques cannot be used. This analysis is shown to provide a powerful and straightforward way to analyse the properties of networks with neuronal specificity, low spike rates and synaptic noise, as well as incorporating the effects of random asymmetric synaptic dilution and limited analog synaptic depth in a natural way. The simulations show that the network properties are very robust both to errors in the stimulus and to the stimulus strength and duration.Anthony N. Burkitt2001-11-18Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1897This item is in the repository with the URL: http://cogprints.org/id/eprint/18972001-11-18ZSubsymbolic Case-Role Analysis
of Sentences with Embedded ClausesA distributed neural network model called SPEC for processing sentences with recursive relative clauses is described. The model is based on separating the tasks of segmenting the input word sequence into clauses, forming the case-role representations, and keeping track of the recursive embeddings into different modules. The system needs to be trained only with the basic sentence constructs, and it generalizes not only to new instances of familiar relative clause structures, but to novel structures as well. SPEC exhibits plausible memory degradation as the depth of the center embeddings increases, its memory is primed by earlier constituents, and its performance is aided by semantic constraints between the constituents. The ability to process structure is largely due to a central executive network that monitors and controls the execution of the entire system. This way, in contrast to earlier subsymbolic systems, parsing is modeled as a controlled high-level process rather than one based on automatic reflex responses.
Risto Miikkulainen1999-10-18Z2011-03-11T08:54:04Zhttp://cogprints.org/id/eprint/552This item is in the repository with the URL: http://cogprints.org/id/eprint/5521999-10-18ZAI: Inventing a new kind of machine.A means-ends approach to engineering an artificial intelligence machine now suggests that we focus on the differences between human capabilities and the best computer programs. These differences suggest two basic limitations in the "symbolic" approach. First, human memory is much more than a storehouse where structures are put away, indexed, and rotely retrieved. Second, human reasoning involves more than searching, matching, and recombining previously stored descriptions of situations and action plans. Indeed, these hypotheses are related: Remembering and reasoning both involve reconceptualization. This short paper outlines recent work in situated cognition, robotics, and neural networks that suggests we frame the problem if AI in terms of inventing a new kind of machine.William J. Clancey1998-04-28Z2011-03-11T08:53:57Zhttp://cogprints.org/id/eprint/440This item is in the repository with the URL: http://cogprints.org/id/eprint/4401998-04-28ZAdaptive perceptual pattern recognition by self-organizing neural networks: Context, uncertainty, multiplicity, and scaleA new context-sensitive neural network, called an "EXIN" (excitatory+ inhibitory) network, is described. EXIN networks self-organize in complex perceptual environments, in the presence of multiple superimposed patterns, multiple scales, and uncertainty. The networks use a new inhibitory learning rule, in addition to an excitatory learning rule, to allow superposition of multiple simultaneous neural activations (multiple winners), under strictly regulated circumstances, instead of forcing winner-take-all pattern classifications. The multiple activations represent uncertainty or multiplicity in perception and pattern recognition. Perceptual scission (breaking of linkages) between independent category groupings thus arises and allows effective global context-sensitive segmentation constraint satisfaction, and exclusive credit attribution. A Weber Law neuron-growth rule lets the network learn and classify input patterns despite variations in their spatial scale. Applications of the new techniques include segmentation of superimposed auditory or biosonar signals, segmentation of visual regions, and representation of visual transparency.J.A. Marshall2003-04-26Z2011-03-11T08:55:15Zhttp://cogprints.org/id/eprint/2905This item is in the repository with the URL: http://cogprints.org/id/eprint/29052003-04-26ZAutomated Understanding of Financial Statements Using Neural Networks and Semantic GrammarsThis article discusses how neural networks and semantic grammars may be used to locate and understand financial statements embedded in news stories received from on-line news wires. A neural net is used to identify where in the news story a financial statement appears to begin. A grammar then is applied to this text in an effort to extract specific facts from the financial statement. Applying grammars to financial statements presents unique parsing problems since the dollar amounts of financial statements are typically arranged in multiple columns, with small paragraphs of text above each column. Text therefore is meant to be read both vertically and horizontally, in contrast to ordinary news text, which is read only horizontally.J. S. Markovitch1999-10-06Z2011-03-11T08:54:03Zhttp://cogprints.org/id/eprint/548This item is in the repository with the URL: http://cogprints.org/id/eprint/5481999-10-06ZDynamical Recurrent Neural Networks: Towards Environmental Time Series Prediction}Dynamical Recurrent Neural Networks (DRNN) (Aussem 1995a) are a class of fully recurrent networks obtained by modeling synapses as autoregressive filters. By virtue of their internal dynamic, these networks approximate the underlying law governing the time series by a system of nonlinear difference equations of internal variables. They therefore provide history-sensitive forecasts without having to be explicitly fed with external memory. The model is trained by a local and recursive error propagation algorithm called temporal-recurrent-backpropagation. The efficiency of the procedure benefits from the exponential decay of the gradient terms backpropagated through the adjoint network. We assess the predictive ability of the DRNN model with meteorological and astronomical time series recorded around the candidate observation sites for the future VLT telescope. The hope is that reliable environmental forecasts provided with the model will allow the modern telescopes to be preset, a few hours in advance, in the most suited instrumental mode. In this perspective, the model is first appraised on precipitation measurements with traditional nonlinear AR and ARMA techniques using feedforward networks. Then we tackle a complex problem, namely the prediction of astronomical seeing, known to be a very erratic time series. A fuzzy coding approach is used to reduce the complexity of the underlying laws governing the seeing. Then, a fuzzy correspondence analysis is carried out to explore the internal relationships in the data. Based on a carefully selected set of meteorological variables at the same time-point, a nonlinear multiple regression, termed {\em nowcasting} (Murtagh et al.\ 1993, 1995), is carried out on the fuzzily coded seeing records. The DRNN is shown to outperform the fuzzy {\em k}-nearest neighbors method.A. AussemF. MurtaghM. Sarazin2001-06-19Z2011-03-11T08:54:41Zhttp://cogprints.org/id/eprint/1593This item is in the repository with the URL: http://cogprints.org/id/eprint/15932001-06-19ZGrounding symbols in sensorimotor categories with neural networksIt is unlikely that the systematic, compositional properties of formal symbol systems -- i.e., of
computation -- play no role at all in cognition. However, it is equally unlikely that cognition is just
computation, because of the symbol grounding problem (Harnad 1990): The symbols in a symbol
system are systematically interpretable, by external interpreters, as meaning something, and that is a
remarkable and powerful property of symbol systems. Cognition (i.e., thinking), has this property too:
Our thoughts are systematically interpretable by external interpreters as meaning something. However,
unlike symbols in symbol systems, thoughts mean what they mean autonomously: Their meaning does
not consist of or depend on anyone making or being able to make any external interpretations of them
at all. When I think "the cat is on the mat," the meaning of that thought is autonomous; it does not
depend on YOUR being able to interpret it as meaning that (even though you could interpret it that
way, and you would be right).Stevan Harnad2001-06-19Z2011-03-11T08:54:41Zhttp://cogprints.org/id/eprint/1596This item is in the repository with the URL: http://cogprints.org/id/eprint/15962001-06-19ZLearned Categorical Perception in Neural Nets: Implications for Symbol GroundingAfter people learn to sort objects into categories they see them differently. Members of
the same category look more alike and members of different categories look more different. This
phenomenon of within-category compression and between-category separation in similarity space is
called categorical perception (CP). It is exhibited by human subjects, animals and neural net models.
In backpropagation nets trained first to auto-associate 12 stimuli varying along a one-dimensional
continuum and then to sort them into 3 categories, CP arises as a natural side-effect because of four
factors: (1) Maximal interstimulus separation in hidden-unit space during auto-association learning, (2)
movement toward linear separability during categorization learning, (3) inverse-distance repulsive force
exerted by the between-category boundary, and (4) the modulating effects of input iconicity, especially
in interpolating CP to untrained regions of the continuum. Once similarity space has been "warped" in
this way, the compressed and separated "chunks" have symbolic labels which could then be combined
into symbol strings that constitute propositions about objects. The meanings of such symbolic
representations would be "grounded" in the system's capacity to pick out from their sensory
projections the object categories that the propositions were about. Stevan HarnadStephen J. HansonJoseph Lubin1999-01-22Z2011-03-11T08:54:02Zhttp://cogprints.org/id/eprint/531This item is in the repository with the URL: http://cogprints.org/id/eprint/5311999-01-22ZMeme and Variations: A Computational Model of Cultural EvolutionThis paper describes a computational model of how ideas, or memes, evolve through the processes of variation, selection, and replication. Every iteration, each neural-network based agent in an artificial society has the opportunity to acquire a new meme, either through 1) INNOVATION, by mutating a previously-learned meme, or 2) IMITATION, by copying a meme performed by a neighbor. Imitation, mental simulation, and using past experience to bias mutation all increase the rate at which fitter memes evolve. Memes at epistatic loci converged more slowly than memes at over- or underdominant loci. The higher the ratio of innovation to imitation, the greater the meme diversity, and the higher the fitness of the fittest meme. Optimization is fastest for the society as a whole with an innovation to imitation ratio of 2:1, but diversity is comprimized.L. Gabora2004-11-20Z2011-03-11T08:55:43Zhttp://cogprints.org/id/eprint/3940This item is in the repository with the URL: http://cogprints.org/id/eprint/39402004-11-20ZMultiNeuron - Neural Networks Simulator for Medical, Physiological, and Psychological Applications This work describes neural software applied in medicine and physiology to:
- investigate and diagnose immune deficiencies; diagnose and study allergic and pseudoallergic reactions; forecast emergence or aggravation of stagnant cardiac insufficiency in patients with cardiac rhythm disorders; forecast development of cardiac arrhythmia after myocardial infarction; reveal relationships between the accumulated radiation dose and a set of immunological, hormonal, and bio-chemical parameters of human blood and find a method to be able to judge by these parameters the dose value; propose a technique for early diagnosis of chor-oid melanomas; Neural networks help also to predict human relations within a group.
Alexander N. GorbanDmitrii A. RossiyevMikhail G. Dorrer1998-06-22Z2011-03-11T08:54:12Zhttp://cogprints.org/id/eprint/706This item is in the repository with the URL: http://cogprints.org/id/eprint/7061998-06-22ZMusings on the roles of logical and non-logical representations in intelligenceThis paper offers a short and biased overview of the history of discussion and controversy about the role of different forms of representation in intelligent agents. It repeats and extends some of the criticisms of the `logicist' approach to AI that I first made in 1971, while also defending logic for its power and generality. It identifies some common confusions regarding the role of visual or diagrammatic reasoning including confusions based on the fact that different forms of representation may be used at different levels in an implementation hierarchy. This is contrasted with the way in the use of one form of representation (e.g. pictures) can be {\em controlled} using another (e.g. logic, or programs). Finally some questions are asked about the role of metrical information in biological visual systems.Aaron Sloman1998-12-09Z2011-03-11T08:54:16Zhttp://cogprints.org/id/eprint/764This item is in the repository with the URL: http://cogprints.org/id/eprint/7641998-12-09ZThe processing of associations versus the processing of relations and symbols: A systematic comparisonA mathematical basis is proposed for the distinction between associative and relational (symbolic) processing. Associations can be contrasted with relations in terms of ordered pairs versus general ordered N-tuples, and unidirectional access versus omnidirectional access. Relations also have additional properties: they can exhibit predicate-argument bindings, they can be arguments to higher-order structures, and they can participate in operations of selection, projection, join, union, intersection, and difference. Relations can be used to represent structures such as lists, trees and graphs, and relational instances can be thought of as propositions. Within neural net architectures, feedforward networks can be identified with associative proceessing, and tensor product networks with relational processing. Relations have the essential properties of symbolic processing; flexibility, accessibility, and utility for repesenting complex data structures.S. PhillipsG. S. HalfordW. H. Wilson2004-03-06Z2011-03-11T08:55:29Zhttp://cogprints.org/id/eprint/3472This item is in the repository with the URL: http://cogprints.org/id/eprint/34722004-03-06ZTEMECOR: An Associative, Spatio-temporal Pattern Memory for Complex State SequencesThe problem of representing large sets of complex state sequences (CSSs)---i.e., sequences in which states can recur multiple times---has thus far resisted solution. This paper describes a novel neural network model, TEMECOR, which has very large capacity for storing CSSs. Furthermore, in contrast to the various back-propagation-based attempts at solving the CSS problem, TEMECOR requires only a single presentation of each sequence. TEMECOR's power derives from a) its use of a combinatorial, distributed representation scheme, and b) its method of choosing internal representations of states at random. Simulation results are presented which show that the
number of spatio-temporal binary feature patterns which can be stored to some criterion accuracy (e.g., 97%) increases faster-than-linearly in the size of the network. This is true for both uncorrelated and correlated pattern sets,
although the rate is slightly slower for correlated patterns.Gerard J. Rinkus2001-06-19Z2011-03-11T08:54:41Zhttp://cogprints.org/id/eprint/1598This item is in the repository with the URL: http://cogprints.org/id/eprint/15982001-06-19ZThoughts as Activation Vectors in Recurrent Nets, or Concentric Epicenters, or...Churchland underestimates the power and purpose of the Turing Test, dismissing it as the trivial game to which
the Loebner Prize (offered for the computer program that can fool judges into thinking it's human) has reduced it, whereas it
is really an exacting empirical criterion: It requires that the candidate model for the mind have our full behavioral capacities
-- so fully that it is indistinguishable from any of us, to any of us (not just for one Contest night, but for a lifetime). Scaling
up to such a model is (or ought to be) the programme of that branch of reverse bioengineering called cognitive science. It's
harmless enough to do the hermeneutics after the research has been successfully completed, but self-deluding and
question-begging to do it before. Stevan Harnad1999-06-14Z2011-03-11T08:53:52Zhttp://cogprints.org/id/eprint/383This item is in the repository with the URL: http://cogprints.org/id/eprint/3831999-06-14ZWords Lie in our WayThe central claim of computationalism is generally taken to be that the brain is a computer, and that any computer implementing the appropriate program would ipso facto have a mind. In this paper I argue for the following propositions: (1) The central claim of computationalism is not about computers, a concept too imprecise for a scientific claim of this sort, but is about physical calculi (instantiated discrete formal systems). (2) In matters of formality, interpretability, and so forth, analog computation and digital computation are not essentially different, and so arguments such as Searle's hold or not as well for one as for the other. (3) Whether or not a biological system (such as the brain) is computational is a scientific matter of fact. (4) A substantive scientific question for cognitive science is whether cognition is better modeled by discrete representations or by continuous representations. (5) Cognitive science and AI need a theoretical construct that is the continuous analog of a calculus. The discussion of these propositions will illuminate several terminology traps, in which it's all too easy to become ensnared.Bruce J. MacLennan2002-01-11Z2011-03-11T08:54:52Zhttp://cogprints.org/id/eprint/2025This item is in the repository with the URL: http://cogprints.org/id/eprint/20252002-01-11ZCell division and migration in a 'genotype' for neural networksMuch research has been dedicated recently to applying genetic algorithms to populations of
neural networks. However, while in real organisms the inherited genotype maps in complex
ways into the resulting phenotype, in most of this research the development process that
creates the individual phenotype is ignored. In this paper we present a model of neural
development which includes cell division and cell migration in addition to axonal growth and
branching. This reflects, in a very simplified way, what happens in the ontogeny of real
organisms. The development process of our artificial organisms shows successive phases of
functional differentiation and specialization. In addition, we find that mutations that affect
different phases of development have very different evolutionary consequences. A single
change in the early stages of cell division/migration can have huge effects on the phenotype
while changes in later stages have usually a less drammatic impact. Sometimes changes that
affect the first developental stages may be retained producing sudden changes in evolutionary
history.Angelo CangelosiDomenico ParisiStefano Nolfi2001-06-18Z2011-03-11T08:54:41Zhttp://cogprints.org/id/eprint/1592This item is in the repository with the URL: http://cogprints.org/id/eprint/15922001-06-18ZComputation Is Just Interpretable Symbol Manipulation: Cognition Isn'tComputation is interpretable symbol manipulation. Symbols are objects that are manipulated on the basis of rules
operating only on the symbols' shapes , which are arbitrary in relation to what they can be interpreted as meaning. Even if one
accepts the Church/Turing Thesis that computation is unique, universal and very near omnipotent, not everything is a computer,
because not everything can be given a systematic interpretation; and certainly everything can't be given every systematic
interpretation. But even after computers and computation have been successfully distinguished from other kinds of things, mental
states will not just be the implementations of the right symbol systems, because of the symbol grounding problem: The
interpretation of a symbol system is not intrinsic to the system; it is projected onto it by the interpreter. This is not true of our
thoughts. We must accordingly be more than just computers. My guess is that the meanings of our symbols are grounded in the
substrate of our robotic capacity to interact with that real world of objects, events and states of affairs that our symbols are
systematically interpretable as being about. Stevan Harnad2002-12-11Z2011-03-11T08:55:07Zhttp://cogprints.org/id/eprint/2645This item is in the repository with the URL: http://cogprints.org/id/eprint/26452002-12-11Z Lexical Disambiguation Based on Distributed Representations of Context FrequencyA model for lexical disambiguation is presented that is based on combining the frequencies of past contexts of ambiguous words. The frequencies are encoded in the word representations and define the words' semantics. A Simple Recurrent Network (SRN) parser combines the context frequencies one word at a time, always producing the most likely interpretation of the current sentence at its output. This disambiguation process is most striking when the interpretation involves semantic flipping, that is, an alternation between two opposing meanings as more words are read in. The sense of ``throwing a ball'' alternates between ``dance'' and ``baseball'' as indicators such as the agent, location, and recipient are input. The SRN parser demonstrates how the context frequencies are dynamically combined to determine the interpretation of such sentences. We hypothesize that several other aspects of ambiguity resolution are based on similar mechanisms, and can be naturally approached from the distributed connectionist viewpoint. Marshall R MayberryDr. Risto Miikkulainen1998-07-03Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/496This item is in the repository with the URL: http://cogprints.org/id/eprint/4961998-07-03ZThe locally linear nested network for robot manipulationWe present a method for accurate representation of high-dimensional unknown functions from random samples drawn from its input space. The method builds representations of the function by recursively splitting the input space in smaller subspaces, while in each of these subspaces a linear approximation is computed. The representations of the function at all levels (i.e., depths in the tree) are retained during the learning process, such that a good generalisation is available as well as more accurate representations in some subareas. Therefore, fast and accurate learning are combined in this method. The method, which is applied to hand-eye coordination of a robot arm, is shown to be superior to other neural networks.P. van der SmagtF. GroenF. van het Groenewoud1998-07-03Z2011-03-11T08:54:00Zhttp://cogprints.org/id/eprint/497This item is in the repository with the URL: http://cogprints.org/id/eprint/4971998-07-03ZMinimisation methods for training feed-forward networksMinimisation methods for training feed-forward networks with back-propagation are compared. Feed-forward neural network training is a special case of function minimisation, where no explicit model of the data is assumed. Therefore, and due to the high dimensionality of the data, linearisation of the training problem through use of orthogonal basis functions is not desirable. The focus is on function minimisation on any basis. Quasi-Newton and conjugate gradient methods are reviewed, and the latter are shown to be a special case of error back-propagation with momentum term. Three feed-forward learning problems are tested with five methods. It is shown that, due to the fixed stepsize, standard error back-propagation performs well in avoiding local minima. However, by using not only the local gradient but also the second derivative of the error function a much shorter training time is required. Conjugate gradient with Powell restarts shows to be the superior method.P. van der Smagt2001-03-31Z2011-03-11T08:54:37Zhttp://cogprints.org/id/eprint/1424This item is in the repository with the URL: http://cogprints.org/id/eprint/14242001-03-31ZA Step in the Right DirectionA review of
W. Thomas Miller, III, Richard S. Sutton, and Paul J. Werbos (Eds.) Neural Networks for Control. Cambridge, Massachusetts: The MIT Press. 1990. pp. 524.
This multi-disciplinary volume concerns the use of artificial neural networks in controlling dynamical processes. As used here 'dynamical' describes processes, such as certain chemical reaction systems, robots, or manufacturing plants, whose operation is governed by known or unknown non-linear models and which, therefore, are subject to certain types of problems related to unpredictability and chaotic performance. Artificial neural networks (ANN) are mathematical models whose components emulate the function of biological nervous systems.
Mary Ann Metzger1998-04-28Z2011-03-11T08:53:57Zhttp://cogprints.org/id/eprint/442This item is in the repository with the URL: http://cogprints.org/id/eprint/4421998-04-28ZA self-organizing neural network that learns to detect and represent visual depth from occlusion eventsVisual occlusion events constitute a major source of depth information. We have developed a neural network model that learns to detect and represent depth relations, after a period of exposure to motion sequences containing occlusion and disocclusion events. The network's learning is governed by a new set of learning and activation rules. The network develops two parallel opponent channels or ``chains'' of lateral excitatory connections for every resolvable motion trajectory. One channel, the ``On'' chain or ``visible'' chain, is activated when a moving stimulus is visible. The other channel, the ``Off'' chain or ``invisible'' chain, is activated when a formerly visible stimulus becomes invisible due to occlusion. The On chain carries a predictive modal representation of the visible stimulus. The Off chain carries a persistent, amodal representation that predicts the motion of the invisible stimulus. The new learning rule uses disinhibitory signals emitted from the On chain to trigger learning in the Off chain. The Off chain neurons learn to interact reciprocally with other neurons that indicate the presence of occluders. The interactions let the network predict the disappearance and reappearance of stimuli moving behind occluders, and they let the unexpected disappearance or appearance of stimuli excite the representation of an inferred occluder at that location. Two results that have emerged from this research suggest how visual systems may learn to represent visual depth information. First, a visual system can learn a nonmetric representation of the depth relations arising from occlusion events. Second, parallel opponent On and Off channels that represent both modal and amodal stimuli can also be learned through the same proceJ. A. MarshallR. K. Alley2001-06-18Z2011-03-11T08:54:41Zhttp://cogprints.org/id/eprint/1588This item is in the repository with the URL: http://cogprints.org/id/eprint/15882001-06-18ZSymbol Grounding is an Empirical Problem: Neural Nets are Just a Candidate Component"Symbol Grounding" is beginning to mean too many things to too many people. My own construal has always
been simple: Cognition cannot be just computation, because computation is just the systematically interpretable manipulation of
meaningless symbols, whereas the meanings of my thoughts don't depend on their interpretability or interpretation by someone
else. On pain of infinite regress, then, symbol meanings must be grounded in something other than just their interpretability if they
are to be candidates for what is going on in our heads. Neural nets may be one way to ground the names of concrete objects and
events in the capacity to categorize them (by learning the invariants in their sensorimotor projections). These grounded
elementary symbols could then be combined into symbol strings expressing propositions about more abstract categories.
Grounding does not equal meaning, however, and does not solve any philosophical problems. Stevan Harnad1998-04-28Z2011-03-11T08:53:57Zhttp://cogprints.org/id/eprint/441This item is in the repository with the URL: http://cogprints.org/id/eprint/4411998-04-28ZUnsmearing visual motion: Development of long-range horizontal intrinsic connectionsHuman vision systems integrate information nonlocally, across long spatial ranges. For example, a moving stimulus appears smeared when viewed briefly (30 ms), yet sharp when viewed for a longer exposure (100 ms) (Burr, 1980). This suggests that visual systems combine information along a trajectory that matches the motion of the stimulus. Our self-organizing neural network model shows how developmental exposure to moving stimuli can direct the formation of horizontal trajectory-specific motion integration pathways that unsmear representations of moving stimuli. These results account for Burr's data and can potentially also model other phenomena, such as visual inertia.K.E. MartinJ.A. Marshall2001-11-18Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1895This item is in the repository with the URL: http://cogprints.org/id/eprint/18952001-11-18ZTrace Feature Map: A Model of Episodic
Associative MemoryAn approach to episodic associative memory is presented, which has several desirable properties as a human memory model. The design is based on topological feature map representation of data. An ordinary feature map is a classifier, mapping an input vector onto a topologically meaningful location on the map. A trace feature map, in addition, creates a memory trace on that location. The traces can be stored episodically in a single presentation, and retrieved with a partial cue. Nearby traces overlap, which results in plausible memory interference behavior. Performance degrades gracefully as the memory is overloaded. More recent traces are easier to recall as are traces that are unique in the memory.Risto Miikkulainen1998-06-24Z2011-03-11T08:53:49Zhttp://cogprints.org/id/eprint/335This item is in the repository with the URL: http://cogprints.org/id/eprint/3351998-06-24ZThe biology of consciousness: Comparative review of Israel Rosenfield, The Strange, Familiar, and Forgotten: An anatomy of Consciousness and Gerald M. Edelman, Bright Air, Brilliant Fire: On the Matter of the MindFor many years, most AI researchers and cognitive scientists have reserved the topic of consciousness for after dinner conversation. Like "intuition," the idea of consciousness appeared to be too vague or general to be a good starting place for understanding cognition. Work on narrowly-defined problems in specialized domains such as medicine and manufacturing focused our concerns on the nature of representation, memory, strategies for problem-solving, and learning. Some writers, notably Ornstein(1972) and Hofstadter (1979), continued to explore the ideas, but implications for cognitive modeling were unclear, suggesting neither experiments, nor new computational mechanisms.W J. Clancey2001-06-18Z2011-03-11T08:54:40Zhttp://cogprints.org/id/eprint/1579This item is in the repository with the URL: http://cogprints.org/id/eprint/15792001-06-18ZCategorical Perception and the Evolution of Supervised Learning in Neural NetsSome of the features of animal and human categorical perception
(CP) for color, pitch and speech are exhibited by neural net simulations of CP with
one-dimensional inputs: When a backprop net is trained to discriminate and then
categorize a set of stimuli, the second task is accomplished by "warping" the
similarity space (compressing within-category distances and expanding
between-category distances). This natural side-effect also occurs in humans and
animals. Such CP categories, consisting of named, bounded regions of similarity
space, may be the ground level out of which higher-order categories are
constructed; nets are one possible candidate for the mechanism that learns the
sensorimotor invariants that connect arbitrary names (elementary symbols?) to the
nonarbitrary shapes of objects. This paper examines how and why such
compression/expansion effects occur in neural nets. Stevan HarnadS.J. HansonJ. Lubin2001-11-18Z2011-03-11T08:54:49Zhttp://cogprints.org/id/eprint/1896This item is in the repository with the URL: http://cogprints.org/id/eprint/18962001-11-18ZNatural Language Processing
with Modular Neural Networks and Distributed LexiconAn approach to connectionist natural language processing is proposed, which is based on hierarchically organized modular Parallel Distributed Processing (PDP) networks and a central lexicon of distributed input/output representations. The modules communicate using these representations, which are global and publicly available in the system. The representations are developed automatically by all networks while they are learning their processing tasks. The resulting representations reflect the regularities in the subtasks, which facilitates robust processing in the face of noise and damage, supports improved generalization, and provides expectations about possible contexts. The lexicon can be extended by cloning new instances of the items, that is, by generating a number of items with known processing properties and distinct identities. This technique combinatorially increases the processing power of the system. The recurrent FGREP module, together with a central lexicon, is used as a basic building block in modeling higher level natural language tasks. A single module is used to form case-role representations of sentences from word-by-word sequential natural language input. A hierarchical organization of four recurrent FGREP modules (the DISPAR system) is trained to produce fully expanded paraphrases of script-based stories, where unmentioned events and role fillers are inferred.Risto MiikkulainenMichael G. Dyer1998-04-30Z2011-03-11T08:53:57Zhttp://cogprints.org/id/eprint/443This item is in the repository with the URL: http://cogprints.org/id/eprint/4431998-04-30ZReview of Rosenfield's "The Invention of Memory"Evidence collected by Bartlett, Collingwood, James, Bransford, Jenkins, and Sacks argues against the memory-as-stored-structures hypothesis, the keystone of expert systems and cognitive modeling research.William J. Clancey1998-03-20Z2011-03-11T08:54:07Zhttp://cogprints.org/id/eprint/615This item is in the repository with the URL: http://cogprints.org/id/eprint/6151998-03-20ZThe Symbol Grounding ProblemThere has been much discussion recently about the scope and limits of purely symbolic models of the mind and about the proper role of connectionism in cognitive modeling. This paper describes the symbol grounding problem: How can the semantic interpretation of a formal symbol system be made intrinsic to the system, rather than just parasitic on the meanings in our heads? How can the meanings of the meaningless symbol tokens, manipulated solely on the basis of their (arbitrary) shapes, be grounded in anything but other meaningless symbols? The problem is analogous to trying to learn Chinese from a Chinese/Chinese dictionary alone. A candidate solution is sketched: Symbolic representations must be grounded bottom-up in nonsymbolic representations of two kinds: (1) iconic representations, which are analogs of the proximal sensory projections of distal objects and events, and (2) categorical representations, which are learned and innate feature-detectors that pick out the invariant features of object and event categories from their sensory projections. Elementary symbols are the names of these object and event categories, assigned on the basis of their (nonsymbolic) categorical representations. Higher-order (3) symbolic representations, grounded in these elementary symbols, consist of symbol strings describing category membership relations (e.g., An X is a Y that is Z). Connectionism is one natural candidate for the mechanism that learns the invariant features underlying categorical representations, thereby connecting names to the proximal projections of the distal objects they stand for. In this way connectionism can be seen as a complementary component in a hybrid nonsymbolic/symbolic model of the mind, rather than a rival to purely symbolic modeling. Such a hybrid model would not have an autonomous symbolic module, however; the symbolic functions would emerge as an intrinsically dedicated symbol system as a consequence of the bottom-up grounding of categories' names in their sensory representations. Symbol manipulation would be governed not just by the arbitrary shapes of the symbol tokens, but by the nonarbitrary shapes of the icons and category invariants in which they are grounded.Stevan Harnad2003-08-12Z2011-03-11T08:55:19Zhttp://cogprints.org/id/eprint/3106This item is in the repository with the URL: http://cogprints.org/id/eprint/31062003-08-12ZThe Symbol Grounding ProblemThere has been much discussion recently about the scope and limits of purely symbolic models of the mind and about the proper role of connectionism in cognitive modeling. This paper describes the symbol grounding problem: How can the semantic interpretation of a formal symbol system be made intrinsic to the system, rather than just parasitic on the meanings in our heads? How can the meanings of the meaningless symbol tokens, manipulated solely on the basis of their (arbitrary) shapes, be grounded in anything but other meaningless symbols? The problem is analogous to trying to learn Chinese from a Chinese/Chinese dictionary alone. A candidate solution is sketched: Symbolic representations must be grounded bottom-up in nonsymbolic representations of two kinds: (1) iconic representations, which are analogs of the proximal sensory projections of distal objects and events, and (2) categorical representations, which are learned and innate feature-detectors that pick out the invariant features of object and event categories from their sensory projections. Elementary symbols are the names of these object and event categories, assigned on the basis of their (nonsymbolic) categorical representations. Higher-order (3) symbolic representations, grounded in these elementary symbols, consist of symbol strings describing category membership relations (e.g., An X is a Y that is Z). Connectionism is one natural candidate for the mechanism that learns the invariant features underlying categorical representations, thereby connecting names to the proximal projections of the distal objects they stand for. In this way connectionism can be seen as a complementary component in a hybrid nonsymbolic/symbolic model of the mind, rather than a rival to purely symbolic modeling. Such a hybrid model would not have an autonomous symbolic module, however; the symbolic functions would emerge as an intrinsically dedicated symbol system as a consequence of the bottom-up grounding of categories' names in their sensory representations. Symbol manipulation would be governed not just by the arbitrary shapes of the symbol tokens, but by the nonarbitrary shapes of the icons and category invariants in which they are grounded.Stevan Harnad2001-06-26Z2011-03-11T08:54:44Zhttp://cogprints.org/id/eprint/1651This item is in the repository with the URL: http://cogprints.org/id/eprint/16512001-06-26ZSymbols and Nets: Cooperation vs. CompetitionCritique of two critiques of connectionism.Stevan Harnad1998-04-05Z2011-03-11T08:53:54Zhttp://cogprints.org/id/eprint/433This item is in the repository with the URL: http://cogprints.org/id/eprint/4331998-04-05ZMother Nature versus the Walking EncyclopediaIn 1982, Feldman and Ballard published "Connectionist models and their properties" in Cognitive Science, helping to focus attention on a family of similarly inspired research strategies just then under way, by giving the family a name: "connectionism." Now, seven years later, the connectionist nation has swelled to include such subfamilies as "PDP" and "neural net models." Since the ideological foes of connectionism are keen to wipe it out in one fell swoop aimed at its "essence", it is worth noting the diversity of not only the models but also the aspirations of the modelers. There is no good reason to suppose that they all pledge allegiance to any one principle or set of principles that could be shown to be false or incoherent. Those who think they are making exciting progress on their chosen projects have no particular reason to declare their own brand orthodox, and other brands ("rival" brands) heretical. Let a thousand flowers bloom and soon enough the sickly plants will succumb, without need of ideological condemnation.Daniel C Dennett1998-06-15Z2011-03-11T08:53:43Zhttp://cogprints.org/id/eprint/197This item is in the repository with the URL: http://cogprints.org/id/eprint/1971998-06-15ZRethinking the language bottleneck: Why don't animals learn to communicate?While most work on the evolution of language has been centered on the evolution of syntax, my focus in this paper is instead on more basic features that separate human communication from the systems of communication used by other animals. In particular, I argue that human language is the only existing system of learned arbitrary reference. While innate communication systems are, by definition, directly transmitted genetically, the transmission of a learned learned systems must be indirect. Learners must acquire the system by being exposed its the use in the community. Although it is reasonable that a learner has access to the utterances that are produced, it is less clear how accessible the meaning is that the utterance is intended to convey. This particularly problematic if the system of communication is symbolic -- where form and meaning are linked in a purely conventional way. Given this, I propose that the ability to transmit a learned symbolic system of communication from one generation to the next represents a key milestone in the evolution of language.Michael Oliphant1998-06-15Z2011-03-11T08:53:43Zhttp://cogprints.org/id/eprint/196This item is in the repository with the URL: http://cogprints.org/id/eprint/1961998-06-15ZThe learning barrier: Moving from innate to learned systems of communicationHuman language is a unique ability. It sits apart from other systems of communication in two striking ways: it is syntactic, and it is learned. While most approaches to the evolution of language have focused on the evolution of syntax, this paper explores the computational issues that arise in shifting from a simple innate communication system to an equally simple one that is learned. Associative network learning within an observational learning paradigm is used to explore the computational difficulties involved in establishing and maintaining a simple learned communication system. Because Hebbian learning is found to be sufficient for this task, it is proposed that the basic computational demands of learning are unlikely to account for the rarity of even simple learned communication systems. Instead, it is the problem of *observing* that is likely to be central -- in particular the problem of determining what meaning a signal is intended to convey.Michael Oliphant2001-06-18Z2011-03-11T08:54:40Zhttp://cogprints.org/id/eprint/1572This item is in the repository with the URL: http://cogprints.org/id/eprint/15722001-06-18ZCategory Induction and RepresentationA provisional model is presented in which categorical perception
(CP) provides our basic or elementary categories. In acquiring a category we
learn to label or identify positive and negative instances from a sample of
confusable alternatives. Two kinds of internal representation are built up in this
learning by "acquaintance": (1) an iconic representation that subserves our
similarity judgments and (2) an analog/digital feature-filter that picks out the
invariant information allowing us to categorize the instances correctly. This
second, categorical representation is associated with the category name.
Category names then serve as the atomic symbols for a third representational
system, the (3) symbolic representations that underlie language and that
make it possible for us to learn by "description." Connectionism is one
possible mechainsm for learning the sensory invariants underlying
categorization and naming. Among the implications of the model are (a) the
"cognitive identity of (current) indiscriminables": Categories and their
representations can only be provisional and approximate, relative to the
alternatives encountered to date, rather than "exact." There is also (b) no
such thing as an absolute "feature," only those features that are invariant
within a particular context of confusable alternatives. Contrary to prevailing
"prototype" views, however, (c) such provisionally invariant features must
underlie successful categorization, and must be "sufficient" (at least in the
"satisficing" sense) to subserve reliable performance with all-or-none,
bounded categories, as in CP. Finally, the model brings out some basic
limitations of the "symbol-manipulative" approach to modeling cognition,
showing how (d) symbol meanings must be functionally grounded in
nonsymbolic, "shape-preserving" representations -- iconic and categorical
ones. Otherwise, all symbol interpretations are ungrounded and
indeterminate. This amounts to a principled call for a psychophysical (rather
than a neural) "bottom-up" approach to cognition.Stevan Harnad2011-12-16T00:58:34Z2011-12-16T00:58:34Zhttp://cogprints.org/id/eprint/7755This item is in the repository with the URL: http://cogprints.org/id/eprint/77552011-12-16T00:58:34ZNatural Transformations of Organismic StructuresThe mathematical structures underlying the theories of organismic sets, (M, R)-systems and molecular sets are shown to be transformed naturally within the theory of categories and functors. Their natural transformations allow the comparison of distinct entities, as well as the modelling of dynamics in “organismic” structures.Prof. Dr. I. C. Baianuibaianu@illinois.edu2004-10-06Z2011-03-11T08:55:42Zhttp://cogprints.org/id/eprint/3822This item is in the repository with the URL: http://cogprints.org/id/eprint/38222004-10-06ZSTRUCTURAL ORDER AND PARTIAL DISORDER IN BIOLOGICAL SYSTEMS:
Structural "Fuzziness" underlying All Biological FunctionsThe presence of structural order and partial disorder is discussed for several important biological molecules such as DNA, enzymes and proteins, as well as for cellular structures such as nerve myelin. The relationship between structural "fuzziness" and biological function is discussed
as an important aspect of biological complexity and biodynamics. The possible effects of partial disorder on the electron density of states in biological structures are predicted based on known quantum theoretical computations for lattices in solids. Important phenomena such as Anderson delocalization, Hall effect and quantum tunneling are predicted to affect biological function. Novel experiments are being proposed by pulsed lasers, pulsed/FT-NMR and optical/NIR spectroscopy to monitor the effects of structural partial disorder and "fuzziness" on biological function. Novel methods for computer analysis of paracrystalline lattices such as nerve myelin and oriented DNA fibers are also proposed based on molecular models that include partial disorder.
Prof. Dr. I.C. Baianuicb2004-10-06Z2011-03-11T08:55:41Zhttp://cogprints.org/id/eprint/3820This item is in the repository with the URL: http://cogprints.org/id/eprint/38202004-10-06ZSTRUCTURAL ORDER AND PARTIAL DISORDER IN BIOLOGICAL SYSTEMS:
STRUCTURAL "FUZZINESS" UNDERLYING BIOLOGICAL FUNCTIONThe presence of structural order and partial disorder is discussed for several important biological molecules such as DNA, enzymes and proteins, as well as for cellular structures such as nerve myelin. The relationship between structural "fuzziness" and biological function is discussed
as an important aspect of biological complexity and biodynamics. The possible effects of partial disorder on the electron density of states in biological structures is predicted based on known quantum theoretical computations for lattices in solids. Important phenomena such as Anderson delocalization, Hall effect and quantum tunneling are predicted to affect biological function. Novel experiments are being suggested by pulsed lasers, pulsed/FT-NMR and optical/NIR spectroscopy in order to monitor the effects of structural partial disorder and "fuzziness" on biological function. Novel methods for computer analysis of paracrystalline lattices such as nerve myelin and oriented DNA fibers are also being proposed based on molecular models that include partial disorder.
Dr. I.C. Baianuicb2011-12-16T00:58:07Z2011-12-16T00:58:07Zhttp://cogprints.org/id/eprint/7753This item is in the repository with the URL: http://cogprints.org/id/eprint/77532011-12-16T00:58:07ZOn Adjoint Dynamical SystemsTransformations of dynamical systems and organismic structures are discussed in terms of adjoint, simple adjoint and weak adjoint functors of organismic supercategories during development and evolution of organisms on markedly different timescales. A representation of nuclear transplants in terms of adjoint functors and a novel interpretation of nuclear transplant experiments is proposed. Three new theorems are proven for adjoint dynamical systems representing multi-potent developing cells and additional results are obtained for weak adjoint systems such as differentiated (specialized) cells.Prof. Dr. I. C. BaianuicbProf.Dr. Dragos Scripcariu2011-12-16T00:58:54Z2011-12-16T00:58:54Zhttp://cogprints.org/id/eprint/7743This item is in the repository with the URL: http://cogprints.org/id/eprint/77432011-12-16T00:58:54ZOrganismic Supercategories: III. Qualitative Dynamics of Systems The representation of biological systems by means of organismic supercategories, developed in previous papers, is further discussed. The different approaches to relational biology, developed by Rashevsky, Rosen and by Baianu and Marinescu, are compared with Qualitative Dynamics of Systems which was initiated by Henri Poincaré (1881). On the basis of this comparison some concrete results concerning dynamics of genetic system, development, fertilization, regeneration, analogies, and oncogenesis are derived.Prof. Dr. I.C. Baianuicb2004-10-06Z2011-12-16T00:59:02Zhttp://cogprints.org/id/eprint/3831This item is in the repository with the URL: http://cogprints.org/id/eprint/38312004-10-06ZOrganismic Supercategories: I. Proposals for a General Unified Theory of Systems- Classical, Quantum, and Complex Biological Systems.
The representation of physical and complex biological systems in terms of organismic supercategories was introduced in 1968 by Baianu and Marinescu in the attached paper which was published in the Bulletin of Mathematical Biophysics, edited by Nicolas Rashevsky. The different approaches to relational biology, developed by Rashevsky, Rosen and by Baianu et al.(1968,1969,1973,1974,1987,2004)were later discussed.
The present paper is an attempt to outline an abstract unitary theory of systems. In the introduction some of the previous abstract representations of systems are discussed. Also a possible connection of abstract representations of systems with a general theory of measure is proposed. Then follow some necessary definitions and authors' proposals for an axiomatic theory of systems. Finally some concrete examples are analyzed in the light of the proposed theory.
An abstract representation of biological systems from the standpoint of the theory of supercategories is presented. The relevance of such representations forG-relational biologies is suggested. In section A the basic concepts of our representation, that is class, system, supercategory and measure are introduced. Section B is concerned with the mathematical representation starting with some axioms and principles which are natural extensions of the current abstract representations in biology. Likewise, some extensions of the principle of adequate design are introduced in section C. Two theorems which present the connection between categories and supercategories are proved. Two other theorems concerning the dynamical behavior of biological and biophysical systems are derived on the basis of the previous considerations. Section D is devoted to a general study of oscillatory behavior in enzymic systems, some general quantitative relations being derived from our representation. Finally, the relevance of these results for a quantum theoretic approach to biology is discussed.
http://www.springerlink.com/content/141l35843506596h/Prof. Dr. I.C. BaianuicbDr. Mircea M. Marinescu