Cogprints

Building large-scale hierarchical models of the world with binary sparse distributed representations

Rachkovskij, Dmitri A. and Kussul, Ernst M. (2000) Building large-scale hierarchical models of the world with binary sparse distributed representations. (Unpublished)

Full text available as:

[img] PDF
554Kb

Abstract

Many researchers agree on the basic architecture of the "world model" where knowledge about the world required for organization of agent's intelligent behavior is represented. However, most proposals on possible implementation of such a model are far from being plausible both from computational and neurobiological points of view. Implementation ideas based on distributed connectionist representations offer a huge information capacity, flexibility of similarity representation, and possibility to use a distributed neural network memory. However, for a long time distributed representations suffered from the "superposition catastrophe". Local representations are vivid, pictorial and easily interpretable, allow for an easy manual construction of hierarchical structures and an economical computer simulation of toy tasks. The problems of local representations show up with scaling to the real world models, and it is unclear how to solve them under reasonable requirements imposed on memory size and speed. We discuss the architecture of Associative-Projective Neural Networks (APNNs) that is based on binary sparse distributed representations of fixed dimensionality for items of various complexity and generality, and provides a promise for scaling up to the full-sized model of the real world. An on-the-fly binding procedure proposed for APNNs overcomes the superposition catastrophe, permitting representation of the order and grouping of structure components. These representations allow a simple estimation of structures' similarity, as well as finding various kinds of associations based on their context-dependent similarity. Structured distributed auto-associative neural network is used as long-term memory, wherein representations of items organized into part-whole (compositional) and concept (generalization) hierarchies are built. Examples of schematic APNN architectures and processes for recognition, prediction, reaction, analogical reasoning, and other tasks required for functioning of an intelligent system, as well as APNN implementations, are considered.

Item Type:Other
Keywords:analogy, analogical mapping,analogical retrieval, APNN, associative-projective neural networks, binary coding, binding, categories, chunking, compositional distributed representations, concepts, concept hierarchy, connectionist symbol processing, context-dependent thinning, distributed memory, distributed representations, Hebb, long-term memory, nested representations, neural assemblies, part-whole hierarchy, representation of structure, sparse coding, taxonomy hierarchy, thinning, working memory, world model
Subjects:Computer Science > Artificial Intelligence
Computer Science > Neural Nets
ID Code:1287
Deposited By: Rachkovskij, Dmitri
Deposited On:07 Feb 2001
Last Modified:11 Mar 2011 08:54

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

Amari, S. (1989). Characteristics of sparsely encoded associative memory. Neural Networks, 2, 445-457.

Amit, D.J. (1989) Modeling brain function: The world of attractor neural networks Cambridge University Press

Amit, D.J. (1995) THE HEBBIAN PARADIGM REINTEGRATED: LOCAL REVERBERATIONS AS INTERNAL REPRESENTATIONS - Behavioral and Brain Science Dec. 1995 - Daniel J. Amit

Amosov, N. M. (1967). Modelling of thinking and the mind. New York: Spartan Books.

Amosov, N. M. (1979). Algorithms of the Mind. Kiev: Naukova Dumka (in Russian)

Amosov, N. M., Baidyk, T. N., Goltsev, A. D., Kasatkin, A. M., Kasatkina, L. M., Kussul, E. M., & Rachkovskij, D. A. (1991). Neurocomputers and intelligent robots. Kiev: Naukova dumka. (In Russian).

Amosov, N.M., Kasatkin, A.M., & Kasatkina L.M.(1975) Active semantic networks in robots with an autonomous control. Advance papers of the Fourth Intern. Joint Conference on Artificial intelligence v.9 pp. 11-20 (in Russian. English version exists but not available to me).

Amosov, N.M., Kasatkin, A.M., Kasatkina, L.M., Talayev, S.A. (1973). Automata and the mindfull behaviour. Kiev: Naukova Dumka (in Russian).

Amosov, N.M., & Kussul, E.M. Possible structure of system for reinforcement and inhibition. In: "Problems of heuristic modelling", Inst. of Cybernetics, Ukrainian Acad. Sci., 1969, no.1, pp. 3-11 (in Russian).

Amosov, N.M., Kussul, E.M., & Fomenko, V.D. (1975) Transport robot with a neural network control system. Advance papers of the Fourth Intern. Joint Conference on Artificial intelligence v.9 p.1-10 (In Russian. English version exists but not available to me).

Anderson J.A. (1972) A simple neural network generating an interactive memory. Mathematical Biosciences, 1970, 197-220

Anderson J.A. (1983). Cognitive and Psychological Computation with Neural Models. IEEE transactions on Systems, Man, and Cybernetics, SMC-13, No 5, 799-814.

Anderson, J.A. & Hinton, G.E., eds. (1981) Parallel models of associative memory. Hillside, NJ: Lawrence Erlbaum Associates.

Anderson J.A., Murphy G.L. (1986). Psychological Concepts in a Parallel System. Physica 22D, 318-336.

Anderson, J.A., Silverstein, J.W., Ritz, S.A., & Jones, R.S. Distinctive features, categorical perception, and probability learning: Some application of a neural model. Psychological Review, 1977, 84, 413-451.

Anderson, J.R., & Bower, G.H. (1973). Human associative memory. Washington, DC: V.H.Winston.

Antomonov, Yu.G. (1969). Systems, Complexity, Dynamics. Kiev: Naukova Dumka (in Russian)

Antomonov, Yu.G. (1974). Principles of Neurodynamics. Kiev: Naukova Dumka (in Russian)

Artykutsa, S. Ya., Baidyk, T. N., Kussul, E. M., & Rachkovskij, D. A. (1991). Texture recognition using neurocomputer. (Preprint 91- 8). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Ashby, W.R. (1956) An Introduction to Cybernetics. London: Chapman & Hall.

Baidyk T.N., Kussul E.M. (1992) Structure of neural assembly. In Proceedings of The RNNS/IEEE Symposium on Neuroinformatics and Neurocomputers. Rostov-on-Don, Russia. pp. 423-434

Baidyk, T. N., Kussul, E. M., & Rachkovskij, D. A. (1990). Numerical-analytical method for neural network investigation. In Proceedings of The International Symposium on Neural Networks and Neural Computing - NEURONET'90 (pp. 217-219). Prague, Czechoslovakia.

Barsalou, L.W. (1999). Perceptual symbol systems. Behavioral and Brain Sciences, 22, 577-609.

Baum, E.B., J. Moody, and F. Wilczek (1988). Internal representations for associative memory. Biological Cybernetics 59, 217-228.

Booch, G (1994) Object-oriented analysis and design. Benjamin/Cummings. Menlo Park, Calif.

Braitenberg V. Cell assemblies in the cerebral cortex. In: Theoretical approaches to complex systems, 171-188, Heim R. and Palm G. (eds.), Springer Verlag, New York, 1978. Braitenberg V. Cell assemblies in the cerebral cortex. Lect. Notes Biomath. - 1978. - 21. - P.171-188.

Caid, WR, Dumais ST, & Gallant SI. Learned vector-space models for document retrieval. Information Processing and Management, Vol. 31, No. 3, pp. 419-429, 1995.

Cangelosi, A. & Harnad, S. (2000) The Adaptive Advantage of Symbolic Theft Over Sensorimotor Toil:Grounding Language in Perceptual Categories. Evolution of Communication (Special Issue on Grounding)

Carpenter, G.A., Grossberg, S. (1987) A massively parallel architecture for a selforganizing neural pattern recognition machine Computer Vision, Graphics and Image processing 37, pp. 54-115.

Christiansen, M.H., Chater, N. & Seidenberg, M.S. (Eds.) (1999). Connectionist models of human language processing: Progress and prospects. Special issue of Cognitive Science, Vol. 23(4), 415-634.

Cowan, N. (2001) The Magical Number 4 in Short-term Memory: A Reconsideration of Mental Storage Capacity. Behavioral and Brain Sciences 24 (1): XXX-XXX.

Deerwester, S., S. T. Dumais, G. W. Furnas, T. K. Landauer, & R. A. Harshman. Indexing by Latent Semantic Analysis. Jornal of the American Society for Information Science, 41(6): 391-407, 1990.

Dorffner, G & E. Prem. Connectionism, Symbol Grounding, and Autonomous Agents. Technical Report TR-93-17, Austrian Research Institute for AI.

Edelman, S. (1998). Representation is representation of similarities. Behavioral and Brain Sciences, 21, 449-498.

Eliasmith, C.& Thagard, P. (in press) Integrating Structure and Meaning: A Distributed Model of Analogical Mapping. Cognitive Science.

Falkenhainer, B., Forbus, K.D., & Gentner, D. (1989). The Structure-Mapping Engine: Algorithm and Examples. Artificial Intelligence, 41, 1-63.

Fedoseyeva, T. V. (1992). The problem of training neural network to recognize word roots. In Neuron-like networks and neurocomputers (pp. 48-54). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Feigelman M.V., Ioffe, L.B. (1987) The augmented models of associative memory - asymmetric interaction and hierarchy of patterns. International Journal of Modern Physics, B1, 51-68.

Feldman, J. A. (1989). Neural Representation of Conceptual Knowledge. In L. Nadel, L. A. Cooper, P. Culicover, & R. M. Harnish (Eds.), Neural connections, mental computation (pp. 68- 103). Cambridge, Massachusetts, London, England: A Bradford Book, The MIT Press.

Forbus, K.D., Gentner, D., & Law, K. (1995). MAC/FAC: A model of similarity-based retrieval. Cognitive Science, 19(2), 141- 205.

Frasconi, P., Gori, M., & Sperduti, A. (1997). A general framework for adaptive processing of data structures. Technical Report DSI- RT-15/97. Firenze, Italy: Universita degli Studi di Firenze, Dipartimento di Sistemi e Informatica.

Frolov, A. A., & Muraviev, I. P. (1987). Neural models of associative memory. Moscow: Nauka. (In Russian).

Frolov, A. A., & Muraviev, I. P. (1988). Informational characteristics of neuronal and synaptic plasticity. Biophysics, 33, 708-715.

Frolov A.A., Husek D., Muraviev I.P. (1997) Informational capacity and recall quality in sparsely encoded Hopfield-like associative memory. Neural Networks. 9, pp. 1012-1025.

Gayler, R. W. (1998). Multiplicative binding, representation operators, and analogy. In K. Holyoak, D. Gentner, & B. Kokinov (Eds.), Advances in analogy research: Integration of theory and data from the cognitive, computational, and neural sciences. (p. 405). Sofia, Bulgaria: New Bulgarian University. (Poster abstract. Full poster available at: http://cogprints.soton.ac.uk/abs/comp/199807020).

Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7, pp 155-170.

Gentner, D. & Markman, A. B. (1995). Analogy-Based Reasoning. In M. A. Arbib (Ed.), Handbook of brain theory and neural networks (pp. 91-93). Cambridge, MA: MIT Press.

Gentner, D., & Markman, A. B. (1997). Structure Mapping in Analogy and Similarity. American Psychologist, 52(1), 45-56.

Gladun, V.P. (1977) Heuristic search in complex environments Kiev: Naukova Dumka. (in Russian)

Gladun V.P. (1994) Processes of new knowledge formation. Sofia: SD "Pedagog 6". (in Russian)

Goltsev A.D. Investigation of mechanisms for assembly organization in neural networks. Ph.D. Thesis, Kiev, 1976 ( in Russian).

Goltsev, A. (1996). An assembly neural network for texture segmentation. Neural Networks, 9(4), 643-653.

Gray, B., Halford, G. S., Wilson, W. H., & Phillips, S. (1997, September 26-28). A Neural Net Model for Mapping Hierarchically Structured Analogs. Proceedings of the Fourth conference of the Australasian Cognitive Science Society, University of Newcastle.

Grossberg, S.Pavlovian pattern learning by nonlinear neural networks. Proceedings of the National Academy of Sciences, 1971, 68, 828-831.

Grossberg, S. (1982) Studies of Mind and Brain: Neural principles of learning, perception, development, cognition and motor control. Boston: Reidel.

Gutfreund H. (1988) Neural network with hierarchically correlated patterns. Physical Review A37, pp.570- 577.

Hadley, R.F., Rotaru-Varga, A., Arnold, D.V., and Cardei, V.C. (2000) ``Syntactic Systematicity Arising from Semantic Predictions in a Hebbian-Competitive Network'', Submitted for Journal Review.

Harnad, S. (1987). Category induction and representation. In S. Harnad (Ed.), Categorical perception: The groundwork of cognition (pp. 535-565). New York: Cambridge University Press.

Harnad, S. (1990). The Symbol Grounding Problem. Physica D 42:335-346.

Hebb, D. O. (1949). The organization of behavior. New York: Wiley.

Hinton, G. E. (1990). Mapping part-whole hierarchies into connectionist networks. Artificial Intelligence, 46, 47-76.

Hinton, G. E., McClelland, J. L., & Rumelhart, D. E. (1986). Distributed representations. In D. E. Rumelhart, J. L. McClelland, & the PDP research group (Eds.), Parallel distributed processing: Exploration in the microstructure of cognition 1: Foundations (pp. 77-109). Cambridge, MA: MIT Press.

Hirahara, M., O. Oka, T. Kindo (2000) Cascade associative memory storing hierarchically correlated patterns with various correlations Nneurl Networks 13(1), pp.51-61.

Holyoak, K. J., & Thagard, P. (1989). Analogical mapping by constraint satisfaction. Cognitive Science, 13, 295-355.

Hopfield, J. J. (1982). Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences, USA, 79, 2554-2558.

Hopfield, J. J., Feinstein, D. I., & Palmer, R. G. (1983). "Unlearning" has a stabilizing effect in collective memories. Nature, 304, 158- 159.

Hummel, J. E. (2000). Where view-based theories break down: The role of structure in shape perception and object recognition. In E. Dietrich and A. Markman (Eds.). Cognitive Dynamics: Conceptual Change in Humans and Machines (pp. 157 - 185). Hillsdale, NJ: Erlbaum.

Hummel, J.E., & Beiderman, I. (1992) Dynamic binding in a neural network for shape recognition. Psychological Review, 99, 480-517.

Hummel, J. E., & Holyoak K. J. (1997). Distributed representations of structure: A theory of analogical access and mapping. Psychological Review, 104, 427-466.

Intrator, N & Edelman, S. (2000) REPRESENTING THE STRUCTURE OF VISUAL OBJECTS. Titles, abstracts, and background material provided by the speakers of NIPS'2000 workshop. http://kybele.psych.cornell.edu/~edelman/NIPS00/material.html

Kakeya, H., & Y.Okabe (1999) Selective retrieval of Memory and concept sequences through neuro-windows. IEEE Transactions on Neural Networks, v.10, No.1, pp. 182-185.

Kanerva, P. (1988). Sparse distributed memory. Cambridge, MA: MIT Press.

Kanerva, P. (1994). The Spatter Code for encoding concepts at many levels. In M. Marinaro and P.G. Morasso (eds.), ICANN '94, Proceedings of International Conference on Artificial Neural Networks (Sorrento, Italy), 1, pp. 226-229. London: Springer- Verlag.

Kanerva, P. (1996). Binary Spatter-Coding of Ordered K-tuples. In C. von der Malsburg, W. von Seelen, J. C. Vorbruggen, & B. Sendhoff (Eds.). Proceedings of the International Conference on Artificial Neural Networks - ICANN'96, Bochum, Germany. Lecture Notes in Computer Science, 1112, 869-873. Berlin: Springer.

Kanerva, P. (1998). Encoding structure in Boolean space. In L. Niklasson, M. Boden, and T. Ziemke (eds.), ICANN 98: Perspectives in Neural Computing (Proceedings of the 8th International Conference on Artificial Neural Networks, Skoevde, Sweden), 1, pp. 387-392. London: Springer.

Kanerva, P., Kristoferson, J., and Holst, A. (2000) "Random indexing of text samples for Latent Semantic Analysis." In L.R. Gleitman and A.K. Josh (eds.), Proc. 22nd Annual Conference of the Cognitive Science Society (U Pennsylvania), p. 1036. Mahwah, New Jersey: Erlbaum.

Kasatkin, A. M., & Kasatkina, L. M. (1991). A neural network expert system. In Neuron-like networks and neurocomputers (pp. 18-24). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Keane, M.T. (1997) What makes an analogy difficult? The effects of order and causal structure on analogical mapping. Journal of Experimental Psychology: Learning, Memory, and Cognition, 23, no 4, 946-967.

Kohonen T (1972) Correlation matrix memories. IEEE Transactions on Computers, C-21, 353-359.

Kussul, E. M. (1980). Tools and techniques for development of neuron-like networks for robot control. Unpublished Dr. Sci. dissertation. Kiev, Ukrainian SSR: V. M. Glushkov Institute of Cybernetics. (In Russian).

Kussul, E. M. (1988). Elements of stochastic neuron-like network theory. In Internal Report "Kareta-UN" (pp. 10-95). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Kussul, E. M. (1992) Associative neuron-like structures. Kiev: Naukova Dumka. (In Russian).

Kussul, E. M. (1993). On some results and prospects of development of associative-projective neurocomputers. In Neuron-like networks and neurocomputers (pp. 4-11). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Kussul, E. M., & Baidyk, T. N. (1990). Design of a neural-like network architecture for recognition of object shapes in images. Soviet Journal of Automation and Information Sciences, 23(5), 53- 58.

Kussul, E. M., & Baidyk, T. N. (1993a). On information encoding in associative-projective neural networks. (Preprint 93-3). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Kussul, E. M., & Baidyk, T. N. (1993b). A modular structure of associative-projective neural networks. (Preprint 93-6). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Kussul, E. M., Baidyk, T. N., Lukovich, V. V., & Rachkovskij, D. A. (1993). Adaptive neural network classifier with multifloat input coding. In Proceedings of NeuroNimes'93, Nimes, France, Oct. 25- 29, 1993. EC2-publishing.

Kussul, E.M., T.N. Baidyk, VV. Lukovich, D.A. Rachkovskij, Adaptive High Performance Classifier Based on Random Threshold Neurons, Proc. of Twelfth European Meeting on Cybernetics and Systems Research (EMCSR- 94), Austria, Vienna, 1994, pp. 1687-1695.

Kussul E.M.,& Fedoseyeva T.V. On audio signls recognition in neural assembly structures. Kiev, 1987, 21 pp, Prepr.87-28, Inst. Of Cybernetics, Ukrainian Acad. Sci. (in Russian).

Kussul, E. M., & Kasatkina, L. M. (1999). Neural network system for continuous handwritten words recognition. In Proceedings of the International Joint Conference on Neural Networks. (Washington, DC).

Kussul, E.M., Lukovich, V.V, Lutsenko, V.N. (1988) Multiprocessor computational devices for robot control in natural environment. Control systems and machines, no 5, pp.102-105 (in Russian).

Kussul, E. M., & Rachkovskij, D. A. (1991). Multilevel assembly neural architecture and processing of sequences. In A .V. Holden & V. I. Kryukov (Eds.), Neurocomputers and Attention: Vol. II. Connectionism and neurocomputers (pp. 577- 590). Manchester and New York: Manchester University Press.

Kussul, E. M., Rachkovskij, D. A., & Baidyk, T. N. (1991a). Associative-Projective Neural Networks: architecture, implementation, applications. In Proceedings of the Fourth International Conference "Neural Networks & their Applications", Nimes, France, Nov. 4-8, 1991 (pp. 463-476).

Kussul, E. M., Rachkovskij, D. A., & Baidyk, T. N. (1991b). On image texture recognition by associative-projective neurocomputer. In C. H. Dagli, S. Kumara, & Y. C. Shin (Eds.), Proceedings of the ANNIE'91 conference "Intelligent engineering systems through artificial neural networks" (pp. 453-458). ASME Press.

Kussul, E.M., D.A. Rachkovskij, and D. Wunsch (1999) The Random Subspace coarse coding scheme for real-valued vectors In: Proceedings of the International Joint Conference on Neural Networks, Washington, DC, July 10-16, 1999.

Lansner, A., & Ekeberg, O. (1985). Reliability and speed of recall in an associative network. IEEE Trans. Pattern Analysis and Machine Intelligence, 7, 490-498.

Lavrenyuk, A. N. (1995). Application of neural networks for recognition of handwriting in drawings. In Neurocomputing: issues of theory and practice (pp. 24-31). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Legendy, C. R. (1970). The brain and its information trapping device. In Rose J. (Ed.), Progress in cybernetics, vol. I. New York: Gordon and Breach.

Marr, D. (1969). A theory of cerebellar cortex. Journal of Physiology, 202, 437-470.

Marr, D. (1982). Vision. Freeman: New York.

Markman, A.B. (1997). Structural alignment in similarity and its influence on category structure. Cognitive Studies, 4(4), 19-37.

Marshall, J.A. (1990) A self-organizing scale-sensitive neural network. Proceedings of the International Joint Conference on Neural Networks, San Diego, CA, vol. 3, 649-654.

McClelland, J.L., Rumelhart, D.E. (1986) A distributed model of human learning and memory. pp. 171-215. in McClelland, J.L., Rumelhart, D.E., & the PDP Research Group (1986). Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 2. Psychological and biological models. Cambridge, MA: MIT Press.

Milner P.M. The cell assembly: Mark II. Psychol. Rev., 64, 242-252 (1957).

Milner, P. M. (1974). A model for visual shape recognition. Psychological Review, 81, 521-535.

Milner, P.M. (1996). Neural representations: some old problems revisited. Journal of Cognitive Neuroscience, 8, 69-77.

Minsky, M. (1975). A framework for representing knowledge. In P.H. Winston (Ed.), The psychology of computer vision (pp. 211-277)!. New York: McGraw-Hill.

Nadal, J.-P, & G Toulouse (1990) Information storage in sparsely coded memory nets Network: Comput. Neural Syst. 1 No 1 61-74

Nakano K. (1972) Associatron - A model of associative memory. IEEE transactions on Systems, Man, and Cybernetics. Vol.SMC2(3), pp.380-388.

Nirenburg, S. and V. Raskin.(2001) Ontological Semantics.

Page, M. Connectionist Modelling in Psychology: A Localist Manifesto Behavioral and Brain Sciences

Palm, G. (1980). On associative memory. Biological Cybernetics, 36, 19-31.

Palm, G. (1982) Neural Assemblies: An Alternative Approach to Artificial Intelligence. Springer Verlag, Berlin, 1982.

Palm, G. (1993). The PAN system and the WINA project. In P. Spies (Ed.), Euro-Arch'93 (pp. 142-156). Springer-Verlag

Parga, N. & Virasoro, M.A. (1986). The ultrametric organization of memories in a neural network. J. Physique, 47, 1857-1864.

Plate, T. A. (1991). Holographic Reduced Representations: Convolution algebra for compositional distributed representations. In J. Mylopoulos & R. Reiter (Eds.), Proceedings of the 12th International Joint Conference on Artificial Intelligence (pp. 30- 35). San Mateo, CA: Morgan Kaufmann.

Plate, T. A. (1995). Holographic Reduced Representations. IEEE Transactions on Neural Networks, 6, 623-641.

Plate, T. (1997). A common framework for distributed representation schemes for compositional structure. In F. Maire, R. Hayward, & J. Diederich (Eds.), Connectionist systems for knowledge representation and deduction (pp. 15-34). Queensland University of Technology.

Plate, T. (1999). Representation, Reasoning and Learning with Distributed Representations. Neural Computing Surveys 2, pp. 15-17 http://www.icsi.berkeley.edu/~jagota/NCS,

Plate, T.A. (2000). Structured Operations with Vector Representations. Expert Systems: The International Journal of Knowledge Engineering and Neural Networks, 17, 29-40.

Pollack, J. B. (1990). Recursive distributed representations. Artificial Intelligence, 46, 77-105.

Pulvermuller, F. (1999) Words in the brain's language Behavioral and Brain Sciences 22, 253-336.

Quillian, M.R. (1968) Semantic memory. In M.Minsky (Ed.), Semantic information processing, Cambridge, Mass.: MIT Press

Rachkovskij, D. A. (1990a). On numerical-analytical investigation of neural network characteristics. In Neuron-like networks and neurocomputers (pp. 13-23). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Rachkovskij, D. A. (1990b). Development and investigation of multilevel assembly neural networks. Unpublished Ph.D. dissertation. Kiev, Ukrainian SSR: V. M. Glushkov Institute of Cybernetics. (In Russian).

Rachkovskij, D. A. (1996). Application of stochastic assembly neural networks in the problem of interesting text selection. In Neural network systems for information processing (pp. 52-64). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Rachkovskij, D.A. (2001). Representation and processing of structures with binary sparse distributed codes. IEEE transactions on Knowledge and Data Engineering (Special Issue).

Rachkovskij, D. A. & Fedoseyeva, T. V. (1990). On audio signals recognition by multilevel neural network. In Proceedings of The International Symposium on Neural Networks and Neural Computing - NEURONET'90 (pp. 281-283). Prague, Czechoslovakia.

Rachkovskij, D. A. & Fedoseyeva T. V. (1991). Hardware and software neurocomputer system for recognition of acoustical signals. In Neuron-like networks and neurocomputers (pp. 62-68). Kiev, Ukraine: V. M. Glushkov Institute of Cybernetics. (In Russian).

Rachkovskij, D. A. & Kussul, E. M. (2001). Binding and Normalization of Binary Sparse Distributed Representations by Context-Dependent Thinning (paper draft available at http://cogprints.soton.ac.uk/abs/comp/199904008 ).

Rumelhart, D.E., Lindsay, P.H., & Norman, D.A. A process model for long-term memory. In E.Tulving & W. Donaldson (Eds.), Organization and memory. New York: Academic Press, 1972.

Rumelhart, D.E., Smolemsky, P., McClelland, J.L., & Hinton, G.E. (1986) Schemata and Sequential Thought Process in PDP Models. pp. McClelland, J.L., Rumelhart, D.E., & the PDP Research Group (1986). Parallel distributed processing: Explorations in the microstructure of cognition: Vol. 2. Psychological and biological models. Cambridge, MA: MIT Press.

Schank, R.C., & Abelson, R.P. (1977). Scripts, plans, goals, and understanding: An inquiry into human knowledge structures. Hillsdale, NJ: Lawrence Erlbaum Associates.

Seidenberg, M.S., & MacDonald, M.C. (1999). A probabilistic constraints approach to language acquisition and processing.Cognitive Science. 23, 569-588.

Shastri, L. & Ajjanagadde, V. (1993). From simple associations to systematic reasoning: connectionist representation of rules, variables, and dynamic bindings using temporal synchrony. Behavioral and Brain Sciences, 16, 417-494.

Sjodin, G. (1998). The Sparchunk Code: a method to build higher- level structures in a sparsely encoded SDM. In Proceedings of IJCNN'98 (pp.1410-1415), IEEE, Piscataway, NJ: IEEE.

Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence, 46, 159-216.

Sommer, F.T., & G. Palm (1999) Improved Bidirectional Retrieval of Sparse Patterns Stored by Hebbian Learning Neural Networks 12 (2) 281 - 297

Sperduti, A. (1994). Labeling RAAM. Connection Science, 6, 429- 459.

Steedman, M (1999) Connectionist Sentence Processing in Perspective, Cognitive Science, 23, 615-634.

Thagard, P., K.J.Holyoak, G.Nelson, D.Gochfeld Analog Retrieval by Constraint Satisfaction Artificial Intelligence 46(1990) 259-310

Thorpe, S. (1995) Localized versus distributed representations. In M.A.Arbib (Ed.), The Handbook of Brain Theory and Neural Networks, pp. 549-552. Cambridge, MA: MIT Press.

Tsodyks, M. V. (1989). Associative memory in neural networks with the Hebbian learning rule. Modern Physics Letters B, 3, 555-560.

Tsodyks M. V. (1990). Hierarchical associative memory in neural networks with low activity level. Modern Physics Letters B, 4, 259-265.

Vedenov, A. A., Ezhov A.A., L.A. Knizhnikova, E.B. Levchenko (1987). "Spurious memory" in model neural networks. (Preprint IAE-4395/1). Moscow: I. V. Kurchatov Institute of Atomic Energy. (in Russian)

Vedenov, A. A. (1988). Modeling of thinking elements. Moscow: Nauka. (In Russian).

von der Malsburg, C. (1981). The correlation theory of brain function. (Internal Report 81-2). Gottingen, Germany: Max-Planck- Institute for Biophysical Chemistry, Department of Neurobiology. von der Malsburg, C. (1985). Nervous structures with dynamical links. Ber. Bunsenges. Phys. Chem., 89, 703-710.

von der Malsburg, C. (1986) Am I thinking assemblies? In G. Palm & A. Aertsen (Eds.), Proceedings of the 1984 Trieste Meeting on Brain Theory (pp. 161-176). Heidelberg: Springer-Verlag.

Willshaw, D. (1981). Holography, associative memory, and inductive generalization. In G. E. Hinton & J. A. Anderson (Eds.), Parallel models of associative memory (pp. 83-104). Hillside, NJ: Lawrence Erlbaum Associates.

Willshaw, D. J., Buneman, O. P., & Longuet-Higgins, H. C. (1969). Non-holographic associative memory. Nature, 222, 960-962.

Metadata

Repository Staff Only: item control page

Cogprints is powered by EPrints 3 which is developed by the School of Electronics and Computer Science at the University of Southampton. More information and software credits.