Cogprints
Login | Create Account

Towards comprehensive foundations of computational intelligence

Duch, Prof Wlodzislaw (2007) Towards comprehensive foundations of computational intelligence. [Book Chapter]

Full text available as:

[img]
Preview
PDF - Requires a PDF viewer such as GSview, Xpdf or Adobe Acrobat Reader
437Kb

Abstract

Although computational intelligence (CI) covers a vast variety of different methods it still lacks an integrative theory. Several proposals for CI foundations are discussed: computing and cognition as compression, meta-learning as search in the space of data models, (dis)similarity based methods providing a framework for such meta-learning, and a more general approach based on chains of transformations. Many useful transformations that extract information from features are discussed. Heterogeneous adaptive systems are presented as particular example of transformation-based systems, and the goal of learning is redefined to facilitate creation of simpler data models. The need to understand data structures leads to techniques for logical and prototype-based rule extraction, and to generation of multiple alternative models, while the need to increase predictive power of adaptive models leads to committees of competent models. Learning from partial observations is a natural extension towards reasoning based on perceptions, and an approach to intuitive solving of such problems is presented. Throughout the paper neurocognitive inspirations are frequently used and are especially important in modeling of the higher cognitive functions. Promising directions such as liquid and laminar computing are identified and many open problems presented.

Item Type:Book Chapter
Keywords:Computational intelligence, neurocognitive informatics, machine learning, neural networks, artificial intelligence, neurocognitive linguistics
Subjects:Computer Science > Language
Computer Science > Machine Learning
Computer Science > Artificial Intelligence
Computer Science > Neural Nets
ID Code:5891
Deposited By:Duch, Prof Wlodzislaw
Deposited On:08 Jan 2008 00:28
Last Modified:08 Jan 2008 00:28

References in Article

Select the SEEK icon to attempt to find the referenced article. If it does not appear to be in cogprints you will be forwarded to the paracite service. Poorly formated references will probably not work.

1. F. Corbacho A. Sierra, J.A. Macias. Evolution of functional link networks. IEEE Transactions

on Evolutionary Computation, 5:54–65, 2001.

2. N.I. Achieser. Theory of Approximation. Frederick Ungar, New York, 1956. Reprinted:

Dover Publications, New York 1992.

3. R. Adamczak,W. Duch, and N. Jankowski. New developments in the feature space mapping

model. In Third Conference on Neural Networks and Their Applications, pages 65–70,

Kule, Poland, Oct 1997.

4. J. A. Anderson, A. Pellionisz, and E. Rosenfeld. Neurocomputing 2. MIT Press, Cambridge,

MA, 1990.

5. J. A. Anderson and E. Rosenfeld. Neurocomputing - foundations of research. MIT Press,

Cambridge, MA, 1988.

6. R. Avnimelech and N. Intrator. Boosted mixture of experts: An ensemble learning scheme.

Neural Computation, 11:483–497, 1999.

7. F.R. Bach and M.I. Jordan. Kernel independent component analysis. Journal of Machine

Learning Research, 3:1–48, 2002.

8. P.M. Baggenstoss. The pdf projection theorem and the class-specific method. IEEE Transactions

on Signal Processing, 51:672–668, 2003.

9. B. Bakker and T. Heskes. Task clustering and gating for bayesian multitask learning. Journal

of Machine Learning Research, 4:83–99, 2003.

10. E. Bauer and R. Kohavi. An empirical comparison of voting classification algorithms:

bagging, boosting and variants. Machine learning, 36:105–142, 1999.

11. Y. Bengio, O. Delalleau, and N. Le Roux. The curse of highly variable functions for local

kernel machines. Advances in Neural Information Processing Systems, 18:107–114, 2006.

12. Y. Bengio, M. Monperrus, and H. Larochelle. Non-local estimation of manifold structure.

Neural Computation, 18:2509–2528, 2006.

13. T. Bilgic¸ and I.B. T¨urks¸en. Measurements of membership functions: Theoretical and empirical

work. In D. Dubois and H. Prade, editors, Fundamentals of Fuzzy Sets, Vol. 1, pages

195–232. Kluver, Boston, 2000.

14. C. M. Bishop. Neural Networks for Pattern Recognition. Oxford University Press, 1995.

15. M. Blachnik, W. Duch, and T. Wieczorek. Selection of prototypes rules context searching

via clustering. Lecture Notes in Artificial Intelligence, 4029:573–582, 2006.

16. E. Bonabeau, M. Dorigo, and G. Theraulaz. Swarm Intelligence: From Natural to Artificial

Systems. Oxford University Press, 1999.

17. L. Breiman. Bias-variance, regularization, instability and stabilization. In C. M. Bishop,

editor, Neural Networks and Machine Learning, pages 27–56. Springer-Verlag, 1998.

18. Y. Burnod. An Adaptive Neural Network. The Cerebral Cortex. Prentice-Hall, London,

1990.

19. G.A. Carpenter and S. Grossberg. Adaptive resonance theory. In M.A. Arbib, editor, The

Handbook of Brain Theory and Neural Networks, 2nd ed, pages 87–90. MIT Press, Cambridge,

MA, 2003.

20. G. Chaitin. Algorithmic Information Theory. Cambridge University Press, 1987.

21. O. Chapelle. Training a support vector machine in the primal. Neural Computation, in

print, 2006.

22. N. Chater. The search for simplicity: A fundamental cognitive principle? Quarterly Journal

of Experimental Psychology, 52A:273–302, 1999.

23. A. Cichocki and S. Amari. Adaptive Blind Signal and Image Processing. Learning Algorithms

and Applications. J. Wiley & Sons, New York, 2002.

24. T. M. Cover. Geometrical and statistical properties of systems of linear inequalities with

applications in pattern recognition. IEEE Transactions on Electronic Computers, 14:326–

334, 1965.

25. G.S. Cree, K. McRae, and C. McNorgan. An attractor model of lexical conceptual processing:

Simulating semantic priming. Cognitive Science, 23(3):371–414, 1999.

26. F. Crestani. Application of spreading activation techniques in information retrieval. Artifical

Intelligence Review, 11(6):453–482, 1997.

27. N. Cristianini and J. Shawe-Taylor. An Introduction to Support Vector Machines and other

Kernel-Based Learning Methods. Cambridge University Press, 2000.

28. P. Dayan and G.E. Hinton. Varieties of helmholtz machines. Neural Networks, 9:1385–

1403, 1996.

29. A.M. de Callata¨y. Natural and Artificial Intelligence. Misconceptions about Brains and

Neural Networks. Elsevier, Amsterdam, 1992.

30. L.N. de Castro and J.I. Timmis. Artificial Immune Systems: A New Computational Intelligence

Approach. Springer, 2002.

31. S. Deneve and A. Pouget. Basis functions for object-centered representations. Neuron,

37:347–359, 2003.

32. T.G. Dietterich and G. Bakiri. Solving multiclass learning problems via error-correcting

output codes. Journal Of Artificial Intelligence Research, 2:263–286, 1995.

33. W. Duch. Neural minimal distance methods. In Proceedings 3-rd Conference on Neural

Networks and Their Applications, pages 183–188, Kule, Poland, Oct 1997.

34. W. Duch. Platonic model of mind as an approximation to neurodynamics. In S i. Amari

and N. Kasabov, editors, Brain-like computing and intelligent information systems, pages

491–512. Springer, 1997.

35. W. Duch. Similarity based methods: a general framework for classification, approximation

and association. Control and Cybernetics, 29:937–968, 2000.

36. W. Duch. Coloring black boxes: visualization of neural network decisions. In Int. Joint

Conf. on Neural Networks, Portland, Oregon, volume I, pages 1735–1740. IEEE Press,

2003.

37. W. Duch. Uncertainty of data, fuzzy membership functions, and multi-layer perceptrons.

IEEE Transactions on Neural Networks, 16:10–23, 2005.

38. W. Duch. Computational creativity. In World Congres on Computational Intelligence,

Vancouver, Canada, pages 1162–1169. IEEE Press, 2006.

39. W. Duch. Filter methods. In I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh, editors,

Feature extraction, foundations and applications, pages 89–118. Physica Verlag, Springer,

Berlin, Heidelberg, New York, 2006.

40. W. Duch. k-separability. Lecture Notes in Computer Science, 4131:188–197, 2006.

41. W. Duch, R. Adamczak, and G. H. F. Diercksen. Distance-based multilayer perceptrons.

In M. Mohammadian, editor, International Conference on Computational Intelligence for

Modelling Control and Automation, pages 75–80, Amsterdam, The Netherlands, 1999. IOS

Press.

42. W. Duch, R. Adamczak, and G. H. F. Diercksen. Neural networks in non-euclidean spaces.

Neural Processing Letters, 10:201–210, 1999.

43. W. Duch, R. Adamczak, and G.H.F. Diercksen. Classification, association and pattern completion

using neural similarity based methods. Applied Mathemathics and Computer Science,

10:101–120, 2000.

44. W. Duch, R. Adamczak, and G.H.F. Diercksen. Feature space mapping neural network

applied to structure-activity relationship problems. In Soo-Young Lee, editor, 7th International

Conference on Neural Information Processing (ICONIP’2000), pages 270–274,

Dae-jong, Korea, 2000.

45. W. Duch, R. Adamczak, and G.H.F. Diercksen. Constructive density estimation network

based on several different separable transfer functions. In 9th European Symposium on

Artificial Neural Networks, Bruges, Belgium, Apr 2001.

46. W. Duch, R. Adamczak, and K. Gra¸bczewski. Extraction of logical rules from backpropagation

networks. Neural Processing Letters, 7:1–9, 1998.

47. W. Duch, R. Adamczak, and K. Gra¸bczewski. A new methodology of extraction, optimization

and application of crisp and fuzzy logical rules. IEEE Transactions on Neural

Networks, 12:277–306, 2001.

48. W. Duch and M. Blachnik. Fuzzy rule-based systems derived from similarity to prototypes.

Lecture Notes in Computer Science, 3316:912–917, 2004.

49. W. Duch and M. Blachnik. Fuzzy rule-based systems derived from similarity to prototypes.

In N.R. Pal, N. Kasabov, R.K. Mudi, S. Pal, and S.K. Parui, editors, Lecture Notes

in Computer Science, volume 3316, pages 912–917. Physica Verlag, Springer, New York,

2004.

50. W. Duch and G. H. F. Diercksen. Feature space mapping as a universal adaptive system.

Computer Physics Communications, 87:341–371, 1995.

51. W. Duch and K. Gra¸bczewski. Heterogeneous adaptive systems. In IEEE World Congress

on Computational Intelligence, pages 524–529. IEEE Press, Honolulu, May 2002.

52. W. Duch and K. Grudzi´nski. Search and global minimization in similarity-based methods.

In International Joint Conference on Neural Networks, page Paper 742, Washington D.C.,

1999. IEEE Press.

53. W. Duch and K. Grudzi´nski. Meta-learning via search combined with parameter optimization.

In L. Rutkowski and J. Kacprzyk, editors, Advances in Soft Computing, pages 13–22.

Physica Verlag, Springer, New York, 2002.

54. W. Duch and L. Itert. Competent undemocratic committees. In L. Rutkowski and

J. Kacprzyk, editors, Neural Networks and Soft Computing, pages 412–417. Physica Verlag,

Springer, 2002.

55. W. Duch and L. Itert. Committees of undemocratic competent models. In L. Rutkowski and

J. Kacprzyk, editors, Proc. of Int. Conf. on Artificial Neural Networks (ICANN), Istanbul,

pages 33–36, 2003.

56. W. Duch and N. Jankowski. Complex systems, information theory and neural networks. In

First Polish Conference on Neural Networks and Their Applications, pages 224–230, Kule,

Poland, Apr 1994.

57. W. Duch and N. Jankowski. Survey of neural transfer functions. Neural Computing Surveys,

1999.

58. W. Duch and N. Jankowski. Taxonomy of neural transfer functions. In International Joint

Conference on Neural Networks, volume III, pages 477–484, Como, Italy, 2000. IEEE

Press.

59. W. Duch and N. Jankowski. Transfer functions: hidden possibilities for better neural networks.

In 9th European Symposium on Artificial Neural Networks, pages 81–94, Brusells,

Belgium, 2001. De-facto publications.

60. W. Duch, N. Jankowski, A. Naud, and R. Adamczak. Feature space mapping: a neurofuzzy

network for system identification. In Proceedings of the European Symposium on Artificial

Neural Networks, pages 221–224, Helsinki, Aug 1995.

61. W. Duch and J. Mandziuk. Quo vadis computational intelligence? In P. Sincak, J. Vascak,

and K. Hirota, editors, Machine Intelligence: Quo Vadis?, volume 21, pages 3–28. World

Scientific, Advances in Fuzzy Systems – Applications and Theory, 2004.

62. W. Duch, R. Setiono, and J. Zurada. Computational intelligence methods for understanding

of data. Proceedings of the IEEE, 92(5):771–805, 2004.

63. R. O. Duda, P. E. Hart, and D.G. Stork. Patter Classification. J. Wiley & Sons, New York,

2001.

64. R.S. Michalski (ed). Multistrategy Learning. Kluwer Academic Publishers, 1993.

65. S. E. Fahlman and C. Lebiere. The cascade-correlation learning architecture. In D. S.

Touretzky, editor, Advances in Neural Information Processing Systems 2, pages 524–532.

Morgan Kaufmann, 1990.

66. J.H. Friedman. Exploratory projection pursuit. Journal of the American Statistical Association,

82:249–266, 1987.

67. K.S. Fu. Syntactic Pattern Recognition and Applications. Prentice-Hall, New York, 1982.

68. W. Gerstner and W.M. Kistler. Spiking Neuron Models. Single Neurons, Populations, Plasticity.

Cambridge University Press, 2002.

69. G. Giacinto and F. Roli. Dynamic classifier selection based on multiple classifier behaviour.

Pattern Recognition, 34:179–181, 2001.

70. A. Gifi. Nonlinear Multivariate Analysis. Wiley, Boston, 1990.

71. Ch. Giraud-Carrier, R. Vilalta, and P. Brazdil. Introduction to the special issue on metalearning.

Machine Learning, 54:197–194, 2004.

72. L. Goldfarb and D. Gay. What is a structural representation? fifth variation. Technical

Report Technical Report TR05-175, Faculty of Computer Science, University of New

Brunswick, Canada, 2005.

73. L. Goldfarb, D. Gay, O. Golubitsky, and D. Korkin. What is a structural representation? a

proposal for a representational formalism. Pattern Recognition, (submitted), 2006.

74. L. Goldfarb and S. Nigam. The unified learning paradigm: A foundation for ai. In V. Honovar

and L. Uhr, editors, Artificial Intelligence and Neural Networks: Steps Toward Principled

Integration, pages 533–559. Academic Press, Boston, 1994.

75. R.L. Gorsuch. Factor Analysis. Erlbaum, Hillsdale, NJ, 1983.

76. K. Gra¸bczewski and W. Duch. The separability of split value criterion. In Proceedings of

the 5th Conf. on Neural Networks and Soft Computing, pages 201–208, Zakopane, Poland,

2000. Polish Neural Network Society.

77. K. Gra¸bczewski and W. Duch. Forests of decision trees. Neural Networks and Soft Computing,

Advances in Soft Computing, pages 602–607, 2002.

78. K. Gra¸bczewski and W. Duch. Heterogenous forests of decision trees. Springer Lecture

Notes in Computer Science, 2415:504–509, 2002.

79. M. Grochowski and N. Jankowski. Comparison of instance selection algorithms. ii. results

and comments. Lecture Notes in Computer Science, 3070:580–585, 2004.

80. S. Grossberg. How does the cerebral cortex work? development, learning, attention, and

3d vision by laminar circuits of visual cortex. Behavioral and Cognitive Neuroscience

Reviews, 2:47–76, 2003.

81. A. Gutkin. Towards formal structural representation of spoken language: An evolving transformation

system (ets) approach. Technical Report PhD Thesis, School of Informatics,

University of Edinburgh, UK, 2005.

82. I. Guyon, S. Gunn, M. Nikravesh, and L. Zadeh. Feature extraction, foundations and applications.

Physica Verlag, Springer, Berlin, Heidelberg, New York, 2006.

83. L. Gy¨orfi, M. Kohler, A. Krzy˙zak, and H. Walk. A Distribution-Free Theory of Nonparametric

Regression. Springer Series in Statistics, Springer-Verlag, New York, 2002.

84. H. Haas H. Jaeger. Harnessing nonlinearity: Predicting chaotic systems and saving energy

in wireless communication. Science, 304:78–80, 2004.

85. S. Haykin. Neural Networks - A Comprehensive Foundation. Maxwell MacMillian Int.,

New York, 1994.

86. R. Hecht-Nielsen. Cogent confabulation. Neural Networks, 18:111–115, 2005.

87. G.E. Hinton, S. Osindero, and Y. Teh. A fast learning algorithm for deep belief nets. Neural

Computation, 18:381–414, 2006.

88. V. Honavar and L. Uhr, editors. Artificial Intelligence and Neural Networks: Steps Toward

Principled Integration. Academic Press, Boston, 1994.

89. G.-B. Huang, L. Chen, and C.-K. Siew. Universal approximation using incremental constructive