ІПММС

Neurocomputers development by IMMSP NASU

   Neurocomputers are modern intellectual IT media that resemble live creatures neural systems. Unlike common computers, they are programmed via learning on examples. Neurocomputers appeared in early 1990s and quickly gained popularity as powerful means for solving non-formalized or excessively complex tasks for which there are no effective analytical methods. Today neurocomputers are widely used almost in every sphere of human activities for solving various problems, such as image recognition, exposure of hidden dependencies and forecast of their development, data search and protection in computer networks, technical process management, state of environment monitoring and site protection, intellectual robots control, etc.
   Neurotechnology department of the Institute of Mathematical Machines and Systems Problems of the Ukraine National Academy of Science was established more than 15 years ago to conduct researches and applications development in the field of neurocomputers. Scientists of the department under the direction of A.M. Reznik received significant results on neural networks theory and its application, developed neurocomputers which are not inferior to world analogues and are significantly superior to them in some cases. The scientists also developed a variety of neurosystem applications on grants and ordered by foreign companies.
   Theoretical studies pursued by scientists of the department include methods of non-iterative neural networks learning and neural associative memory, multimodule neurocomputers and their learning methods, recurrent dynamic neural networks and neuro-management methods, brain memory mechanisms modeling.

The most important results of theoretical studies:

New method for increasing efficiency of neural associative memory by desaturating weight matrix. The method doubles associative memory capacity and obstacle resistance (1996).
Method and technology of structure optimisation of associative neural networks. Developed technology allows decrease required amount of physical memory in 3-4 times (1998).
Reversible neural associative memory allows not only to memorize new data, but also to delete obsolete patterns from memory (2001).
Architecture, design and training methods for application of multimodular neural networks for complex real world problems (2002).
Partially destroyed neural associative memory restoring method. Method is similar to amnesia treatment (2003).
Kernel neural associative memory for data flows processing (2004) and its reconsolidation method that allows to modify memory content when tracked object is changing (2009).
Model of Hopfield neural ensembles of neocortex (2005-2012).
Dynamic associative memory based on open recurrent network for dynamic objects control (2009).
Neurocontrol method based on control perturbation that allows to simplify control system (2011).
Pseudoregularisation method for Kalman filter based on neurocontrols training. The method allows to increase control robustness in 3 times (2012).

Neurocomputers development:

Experimental series of software neurocomputers "Neuroconstructor" that implements new methods of non-iterative neural networks training for the first time (1996-2000).
Multifunctional neurocomputer NEUROLAND that provides full cycle of applied systems development. It is the first neurocomputer with implemented reversive associative memory. NEUROLAND is used for neural networks studying in KPI (2002).
First computer-aided design system for multimodular neural networks MNN CAD. The system is extensively used for applied neural systems development (2002).
Prototype of intelligent neural system "Neurointellect" that combines neurotechnology methods and artificial intelligence (2009).

Applied neural systems:

Neurosystem for car airbags control (for ATI, USA, 2003).
Neurosystem for pedestrian detection using one video camera (for ATI, USA, 2004).
Odor recognition system (INTAS grant, 2004).
Three-dimentional gestures recognition system (for Samsung, Korea, 2009).
Experimental biometrical identification system "Biocon" (2007).
Business analysis system TrendCaster, that allows to increase forecasts quality by 20% (2010).

Recently, scientists of the department have focused on neurodynamic problems research related to non-stationery processes forecast, such as market dynamics, adaptive behavior management, state of environment monitoring, modeling of operative and long-term brain memory. This problematic meets modern tendencies in neurotechnology and artificial intelligence methods, transfer from model tasks to real non-homogenous and unstable objects and processes. Among our new results there are technology and program for business analytics, biometric identification system for computer network users, multimodal system of neural management, original method of nuclear associative memory reconsolidation, dynamic associative memory on the basis of open recurrent neural network. These results reflect strengthening of focus on more complex dynamic problems and increase in their intellectual level. For the time being, there are problems of brain memory mechanism modeling on the agenda. Their solution involves reconsideration of neural activity modeling concept basis. This refers not only to transition from static to spike neuron models, but also to brain neural ensembles activity principles reconsideration. Our proposed model of Hopfield neuron ensembles is one of the first steps to this reconsideration. Theoretical explanation of amnesia treatment effect and its experimental evidence based on this model show prospects of this line of research. But there arises a new problem connected to sophistication of objects and processes applied for neural networks learning, to loss of their stationarity and stability. A search of fundamentally new learning methods close to methods of brain perception of information which uses not operative memory alone but genetically inherited experience for identifying and memorizing of useful data.

Along with scientific studies scientists of neurotechnology department teach fundamental concepts of neurotechnology and artificial intelligence in KPI and MIPT (KD), work on doctorate and candidate theses. During the time of the department existence there were defended 6 candidate and one doctorate thesis. There have been published over 80 papers in domestic and foreign scientific journals.

Important publications:

1.Reznik A.M., Gorodnichy D.O., Sitchov A.S. Feedback Connections Regulations in Neural Networks with Projective Learning Algorithm // Cybernetics and System Analysis. - 1996. - №6. - pp. 153-162.
2. Sitchov A.S. Connections Selection in Neural Networks with Pseudoinverse Learning Algorithm // Mathematical Machines and Systems. - 1998. - № 2.
3. Reznik A.M. "Non-Iterative Learning for Neural Networks" Proceedings International Joint Conference on Neural Networks (Washington DC, July 10-16, 1999, №548).
4. Kirichenko N.F., Reznik A.M., Schetenyuk S.P. Matrices Pseudoinversion in Associative Memory Design // Cybernetics and System Analysis. - 2001. - № 3. - pp. 18-27.
5. Kussul N.N., Kussul M.E. Implementation of Dynamic Programming Method as Neural Network Based on Classifiers with Feature Space Transformation // Control Machines and Systems. - 2001. - № 1. - pp. 52-58.
6. Reznik A.M, Kalyna E.A., Sadovaya E.G., Dekhtyarenko O.K., Sitchov A.S., Galinskaya A.A. Multifunctional Neurocomputer NEUROLAND // Mathematical Machines and Systems. - 2003. - № 1. - pp. 36-45.
7. Reznik A.M, Kussul M.E., Sitchov A.S., Sadovaya E.G., Kalyna E.A. Computer Aided Design System for Modular Neural Networks CAD MNN // Mathematical Machines and Systems. - 2002. - № 3. - pp. 3-15.
8. A.M. Reznik, A.S. Sitchov, O.K. Dekhtyarenko, D.W. Nowicki. Associative Memories with "Killed" Neurons: the Methods of Recovery // Proc. of the International Joint Conference on Neural Networks, July 20-24, 2003 (Portland, Oregon).
9. Galinskaya A.A. Design and Learning of Modular Classifiers for Applied Problems // Mathematical Machines and Systems. - 2003. - № 2 - pp. 77-86.
10. Reznik A.M., Galinskaya A.A, Dekhtyarenko O.K., Nowicki D.W. Preprocessing of Matrix QCM Sensors Data for the Classification by Means of Neural Network // Sensors and Actuators B. - 2004. - pp. 158-163.
11. Kussul M.E., Sitchov A.S. Neural Network Classifier for Car Security Systems // Mathematical Machines and Systems. - 2004. - № 2. - pp. 15-21.
12. Nowicki D.W. Geometrical Methods in Neural Associative Memory Theory: Network Clasterisation Algorithm Development // Mathematical Machines and Systems. - 2004. - № 4. - pp. 29-37.
13. Kussul M.E., Galinskaya A.A. "Allowed" and "Forbidden" Designs of Modular Networks // Mathematical Machines and Systems. - 2005. - № 3. - pp. 24-35.
14. Nowicki D., Dekhtyarenko O. Averaging on Riemannian manifolds and unsupervised learning using neural associative memory // European Symposium on Artificial Neural Networks (ESANN'05), pp. 181-186, Apr. 2005.
15. Reznik A.M. General Theory of Development // Mathematical Machines and Systems. - 2005. - № 1. - pp. 84-98.
16. Reznik A.M. Hopfield Ensembles in Lateral Cortex Structures // Mathematical Machines and Systems. - 2006. - № 1. - pp. 3-12.
17. Kussul M.E., Sadovaya O.G., Sitchov A.S. Pedestrian Recognition System with Single Camera // Mathematical Machines and Systems. - 2006. - № 3. - pp. 36-43.
18. Reznik A.M. On the Nature of Intelligence // Mathematical Machines and Systems. - 2008. - № 1. - pp. 23-45.
19. Reznik A.М., Dziuba D.A., Chernodub A.M. "Biocon" - Biometrical Identification System // proc. of "Decision Support Systems: Theory and Implementations" conference, 8 Jun 2009,
pp. 189-192.
20. Reznik A.М., Dziuba D.A. Dynamic Associative Memory Based on Open Recurrent Neural Network // Proceeding of IJCNN'09, Atlanta, Georgia, USA, June 14-19, 2009.
21. Nowicki D., Siegelmann H. (2009). The Secret Life of Kernels: Reconsolidation in Flexible memories. // Frontiers in Systems Neuroscience. Conference Abstract: Computational and systems neuroscience. doi: 10.3389/conf.neuro.06.2009.03.271.
22. Kussul M.E., Sitchov A.S., Sadovaya E.G., Antonenko A.O. Decision Support System "Business Analytics" // Mathematical Machines and Systems. - 2010. - №3. - pp. 96-103.
23. Reznik A.М., Dziuba D.A. Dynamic Neural Cortical Ensembles // Proceeding of XIV conference «NEUROINFORMATICS -2012», pp. 37-44.
24. Dziuba D.A., Chernodub A.M. Control Perturbation Method for Real-Time Neurocontrollers Modification // Mathematical Machines and Systems. - 2011. - № 1. - pp. 20-28.