It has been suggested that Artificial neural network be merged into this article or section. (Discuss)

Traditionally, the term neural network had been used to refer to a network or circuitry of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes. Thus the term 'Neural Network' has two distinct connotations: Image File history File links Mergefrom. ... An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ... Drawing by Santiago Ramón y Cajal of neurons in the pigeon cerebellum. ... An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ... An artificial neuron (also called a node or Nv neuron or Binary neuron or McCulloch-Pitts neuron) is an abstraction of biological neurons and the basic unit in an artificial neural network. ...

1. Biological neural networks are made up of real biological neurons that are connected or functionally-related in the peripheral nervous system or the central nervous system. In the field of neuroscience, they are often identified as groups of neurons that perform a specific physiological function in laboratory analysis.
2. Artificial neural networks are made up of interconnecting artificial neurons (programming contructs that mimics the properties of biological neurons). Artificial neural networks may either be used to gain an understanding of biological neural networks, or for solving artificial intelligence problems without necessarily creating a model of real biological system.

Please see the corresponding articles for details on artificial neural networks or biological neural networks. This article focuses on the relationship between the two concepts. In cognitive neuroscience, a neural network (also known as a neuronal network or biological neural network to distinguish from artificial neural networks) is a population of interconnected neurons. ... The Peripheral nervous system resides or extends outside the CNS central nervous system (the brain and spinal cord) to serve the limbs and organs. ... A diagram showing the CNS: 1. ... Drawing of the cells in the chicken cerebellum by S. Ramón y Cajal Neuroscience is a field that is devoted to the scientific study of the nervous system. ... An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ...

Characterization

In general a biological neural network is composed of a group or groups of chemically connected or functionally associated neurons. A single neuron may be connected to many other neurons and the total number of neurons and connections in a network may be extensive. Connections, called synapses, are usually formed from axons to dendrites, though dendrodendritic microcircuits ^[1] and other connections are possible. Apart from the electrical signaling, there are other forms of signaling that arise from neurotransmitter diffusion, which have an effect on electrical signaling. As such, neural networks are extremely complex. Whilst a detailed description of neural systems is nebulous, progress is being charted towards a better understanding of basic mechanisms. Synapses allow nerve cells to communicate with one another through axons and dendrites, converting electrical signals into chemical ones. ... An axon, or nerve fiber, is a long slender projection of a nerve cell, or neuron, which conducts electrical impulses away from the neurons cell body or soma. ... In biology, a dendrite is a slender, typically branched projection of a nerve cell, or neuron, which conducts the electrical stimulation received from other cells to the body or soma of the cell from which it projects. ... Chemical structure of D-aspartic acid, a common amino acid neurotransmitter. ...

Simplified view of an artificial neural network

Artificial intelligence and cognitive modeling try to simulate some properties of neural networks. While similar in their techniques, the former has the aim of solving particular tasks, while the latter aims to build mathematical models of biological neural systems. Image File history File links No higher resolution available. ... Image File history File links No higher resolution available. ... AI redirects here. ... The term cognitive model has basically two meanings. ...

In the artificial intelligence field, artificial neural networks have been applied successfully to speech recognition, image analysis and adaptive control, in order to construct software agents (in computer and video games) or autonomous robots. Most of the currently employed artificial neural networks for artificial intelligence are based on statistical estimation, optimization and control theory. AI redirects here. ... Speech recognition (in many contexts also known as automatic speech recognition, computer speech recognition or erroneously as Voice Recognition) is the process of converting a speech signal to a sequence of words, by means of an algorithm implemented as a computer program. ... Image analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques. ... Look up control in Wiktionary, the free dictionary. ... In computer science, a software agent is a piece of autonomous, or semi-autonomous proactive and reactive, computer software. ... Computer and video games redirects here. ... Autonomous robots are robots which can perform desired tasks in unstructured environments without continuous human guidance. ... There are many different ways of discussing statistical estimation. ... In mathematics, the term optimization, or mathematical programming, refers to the study of problems in which one seeks to minimize or maximize a real function by systematically choosing the values of real or integer variables from within an allowed set. ... For control theory in psychology and sociology, see control theory (sociology). ...

The cognitive modelling field is the physical or mathematical modeling of the behaviour of neural systems; ranging from the individual neural level (e.g. modelling the spike response curves of neurons to a stimulus), through the neural cluster level (e.g. modelling the release and effects of dopamine in the basal ganglia) to the complete organism (e.g. behavioural modelling of the organism's response to stimuli). The term cognitive model has basically two meanings. ...

The brain, neural networks and computers

Neural networks, as used in artificial intelligence, have traditionally been viewed as simplified models of neural processing in the brain, even though the relation between this model and brain biological architecture is debated. To answer this question, David Marr has proposed various levels of analysis which provide us with a plausible answer for the role of neural networks in the understanding of human cognitive functioning. David Marr (January 19, 1945 - November 17, 1980) was a British psychologist. ...

A subject of current research in theoretical neuroscience is the question surrounding the degree of complexity and the properties that individual neural elements should have to reproduce something resembling animal intelligence.

Historically, computers evolved from the von Neumann architecture, which is based on sequential processing and execution of explicit instructions. On the other hand, the origins of neural networks are based on efforts to model information processing in biological systems, which may rely largely on parallel processing as well as implicit instructions based on recognition of patterns of 'sensory' input from external sources. In other words, rather than sequential processing and execution, at their very heart, neural networks are complex statistical processors. Design of the Von Neumann architecture For the robotic architecture also named after Von Neumann, see Von Neumann machine The von Neumann architecture is a computer design model that uses a single storage structure to hold both instructions and data. ...

Neural networks and artificial intelligence

Main article: Artificial neural network

An artificial neural network (ANN), also called a simulated neural network (SNN) or commonly just neural network (NN) is an interconnected group of artificial neurons that uses a mathematical or computational model for information processing based on a connectionist approach to computation. In most cases an ANN is an adaptive system that changes its structure based on external or internal information that flows through the network. An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ... An artificial neuron (also called a node or Nv neuron or Binary neuron or McCulloch-Pitts neuron) is an abstraction of biological neurons and the basic unit in an artificial neural network. ... A mathematical model is an abstract model that uses mathematical language to describe the behaviour of a system. ... In general, information processing is the changing (processing) of information in any manner detectable by an observer. ... Connectionism is an approach in the fields of artificial intelligence, cognitive science, neuroscience, psychology and philosophy of mind. ... Look up computation in Wiktionary, the free dictionary. ... An adaptive system is a system that is able to adapt its behavior according to changes in its environment or in parts of the system itself. ...

In more practical terms neural networks are non-linear statistical data modeling or decision making tools. They can be used to model complex relationships between inputs and outputs or to find patterns in data. To do: 20th century mathematics chaos theory, fractals Lyapunov stability and non-linear control systems non-linear video editing See also: Aleksandr Mikhailovich Lyapunov Dynamical system External links http://www. ... For Wikipedia statistics, see m:Statistics Statistics is the science and practice of developing human knowledge through the use of empirical data expressed in quantitative form. ... In computer science, data modeling is the process of creating a data model by applying a data model theory to create a data model instance. ... Decision making is the cognitive process of selecting a course of action from among multiple alternatives. ... Pattern recognition is a field within the area of machine learning. ...

Background

An artificial neural network involves a network of simple processing elements (artificial neurons) which can exhibit complex global behaviour, determined by the connections between the processing elements and element parameters. One classical type of artificial neural network is the Hopfield net. An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ... The artificial neuron (also called node) is the basic unit of an artificial neural network, simulating a biological neuron. ... A Hopfield net is a form of recurrent artificial neural network invented by John Hopfield. ...

In a neural network model simple nodes, which can be called variously "neurons", "neurodes", "Processing Elements" (PE) or "units", are connected together to form a network of nodes — hence the term "neural network". While a neural network does not have to be adaptive per se, its practical use comes with algorithms designed to alter the strength (weights) of the connections in the network to produce a desired signal flow. The artificial neuron (also called node) is the basic unit of an artificial neural network, simulating a biological neuron. ...

In modern software implementations of artificial neural networks the approach inspired by biology has more or less been abandoned for a more practical approach based on statistics and signal processing. In some of these systems neural networks, or parts of neural networks (such as artificial neurons) are used as components in larger systems that combine both adaptive and non-adaptive elements. Neural network software is used to simulate, research, develop and apply artificial neural networks, biological neural networks and in some cases a wider array of adaptive systems. ... An artificial neuron (also called a node or Nv neuron or Binary neuron or McCulloch-Pitts neuron) is an abstraction of biological neurons and the basic unit in an artificial neural network. ...

Applications

The utility of artificial neural network models lies in the fact that they can be used to infer a function from observations. This is particularly useful in applications where the complexity of the data or task makes the design of such a function by hand impractical.

Real life applications

The tasks to which artificial neural networks are applied tend to fall within the following broad categories:

• Function approximation, or regression analysis, including time series prediction and modelling.
• Classification, including pattern and sequence recognition, novelty detection and sequential decision making.
• Data processing, including filtering, clustering, blind signal separation and compression.

Application areas include system identification and control (vehicle control, process control), game-playing and decision making (backgammon, chess, racing), pattern recognition (radar systems, face identification, object recognition and more), sequence recognition (gesture, speech, handwritten text recognition), medical diagnosis, financial applications, data mining (or knowledge discovery in databases, "KDD"), visualisation and e-mail spam filtering. The need for function approximations arises in many branches of applied mathematics, and computer science in particular. ... In statistics, regression analysis examines the relation of a dependent variable (response variable) to specified independent variables (explanatory variables). ... In statistics and signal processing, a time series is a sequence of data points, measured typically at successive times, spaced apart at uniform time intervals. ... Statistical classification is a type of supervised learning problem in which labeled training data is used to create a function that will correctly predict the label of future data. ... Pattern recognition is a field within the area of machine learning. ... For other uses, see Data entry clerk. ... Blind signal separation, a. ... Kurt Thearling, An Introduction to Data Mining (also available is a corresponding online tutorial) Dean Abbott, I. Philip Matkovsky, and John Elder IV, Ph. ... E-mail spam, also known as bulk e-mail or junk e-mail is a subset of spam that involves sending nearly identical messages to numerous recipients by e-mail. ...

Neural network software

Main article: Neural network software Neural network software is used to simulate, research, develop and apply artificial neural networks, biological neural networks and in some cases a wider array of adaptive systems. ...

Neural network software is used to simulate, research, develop and apply artificial neural networks, biological neural networks and in some cases a wider array of adaptive systems. This article is about the general term. ... This article is about the concept. ... “Software development” redirects here. ... An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ... In cognitive neuroscience, a neural network (also known as a neuronal network or biological neural network to distinguish from artificial neural networks) is a population of interconnected neurons. ... An adaptive system is a system that is able to adapt its behavior according to changes in its environment or in parts of the system itself. ...

Learning paradigms

There are three major learning paradigms, each corresponding to a particular abstract learning task. These are supervised learning, unsupervised learning and reinforcement learning. Usually any given type of network architecture can be employed in any of those tasks. Supervised learning is a machine learning technique for creating a function from training data. ... Unsupervised learning is a method of machine learning where a model is fit to observations. ... Reinforcement learning refers to a class of problems in machine learning which postulate an agent exploring an environment in which the agent perceives its current state and takes actions. ...

Supervised learning

In supervised learning, we are given a set of example pairs and the aim is to find a function f in the allowed class of functions that matches the examples. In other words, we wish to infer how the mapping implied by the data and the cost function is related to the mismatch between our mapping and the data. Supervised learning is a machine learning technique for creating a function from training data. ...

Unsupervised learning

In unsupervised learning we are given some data x, and a cost function to be minimized which can be any function of x and the network's output, f. The cost function is determined by the task formulation. Most applications fall within the domain of estimation problems such as statistical modeling, compression, filtering, blind source separation and clustering. Unsupervised learning is a method of machine learning where a model is fit to observations. ... A statistical model is used in applied statistics. ... “Source coding” redirects here. ... A mail filter is a piece of software which takes an input of an e-mail message. ... Blind signal separation, a. ... Clustering is the classification of objects into different groups, or more precisely, the partitioning of a data set into subsets (clusters), so that the data in each subset (ideally) share some common trait - often proximity according to some defined distance measure. ...

Reinforcement learning

In reinforcement learning, data x is usually not given, but generated by an agent's interactions with the environment. At each point in time t, the agent performs an action y_t and the environment generates an observation x_t and an instantaneous cost c_t, according to some (usually unknown) dynamics. The aim is to discover a policy for selecting actions that minimises some measure of a long-term cost, i.e. the expected cumulative cost. The environment's dynamics and the long-term cost for each policy are usually unknown, but can be estimated. ANNs are frequently used in reinforcement learning as part of the overall algorithm. Tasks that fall within the paradigm of reinforcement learning are control problems, games and other sequential decision making tasks. Reinforcement learning refers to a class of problems in machine learning which postulate an agent exploring an environment in which the agent perceives its current state and takes actions. ... Look up control in Wiktionary, the free dictionary. ... For other uses, see Game (disambiguation). ...

Learning algorithms

There are many algorithms for training neural networks; most of them can be viewed as a straightforward application of optimization theory and statistical estimation. In mathematics, the term optimization, or mathematical programming, refers to the study of problems in which one seeks to minimize or maximize a real function by systematically choosing the values of real or integer variables from within an allowed set. ... There are many different ways of discussing statistical estimation. ...

Evolutionary computation methods, simulated annealing, expectation maximization and non-parametric methods are among other commonly used methods for training neural networks. See also machine learning. In computer science evolutionary computation is a subfield of artificial intelligence (more particularly computational intelligence) involving combinatorial optimization problems. ... For other uses, see Annealing. ... In statistical computing, an expectation-maximization (EM) algorithm is an algorithm for finding maximum likelihood estimates of parameters in probabilistic models, where the model depends on unobserved latent variables. ... The branch of statistics known as non-parametric statistics is concerned with non-parametric statistical models and non-parametric statistical tests. ... As a broad subfield of artificial intelligence, machine learning is concerned with the design and development of algorithms and techniques that allow computers to learn. At a general level, there are two types of learning: inductive, and deductive. ...

Recent developments in this field also saw the use of particle swarm optimization and other swarm intelligence techniques used in the training of neural networks. Particle swarm optimization (PSO) is a stochastic, population-based computer problem-solving algorithm; it is a kind of swarm intelligence that is based on social-psychological principles and provides insights into social behavior, as well as contributing to engineering applications. ... Swarm intelligence (SI) is an artificial intelligence technique based around the study of collective behavior in decentralized, self-organized systems. ...

Neural networks and neuroscience

Theoretical and computational neuroscience is the field concerned with the theoretical analysis and computational modeling of biological neural systems. Since neural systems are intimately related to cognitive processes and behaviour, the field is closely related to cognitive and behavioural modeling.

The aim of the field is to create models of biological neural systems in order to understand how biological systems work. To gain this understanding, neuroscientists strive to make a link between observed biological processes (data), biologically plausible mechanisms for neural processing and learning (biological neural network models) and theory (statistical learning theory and information theory).

Types of models

Many models are used in the field, each defined at a different level of abstraction and trying to model different aspects of neural systems. They range from models of the short-term behaviour of individual neurons, through models of how the dynamics of neural circuitry arise from interactions between individual neurons, to models of how behaviour can arise from abstract neural modules that represent complete subsystems. These include models of the long-term and short-term plasticity of neural systems and its relation to learning and memory, from the individual neuron to the system level.

Current research

While initially research had been concerned mostly with the electrical characteristics of neurons, a particularly important part of the investigation in recent years has been the exploration of the role of neuromodulators such as dopamine, acetylcholine, and serotonin on behaviour and learning. Neuromodulators modulate regions or circuits of the brain. ... For other uses, see Dopamine (disambiguation). ... The chemical compound acetylcholine, often abbreviated as ACh, was the first neurotransmitter to be identified. ... Serotonin (pronounced ) (5-hydroxytryptamine, or 5-HT) is a monoamine neurotransmitter synthesized in serotonergic neurons in the central nervous system (CNS) and enterochromaffin cells in the gastrointestinal tract of animals including humans. ...

See also

• Neuroscience
• Cognitive science

Drawing of the cells in the chicken cerebellum by S. Ramón y Cajal Neuroscience is a field that is devoted to the scientific study of the nervous system. ... Cognitive science is usually defined as the scientific study either of mind or of intelligence (e. ...

References

• Peter Dayan, L.F. Abbott. Theoretical Neuroscience. MIT Press. 
• Wulfram Gerstner, Werner Kistler. Spiking Neuron Models:Single Neurons, Populations, Plasticity. Cambridge University Press. 

History of the neural network analogy

Main article: Connectionism

The concept of neural networks started in the late-1800s as an effort to describe how the human mind performed. These ideas started being applied to computational models with the Perceptron. Connectionism is an approach in the fields of artificial intelligence, cognitive science, neuroscience, psychology and philosophy of mind. ... The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. ...

In early 1950s Friedrich Hayek was one of the first to posit the idea of spontaneous order in the brain arising out of decentralized networks of simple units (neurons). In the late 1940s, Donald Hebb made one of the first hypotheses for a mechanism of neural plasticity (i.e. learning), Hebbian learning. Hebbian learning is considered to be a 'typical' unsupervised learning rule and it (and variants of it) was an early model for long term potentiation. Friedrich August von Hayek, CH (May 8, 1899 in Vienna – March 23, 1992 in Freiburg) was an Austrian-born British economist and political philosopher known for his defense of liberal democracy and free-market capitalism against socialist and collectivist thought in the mid-20th century. ... Donald Olding Hebb (July 22, 1904-August 20, 1985) was an influential psychologist, particularly in the area of neuropsychology, where he sought to understand how the function of neurons contributed to psychological processes such as learning. ... Hebbian learning is a hypothesis for how neuronal connections are enforced in mammalian brains; it is also a technique for weight selection in artificial neural networks. ... In neuroscience, long-term potentiation (LTP) is the strengthening (or potentiation) of the connection between two nerve cells which lasts for an extended period of time (minutes to hours in vitro and hours to days and months in vivo). ...

The Perceptron is essentially a linear classifier for classifying data specified by parameters and an output function f = w'x + b. Its parameters are adapted with an ad-hoc rule similar to stochastic steepest gradient descent. Because the inner product is linear operator in the input space, the Perceptron can only perfectly classify a set of data for which different classes are linearly separable in the input space, while it often fails completely for non-separable data. While the development of the algorithm initially generated some enthusiasm, partly because of its apparent relation to biological mechanisms, the later discovery of this inadequacy caused such models to be abandoned until the introduction of non-linear models into the field. The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. ... In geometry, when two sets of points in a two-dimensional graph can be completely separated by a single line, they are said to be linearly separable. ...

The Cognitron (1975) was an early multilayered neural network with a training algorithm. The actual structure of the network and the methods used to set the interconnection weights change from one neural strategy to another, each with its advantages and disadvantages. Networks can propagate information in one direction only, or they can bounce back and forth until self-activation at a node occurs and the network settles on a final state. The ability for bi-directional flow of inputs between neurons/nodes was produced with the Hopfield's network (1982), and specialization of these node layers for specific purposes was introduced through the first hybrid network. A Hopfield net is a form of recurrent artificial neural network invented by John Hopfield. ... The term hybrid neural network can have two meanings: biological neural networks interacting with artificial neuronal models, and Artificial neural networks with a symbolic part (or, conversely, symbolic computations with a connectionist part). ...

The parallel distributed processing of the mid-1980s became popular under the name connectionism. Connectionism is an approach in the fields of artificial intelligence, cognitive science, neuroscience, psychology and philosophy of mind. ... Connectionism is an approach in the fields of artificial intelligence, cognitive science, neuroscience, psychology and philosophy of mind. ...

The rediscovery of the backpropagation algorithm was probably the main reason behind the repopularisation of neural networks after the publication of "Learning Internal Representations by Error Propagation" in 1986 (Though backpropagation itself dates from 1974). The original network utilised multiple layers of weight-sum units of the type f = g(w'x + b), where g was a sigmoid function or logistic function such as used in logistic regression. Training was done by a form of stochastic steepest gradient descent. The employment of the chain rule of differentiation in deriving the appropriate parameter updates results in an algorithm that seems to 'backpropagate errors', hence the nomenclature. However it is essentially a form of gradient descent. Determining the optimal parameters in a model of this type is not trivial, and steepest gradient descent methods cannot be relied upon to give the solution without a good starting point. In recent times, networks with the same architecture as the backpropagation network are referred to as Multi-Layer Perceptrons. This name does not impose any limitations on the type of algorithm used for learning. Backpropagation is a supervised learning technique used for training artificial neural networks. ... The logistic curve A sigmoid function is a mathematical function that produces a sigmoid curve — a curve having an S shape. ... Logistic curve, specifically the sigmoid function A logistic function or logistic curve models the S-curve of growth of some set P. The initial stage of growth is approximately exponential; then, as competition arises, the growth slows, and at maturity, growth stops. ... Logistic regression is a statistical regression model for Bernoulli-distributed dependent variables. ...

The backpropagation network generated much enthusiasm at the time and there was much controversy about whether such learning could be implemented in the brain or not, partly because a mechanism for reverse signalling was not obvious at the time, but most importantly because there was no plausible source for the 'teaching' or 'target' signal.

Criticism

A. K. Dewdney, a former Scientific American columnist, wrote in 1997, “Although neural nets do solve a few toy problems, their powers of computation are so limited that I am surprised anyone takes them seriously as a general problem-solving tool.” (Dewdney, p.82) Scientific American is a popular-science magazine, published (first weekly and later monthly) since August 28, 1845, making it the oldest continuously published magazine in the United States. ...

Arguments against Dewdney's position are that neural nets have been successfully used to solve many complex and diverse tasks, ranging from autonomously flying aircraft [1] to detecting credit card fraud [2].

Technology writer Roger Bridgman commented on Dewdney's statements about neural nets: "Neural networks, for instance, are in the dock not only because they have been hyped to high heaven, (what has not?) but also because you could create a successful net without understanding how it worked: the bunch of numbers that captures its behaviour would in all probability be "an opaque, unreadable table...valueless as a scientific resource".

In spite of his emphatic declaration that science is not technology, Dewdney seems here to pillory neural nets as bad science when most of those devising them are just trying to be good engineers. An unreadable table that a useful machine could read would still be well worth having."[3]

See also

ADALINE (Adaptive Linear Neuron or later Adaptive Linear Element) is a single layer neural network. ... An artificial neural network (ANN), often just called a neural network (NN), is a mathematical model or computational model based on biological neural networks. ... Biological Cybernetics investigates communication and control processes in living organisms and ecosystems. ... Biologically-inspired computing (also bio-inspired computing) is a field of study that loosely knits together subfields related to the topics of connectionism, social behaviour and emergence. ... A cognitive architecture is a blueprint for intelligent agents. ... Neural network software is used to simulate, research, develop and apply artificial neural networks, biological neural networks and in some cases a wider array of adaptive systems. ... In the field of artificial intelligence, neuro-fuzzy refers to hybrids of artificial neural networks and fuzzy logic. ... Connectionism today generally refers to an approach in the fields of cognitive psychology, cognitive science and philosophy of mind which models mental or behavioral phenomena with neural networks, and is associated with a certain set of arguments for why this is a good idea. ... Predictive analytics encompasses a variety of techniques from statistics and data mining that process current and historical data in order to make “predictions” about future events. ... A radial basis function network is an artificial neural network which uses radial basis functions as activation functions. ... Simulated reality is the idea that reality could be simulated — often computer-simulated — to a degree indistinguishable from true reality. ... Support vector machines (SVMs) are a set of related supervised learning methods used for classification and regression. ... A tensor product network, in neural networks, is a network that exploits the properties of tensors to model associative concepts such as variable assignment. ... This article or section does not cite its references or sources. ...

References

• Abdi, H. "A neural network primer. Journal of Biological Systems, 2, 247-281, (1994)".
• Abdi, H. "[4] (2003). Neural Networks. In M. Lewis-Beck, A. Bryman, T. Futing (Eds): Encyclopedia for research methods for the social sciences. Thousand Oaks (CA): Sage. pp. 792-795.]".
• Abdi, H. "[5] (2001). Linear algebra for neural networks. In N.J. Smelser, P.B. Baltes (Eds.): International Encyclopedia of the Social and Behavioral Sciences. Oxford (UK): Elsevier.]".
• Abdi, H., Valentin, D., Edelman, B.E. (1999). Neural Networks. Thousand Oaks: Sage.
• Anderson, James A. (1995). An Introduction to Neural Networks. ISBN 0-262-01144-1. 
• Arbib, Michael A. (Ed.) (1995). The Handbook of Brain Theory and Neural Networks. 
• Alspector, U.S. Patent 4,874,963  "Neuromorphic learning networks". October 17, 1989.
• Agree, Philip E., et al. (1997). Comparative Cognitive Robotics: Computation and Human Experience. Cambridge University Press. ISBN 0-521-38603-9. , p. 80
• Bar-Yam, Yaneer (2003). Dynamics of Complex Systems, Chapter 2. 
• Bar-Yam, Yaneer (2003). Dynamics of Complex Systems, Chapter 3. 
• Bar-Yam, Yaneer (2005). Making Things Work.  See chapter 3.
• Bertsekas, Dimitri P. (1999). Nonlinear Programming. 
• Bertsekas, Dimitri P. & Tsitsiklis, John N. (1996). Neuro-dynamic Programming. 
• Boyd, Stephen & Vandenberghe, Lieven (2004). Convex Optimization. 
• Dewdney, A. K. (1997). Yes, We Have No Neutrons: An Eye-Opening Tour through the Twists and Turns of Bad Science. Wiley, 192 pp.  See chapter 5.
• Fukushima, K. (1975). "Cognitron: A Self-Organizing Multilayered Neural Network". Biological Cybernetics 20: 121–136. 
• Frank, Michael J. (2005). "Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Non-medicated Parkinsonism". Journal of Cognitive Neuroscience 17: 51–72. 
• Gardner, E.J., & Derrida, B. (1988). "Optimal storage properties of neural network models". Journal of Physics A 21: 271–284. 
• Krauth, W., & Mezard, M. (1989). "Storage capacity of memory with binary couplings". Journal de Physique 50: 3057–3066. 
• Maass, W., & Markram, H. (2002). "On the computational power of recurrent circuits of spiking neurons". Journal of Computer and System Sciences 69(4): 593–616. 
• MacKay, David (2003). Information Theory, Inference, and Learning Algorithms. 
• Mandic, D. & Chambers, J. (2001). Recurrent Neural Networks for Prediction: Architectures, Learning algorithms and Stability. Wiley. 
• Minsky, M. & Papert, S. (1969). An Introduction to Computational Geometry. MIT Press. 
• Muller, P. & Insua, D.R. (1995). "Issues in Bayesian Analysis of Neural Network Models". Neural Computation 10: 571–592. 
• Reilly, D.L., Cooper, L.N. & Elbaum, C. (1982). "A Neural Model for Category Learning". Biological Cybernetics 45: 35–41. 
• Rosenblatt, F. (1962). Principles of Neurodynamics. Spartan Books. 
• Sutton, Richard S. & Barto, Andrew G. (1998). Reinforcement Learning : An introduction. 
• Van den Bergh, F. Engelbrecht, AP. "Cooperative Learning in Neural Networks using Particle Swarm Optimizers". CIRG 2000.
• Wilkes, A.L. & Wade, N.J. (1997). "Bain on Neural Networks". Brain and Cognition 33: 295–305. 
• Wasserman, P.D. (1989). Neural computing theory and practice. Van Nostrand Reinhold. 
• Jeffrey T. Spooner, Manfredi Maggiore, Raul Ord onez, and Kevin M. Passino, Stable Adaptive Control and Estimation for Nonlinear Systems: Neural and Fuzzy Approximator Techniques, John Wiley and Sons, NY, 2002.