Neural Networks and Chaos theory

Two of the hottest words in psychology today are PDP and Chaos. Not surprisingly, there are some interesting things each has to say about the other, particularly on the issue of human creativity.

Chaos Theory

Complexity fills our world. From trees to crystals, from dripping faucets to the human mind, the universe's seemingly endless variety reigns. Nature's inventiveness never fails to surprise. We can find no identical snowflakes, no two weather patterns that behave in exactly the same way. Indeed, it seems nature constantly evolves in new and unpredictable ways. And why shouldn't it? After all, it seems like there are literally trillions of different variables interacting at any given instant in the real world: a far cry from the simple linear models seen in the conventional sciences, with only two or three variables interacting at once. At times it seems as if nothing could be less deterministic, less removed from our own chaotic experiences, and yet, science has produced some pretty amazing things, just using linear models.

And now, a branch of science is developing which studies non-linear models, a branch which takes into account the interactions of many variables and makes predictions about the resultant outcomes. It uses a theory which studies seemingly random events and looks for the underlying order while at the same time discovering the incredible complexity seen in even very simple systems. It is called Chaos theory, and it may revolutionize the world. (A brief summary of the two major points of Chaos theory follows, but for a more exaustive description try the attractors, or patterns into which apparently random events will fall (Stewart, 1989). This is best depicted in the following illustration. First, picture a large ice cream cone, filled with vegetable oil. Now sprinkle some apple seeds across the top. No matter where one put the apple seeds to begin, eventually they will all settle to the point at the bottom. In Chaos terminology, the seeds would be said to be "attracted to" the bottom of the cone. Likewise, in real world situations, it seems that behaviors which appear random and start out quite complex, eventually settle down into some kind of pattern. The pattern isn't necessarily linear, but it is a recognizable pattern nonetheless, even though individual instances appear quite unrelated to each other. In nature, the weather and fluids, bacteria populations and pinecones have all been observed to coallese around the form of an attractor. In other words, there is a definite pattern to their behavior over extended periods of time. Why this should be the case leads us to the second tenant of chaos theory...

Complex things can arise from very simple rules if those rules are applied to themselves through a process called iteration. For example, in nature, a complex tree may be created by simply instructing the cells to grow forward by a certain amount, and then branch. In this manner, a very simple rule can produce the complex. Of course, in real life, extraneous variables like gravity, temperature, and nurishment effect the exact timing of this branching but the resulting patterns are immediately recognizable nonetheless. In other words, the differences seen are caused by extraneous factors, while the similarities are the result of simple principles being iterated, or applied over and over again. Indeed, the very similarity between trees, provides ample support for the notion that some fundamental principle must be at work, and that it is only extraneous factors which produce the variety seen. This leads to an interesting question: "could human creativity be merely the result of iteration producing the complex from the simple?

Chaotic nature of Creativity

Nature's inventiveness never fails to surprise. We can find no identical snowflakes, no two weather patterns that behave in exactly the same way. Indeed, it seems nature constantly evolves and reinvents itself anew each day. One might even conclude that nature exibits its own brand of creativity, albeit on a much more limited scale than our own.

Yet this aparently simple observation leads, quite naturally, to the hypothesis that our own human creativity may merely be a more developed form of nature's own type of "creativity:" similar to that seen in chaos theory. Does anything like iteration happen in the creative process? Well, certainly some very fine ideas result when one starts with a simple premise and takes it to extremes. After all, isn't the chair simply a glorified rock, paper an idealized writing surface, and the pencil merely a lump of graphite that got out of hand? Even the mighty computer wasn't invented until someone took an electric calculator and iterated it to its natural extreme. In fact, it seems much of our "so called creativity" comes about when one person takes someone else's ideas and expands upon them: creating a pattern of expansion and branching not unlike a simple tree. Indeed, deductive reasoning itself is based on the premise that from very simple, general rules one can deduce more specific complex behaviors. And so it seems that we do appear to use iteration in our own creative processes. Could PDP models and neural networks provide an explanation for why this occurs?

Chaotic Nets?

When it comes to non-linearity and iteration, neural networks are the epitomy. Even the simplest PDP models involve four or five interconnected units evolving silmultaneously, and the more complex models involve thousands. This situation makes such networks ideal for Chaosticians to examine as mini-models for the natural world.

Not surprisingly, neural networks have been shown to exhibit the kinds of properties predicted in Chaos theory: namely organized complexity from iteration of simple rules. And from here, it is only a small step into the mind itself. In other words, assuming that neural networks are accurate models of the human brain, the fact that they exhibit the complexity predicted by Chaos Theory, make it all the more likely that the brain itself should exhibit the same kinds of complexity, perhaps in the form of creativity. It would be very interesting to see if a neural network, looking at the the same kind of data set which was available to a famous inventor, could come up with the same kind of solution. Such a finding would have enormous implications about the nature of human creativity, and indeed, free will itself.

Make no mistake, the chaotic world is mysterious. No one has found an explanation for why random events and variables should organize the way they do, yet it is undeniable that this oganization occurs: from computer inteference to chemical reactions. Even the most ardent opponents of Chaos Theory admit that this underlying organization exists. And since this organization shows up in so many different places, it could only be expected, in retrospect, that it would appear in the human mind as well. So when one sees creative ideas that look suspicously like iteration and flashes of insight that work like attractors, it comes as no great surprise, but rather merely the expected in an unexamined place.


Onward to the Index.

Back to Table of Contents.


Last Modified: Sept 11, 1995.

George Hollich
Temple University
GHollich@astro.ocis.temple.edu