Moving to

March 21, 2007 at 2:51 pm | Posted in Uncategorized | Leave a comment

Dear Visitor,

as I simply don’t find the time to maintain two separate blog systems, I am moving and merging this blog with our Web site’s blog at – thank you for visiting and make sure to drop by.

Best from Patrick

Uh oh, mystic stuff

June 11, 2006 at 5:15 pm | Posted in Concentration | Leave a comment

Found that through technorati. Don't know what to make of it but did not want to loose it. Read on:

You offer meditation as a means for my leaving my misery behind and all I do is resist. The thought of being still and silent doesn’t excite me; in fact it scares me. Could you explain my resistence to meditation?
    The thought of stillness and silence excites nobody. It is not your personal problem. It is the problem of human mind as such, because to be still, to be silent, means to be in a state of no-mind.

Mind cannot be still. It needs continuous thinking, worrying. The mind functions like a bicycle; if you go on pedaling it, it continues. The moment you stop the pedaling, you are going to fall down. Mind is a two-wheeled vehicle just like a bicycle, and your thinking is a constant pedaling. Even sometimes if you are a little bit silent you immediately start worrying, “Why am I silent?” Anything will do to create worrying, thinking, because mind can exist only in one way — in running, always running after something or running from something, but always running. In the running is the mind. The moment you stop, the mind disappears.

Right now you are identified with the mind. You think you are it. From there comes the fear. If you are identified with the mind, naturally if mind stops you are finished, you are no more. And you don’t know anything beyond mind.

The reality is you are not mind, you are something beyond mind; hence it is absolutely necessary that the mind stops so that for the first time you can know that you are not mind, because you are still there. Mind is gone, you are still there…and with greater joy, greater glory, greater light, greater consciousness, greater being. Mind was pretending, and you had fallen into the trap.

What you have to understand is the process of identification…how one can get identified with something which one is not.

The ancient parable in the East is that a lioness was jumping from one hillock to another hillock and just in the middle she gave birth to a kid. The kid fell down on the road where a big crowd of sheep was passing. Naturally he mixed with the sheep, lived with the sheep, behaved like a sheep. He had no idea, not even in his dreams, that he is a lion. How could he have? All around him were sheep and more sheep. He had never roared like a lion; a sheep does not roar. He had never been alone like a lion; a sheep is never alone. She is always in the crowd; the crowd is cozy, secure, safe. If you see sheep walking, they walk so close together that they are almost stumbling on each other. They are so afraid to be alone.

But the lion started growing up. It was a strange phenomenon. He was identified mentally with being a sheep, but biology does not go according to your identification; nature is not going to follow you.

He became a beautiful young lion, but because things happened so slowly the sheep also became accustomed to the lion while the lion was becoming accustomed to the sheep. The sheep thought he is a little crazy, naturally. He’s not behaving — a little cuckoo — and he goes on growing. It is not supposed to be so. And pretending to be a lion…but he is not a lion. They have seen him from his very birth, they have brought him up, they have given their milk to him. And he was a nonvegetarian by nature. No lion is vegetarian, but this lion was vegetarian because sheep are vegetarian. He used to eat grass with great joy.

They accepted this little difference, that he is a little big and looks like a lion. A very wise sheep said, “It is just a freak of nature. Once in a while it happens.” And he himself also accepted that it is true. His color is different, his body is different; he must be a freak, abnormal. But the idea that he is a lion was impossible! He was surrounded by all those sheep, and sheep psychoanalysts gave him explanations: “You are just a freak of nature. Don’t be worried. We are here to take care of you.”

But one day an old lion passed and saw this young lion far above the crowd of sheep. He could not believe his eyes! He had never seen such a thing nor had he ever heard in the history of the whole past that a lion was in the middle of a crowd of sheep and no sheep was afraid. And the lion was walking exactly like the sheep, grazing on grass.

The old lion could not believe his eyes. He forgot he was going to catch a sheep for his breakfast. He completely forgot the breakfast. It was something so strange that he tried to catch the young lion. But he was old, and the young lion was young — he ran away. Although he believed that he was a sheep, when there was danger the identification was forgotten. He ran like a lion, and the old lion had great difficulty in catching him. But finally the old lion got hold of him and he was crying and weeping and saying, “Just forgive me, I am a poor sheep.” The old lion said, “You idiot! You simply stop and come with me to the pond.”

Just nearby there was a pond. He took the young lion there. The young lion was not going willingly. He went reluctantly, but what can you do against a lion if you are only a sheep? He may kill you if you don’t follow him, so he went with him. The pond was silent, with no ripples, almost like a mirror. And the old lion said to the young, “Just look. Look at my face and look at your face. Look at my body and look at your body in the water.”

In a second there came a great roar! All the hills echoed it. The sheep disappeared; he was a totally different being — he recognized himself. The identification with sheep was not a reality, it was just a mental concept. Now he had seen the reality. And the old lion said, “Now I don’t have to say anything. You have understood.”

The young lion could feel strange energy he had never felt…as if it had been dormant. He could feel tremendous power, and he had always been a weak, humble sheep. All that humbleness, all that weakness, simply evaporated.

This is an ancient parable about the master and the disciple. The function of the master is only to bring the disciple to see who he is and that what he goes on believing is not true.

Your mind is not created by nature. Try to keep the distinction always: your brain is created by nature. Your brain is the mechanism that belongs to the body, but your mind is created by the society in which you live — by the religion, by the church, by the ideology that your parents followed, by the educational system that you were taught in, by all kinds of things. That’s why there is a Christian mind and a Hindu mind, a Mohammedan mind and a communist mind. Brains are natural, but minds are a created phenomenon. It depends on which flock of sheep you belong to. Was the flock of the sheep Hindu? Then naturally you will behave like a Hindu.

Meditation is the only method that can make you aware that you are not the mind; and that gives you a tremendous mastery. Then you can choose what is right with your mind and what is not right with your mind, because you are distant, an observer, a watcher. Then you are not so much attached to the mind, and that is your fear.

You have completely forgotten yourself; you have become the mind. The identification is complete. So when I say, “Be silent. Be still. Be alert and watchful of your thought processes,” you freak out, you become afraid. It looks like death. In a way you are right but it is not your death, it is the death of your conditionings. Combined they are called your mind.

Once you are capable of seeing the distinction clearly — that you are separate from the mind and the mind is separate from the brain — it immediately happens. Simultaneously, as you withdraw from the mind, you suddenly see that the mind is in the middle; on both sides there is brain and consciousness.

The brain is simply a mechanism. Whatever you want to do with it, you can do. Mind is the problem, because others make it for you. It is not you, it is not even your own; it is all borrowed.

The priests, the politicians, the people who are in power, the people who have vested interests, don’t want you to know that you are above mind, beyond mind. Their whole effort has been to keep you identified with the mind, because mind is managed by them, not by you. You are being deceived in such a subtle way. The managers of your mind are outside.

When the consciousness becomes identified with the mind, then the brain is helpless. The brain is simply mechanical. Whatever mind wants, the brain does. But if you are separate, then the mind loses its power; otherwise it is sovereign. And you are afraid of meditation because of that.

I don’t belong to any religion, I don’t belong to any political ideology, I don’t belong to any nation. I don’t have myself filled up with all kinds of nonsense called “holy scriptures.” I have simply pushed the mind aside. I use the brain directly; there is no need of any conditioning, there is no need of any mediator.

But your fear is understandable. You have been brought up with certain concepts, and perhaps you are afraid to lose them.
Osho: The Path of the Mystic 

Everything short of pseudocode

May 25, 2006 at 8:27 pm | Posted in Jeff Hawkins, Numenta | 4 Comments

Numenta published a paper dated May 10th, 2006, entitled Hierarchical Temporal Memory about the theory and concepts of HTM. In a nutshell, they propose to use bayesian networks and belief propagation to model the human neocortex.

Originally, I intended to summarize the paper by stringing quotes together. But the last page says it all, so lets start at the end:

Technically, HTMs can be considered a form of Bayesian network where the network consists of a collection of nodes arranged in a tree-shaped hierarchy. Each node in the hierarchy self-discovers a set of causes in its input through a process of finding common spatial patterns and then finding common temporal patterns. Unlike many Bayesian networks, HTMs are s lf-training, have a well-defined parent/child relationship between each node, inherently handle time-varying data, and afford mechanisms for covert attention. Sensory data is presented at the “bottom” of the hierarchy. To train an HTM, it is necessary to present continuous, timevarying, sensory input while the causes underlying that sensory data persist in the environment. That is, you either move the senses of the HTM through the world, or the objects in the world move relative to the HTM’s senses. During inference, information flows up the hierarchy starting at the lowest level nodes closest to sensory input. As the information rises up the hierarchy, beliefs are formed at successively higher nodes, each representing causes over larger and larger spatial areas and longer and longer temporal periods. Belief propagation-like techniques lead all nodes in the network to quickly reach beliefs that are consistent with the bottoms-up sensory data. Top-down predictions can influence the inference process by biasing the network to settle on predicted causes. HTMs are memory systems. By this we mean that HTMs must learn about their world. You sometimes can supervise the learning process but you can’t program an HTM. Everything an HTM learns is stored in memory matrices at each node. These memory matrices represent the spatial quantization points and sequences learned by the node.

Ok? Lets go back to the beginning. By the way, what you won't find in the paper are algorithms, pseudo-code, proposed data structures, implementation hints, in fact anything tangible to get you going as a developer. Never mind, lets read on:

What do HTM's do? They discover causes (e.g. objects) in the world. One of the goals of an HTM is to discover from the raw sensory input that objects like “cars” and “words” exist. Sensory data will be a topologically arrayed collection of inputs,
where each input measures a local and simple quantity. There are two essential characteristics of sensory data. First, the sensory data must measure something that is directly or indirectly impacted by the causes in the world that you might be interested in. Second, the sensory data must change and flow continuously through time, while the causes underlying the sensory data remain relatively stable. At any moment in time, given current and past input, an HTM will assign a likelihood that individual causes are currently being sensed.


HTMs consist of a hierarchy of memory nodes where each node learns causes and forms beliefs. Part of the learning algorithm (never explained in detail) performed by each node is to store likely sequences of patterns. By combining memory of likely sequences with current input, each node has the ability to make predictions of what is likely to happen next. When an HTM predicts what is likely to happen next, the prediction can act as what is called a “prior probability”, meaning it biases the system to infer the predicted causes.


HTMs are structured as a hierarchy of nodes, where each node is performing the same learning algorithm. Each node in an HTM generally has a fixed number of causes and a fixed number of output variables. The nodes do not “add” causes as they are discovered, instead, over the course of training the meaning of the outputs gradually change. This happens at all levels in the hierarchy simultaneously.

The basic operation of each node is divided into two steps. The first step is to assign the node’s input pattern to one of a set of quantization points (representing common spatial patterns of input). In the second step, the node looks for common sequences of these quantization points. The set of these sequence variables is the output of the node, and is passed up the hierarchy to the parent(s) of the node. A node also can send information to its children. The messages going down the hierarchy represent the distribution over the quantization points, whereas the messages going up the hierarchy represent the distribution over the sequences.


Why is the use of a hierarchy important? HTMs try to match inputs to previously seen patterns, but they do so a piece at a time and in a hierarchy. A node couldn’t store every pattern that it would likely see in its lifetime. Instead, the node stores a limited, fixed number, of patterns, say 50 or 100. These stored patterns are the quantization points. You can think of the quantization points as the most common patterns seen by the node during training. Further training will not increase the number of quantization points, but it can change them. At every moment, the node takes a new and novel input and determines how close (never explained how) it is to each stored quantization point. After sufficient initial training, most new learning occurs in the upper levels of the HTM hierarchy. When training a new HTM from scratch, the lower-level nodes become stable before the upper-level nodes, reflecting the common sub-properties of causes in the world. HTMs do not just exploit the hierarchical spatial structure of the world. They take advantage of the hierarchical temporal structure of the world as well. Nodes at the bottom of an HTM find temporal correlations among patterns that occur relatively close together in both space and time: “pattern B immediately follows pattern A”.


When designing an HTM system for a particular problem, it is important to ask whether the problem space (and the corresponding sensory data) have hierarchical structure. For example, if you desire an HTM to understand financial markets, you might want to present data to the HTM where adjacent sensory input data are likely to be correlated in space and time. Perhaps this means first grouping stock prices by category, and then by industry segment. (E.g. technology stocks such as semiconductors, communications, and biotechnology would get grouped together in the first level. At the next level, the technology group is combined with manufacturing, financial, and other groups.).


A connected graph where each node in the graph represents a belief or set of beliefs is commonly referred to as a Bayesian
network. Thus, HTMs are similar to Bayesian networks. In a Bayesian network, beliefs at one node can modify the beliefs at another node if the two nodes are connected via a conditional probability table (CPT). A CPT is a matrix of numbers where the
columns of the matrix correspond to the individual beliefs from one node and the rows correspond to the individual beliefs from
the other node. Multiplying a vector representing the belief in a source node times the CPT results in a vector in the dimension and “language” of beliefs in the destination node. Belief Propagation (BP) is a mathematical technique that is used
with Bayesian networks. BP doesn’t iterate to reach its final state; it happens in one pass. HTM uses a variation of Belief Propagation to do inference. The sensory data imposes a set of beliefs at the lowest level in an HTM hierarchy, and by the time the beliefs propagate to the highest level, each node in the system represents a belief that is mutually consistent with all the other nodes. The highest level nodes show what highest level causes are most consistent with the inputs at the lowest levels. BP is that it is possible to make large systems that settle rapidly. The time it takes for an HTM to infer its input increases linearly with the number of levels in the hierarchy. However, the memory capacity of the HTM increases exponentially with the number of levels. HTM networks can have millions of nodes, yet still have the longest path be short, say five or ten steps. Because basic belief propagation has no way of handling time-varying data, the concept of time must be added to do inference in these domains. The nodes in an HTM are also more sophisticated than in standard BP.


In summary, there are three sources of dynamic change in an HTM. One occurs because of the changing sensory input. The
second occurs as each node uses its memory of sequences to predict what will happen next and passes this prediction down
the hierarchy. The third happens only during training and at a much slower time scale.


Let’s say a node identifies the fifty most common spatial patterns found in its input. Let’s label the learned spatial patterns sp1 thru sp50. Suppose the node observes that over time sp4 often follows sp7. Assume a node stores the 100 most common temporal sequences. Here then is what nodes in an HTM do. At every point in time, a node looks at its input and assigns a probability that this input matches each element in a set of commonly occurring spatial patterns. Then the node takes this probability distribution and combines it with previous state information to assign a probability that the current input is part of a set of commonly occurring temporal sequences. The distribution over the set of sequences is the output of the node and is passed up the hierarchy. Finally, if the node is still learning, then it might modify the set of stored spatial and temporal patterns to reflect the new input. In summary, we can say that each node in an HTM first learns to represent the most commonly occurring spatial patterns in its input. Then it learns to represent the most commonly occurring sequences of those spatial patterns. The node’s outputs going up the hierarchy are variables that represent the sequences, or more precisely, the probability those sequences are active at this moment in time. A node also may pass predicted spatial patterns down the hierarchy.


The fact that a node always sees distributions means it is generally not practical to simply enumerate and count spatial and
temporal patterns. Probabilistic techniques must be used. For example, the idea of a sequence in an HTM is generally not as
clean as the sequence of notes in a melody. For most causes in the world, it is not clear when a sequence begins or ends. Nodes in an HTM have to decide when the change in the input pattern is sufficient to mark it as a new event. There is much prior art on how to learn spatial patterns with messy real world data. Some of these models try to precisely model parts of the visual cortex. There is less prior art on learning sequences from distributions, at least not in ways that will work in an HTM.


Recall that Bayesian networks send belief messages between nodes. Further recall that CPTs (Conditional Probability Tables)
are two-dimensional memory matrices that convert a belief in one node into the dimension and language of the belief in another node. The CPT allows the belief at one node to modify the belief at another node. Earlier we illustrated CPTs with the example of nodes representing temperature and precipitation. After that, we didn’t explain how the CPTs were learned. Well, we did, but in different language. In an HTM, the CPTs used in passing information from node to node going up the hierarchy are formed as a result of learning the quantization points. The quantization function itself is the CPT. By contrast, in a traditional Bayesian network the causes at each node would be fixed, and the CPT would be created by pairing instantaneous beliefs between two nodes. We can’t do this in HTMs because the causes represented by each node are not fixed and have to be learned. Learning the quantization points is in essence a method of creating a CPT on the fly.


teamslide is ready

April 24, 2006 at 4:17 pm | Posted in Online Software | Leave a comment

Our first web application to perform live web presentations with nothing but a browser (no plug-in's, no Flash), teamslide, was prepared for launch today. Give it a try.

Hodgkin-Huxley history? Faster nerve cells

April 23, 2006 at 2:35 pm | Posted in Research, Theory | Leave a comment

An interdisciplinary team of physicists and neurophysiologists from the Max Planck Institute for Dynamics and Self-Organization in Göttingen and the Ruhr-Universität Bochum has now examined more closely the speed and threshold of action potentials in nerve cells of the cerebral cortex of the mammal brain. Some nerve cells function like high-pass filters; fast signals are transmitted well, slow signals are suppressed. Both aspects of the initiation of the action potential play different roles. The large variability of the threshold potentials allows the cells to ignore slowly varying stimuli. The cells continuously increase their threshold so that in many cases no impulse is initiated at all. The fast activation of action potentials, on the other hand, helps the cells to transmit fast changing signals, even with high precision. According to the Hodgkin-Huxley model, the cells would lack the ability to do this. The better cognitive ability of higher animals, such as cats or humans compared to squid or snails, is not only attributed to the higher number of neurons in the brains of these animals, but also to the manner in which the neurons process signals. To do so, these higher animals presumably use molecular mechanisms which the lower animals do not possess. [link]

Have a hard time focusing?

April 23, 2006 at 12:34 pm | Posted in Concentration | Leave a comment

As an aside: "With all the pressing demands on your time, it is a marvel that you can ever focus on a single issue. Focus on the here and now, otherwise, you will let problems slide, and great opportunities will quickly slip through your hands. Here are some pointers on staying focused. Face unpleasant tasks today to remove those stressors from your agenda. While daydreaming boosts your creativity, at some point you need to start implementing. Even if your course of action is imperfect, doing something is better than always procrastinating. Develop active listening skills through the art of paraphrasing. This ensures that you verify your understanding of a conversation's key points with the person speaking with you. For example, paraphrasing could be "Juan, are you advocating that we expand our parts inventory? Your points about keeping customers happy with quicker service turnarounds are worth considering. I will get back to you on Wednesday." Forcing yourself to repeat a conversation's key points back to the speaker trains you to focus on the conversation, and setting a date to make a decision prevents procrastination. Be Brief. If you can't get your key points across in one minute, you are not focusing. Using the "Five W's" (Who, What, When, Where and Why) lets you concentrate on the key issues. By using these key points, you immediately eliminate extraneous material better left unsaid. The Five W's also help you determine whether a speaker has unintentionally left out important information. And learn to compartmentalize. Effective people focus on one issue at a time. They compartmentalize problems by giving an issue or conversation their undivided attention no matter what else is going on around them. [Source: "Have a hard time focusing?", Kent R. Davies, 2004, link]

Three laws of memory, from 1930

April 23, 2006 at 12:23 pm | Posted in Concentration | 2 Comments

And I (almost) quote: "The best memory systems are built around three basic laws: Concentration, Association, and Repetition. Civilized, intellectual man must be a master of concentration. Imbeciles and idiots are almost devoid of concentrated attention. One chief difference between the mind of a monkey and the mind of a Plato is that a monkey cannot concentrate. He changes from one thing to another almost every instant. The sign royal of brain power is concentration. In relation to the other laws, we must give it first place, using it as a basis for the association and repetition of facts to be remembered. Our ideas are always associated in some way. They come in groups, and each group, in turn, is associated with some other group. Association refers to the ability to recall one fact by relating it to another fact which has been firmly fixed in the mind. In other words, it enables one to remember and recall something, which may be new, elusive, or difficult, by recollecting something which is a first cousin to it, with which the mind is well acquainted. The third great law is Repetition, the follow-up. Without it, memory is only a transient thing. This is the law that makes for permanence. Make any memory impression lasting and enduring, review it frequently, go over it again and again, until it is firmly fixed in the mind. [Source: "Three laws of memory", published 1930, link]

Concentration, pointer #2

April 23, 2006 at 12:08 pm | Posted in Concentration | Leave a comment

Intelligent practice and drill. "For purposeful practice, it is much better to concentrate on something positive rather than negative. Think of this one thing for a given period of time. Allow no outside ideas of any nature to enter your mind, stop them right in their tracks. Initially, you will be surprised to find how difficult this is at first, to hold steady concentration for even one minute. Try it, do it. Any moment of leisure will serve, just your brain and your will. This one topic, imagine it as a sun in the center of your mental universe. Other ideas, ideas sympathetic with it, will rotate about it, as planets about their orb of light; but they must never eclipse it for a second. If the eclipse comes, just tighten the will a bit and try again. Do not be discouraged by the fact that at first you will find a constant struggle going on in your brain. Take comfort from the fact that you are beginning to observe just what is going on in the most important part of your being. After you have noted the starting minute and second exactly, and have begun concentration on the chosen topic, almost at once thought altogether foreign will seek to crowd to the center of the mind. When unrelated ideas truly reach the center of the mind's thought, concentration is broken. Look at your watch then. Note the time that has elapsed. It will be a matter of seconds at first—of hours afterward. When one becomes able to retain the entire attention on a single theme for the space of one minute, without a moment's intrusion of an unrelated thought, then there is already developed the power of mind control, which is the first essential of a masterful memory." [Source: "Concentration - intelligent practice and drill", published 1930, link]

Concentration, pointers #1

April 23, 2006 at 11:54 am | Posted in Concentration | 1 Comment

Actually, the one thing that interests me most in getting explained: Concentration. "The most important intellectual habit of man. Make it a business of doing one thing at a time. It involves a lot of mental discipline, habit-forming and brain-building. The working principle: relaxation precedes perfect concentration. The next step is to free the mind. And to make use of the third aid: right conditions (at least initially, until one is able to concentrate under any condition). Seek a place free from all interruptions, with no outside influences, and in a pleasing environment. Solitude has always been a requisite for great work. The fourth aid: make a daily schedule. It helps to focus the mind, hold it steadily to one thing at a time and in the right order." [Source: "How to concentrate", published 1930, link]

Decline in Perceptual Ability as Infants Get Older

April 19, 2006 at 3:42 am | Posted in Research | 1 Comment

New research finds a decline in perceptual ability as infants get older. These new findings contrast with previous studies that have shown that perceptual abilities improve during childhood. You gain some, you loose some.

Next Page »

Create a free website or blog at | The Pool Theme.
Entries and comments feeds.


Get every new post delivered to your Inbox.