How is Lucid Dreaming Possible?
The question may just as well read "why aren't all dreams lucid?" We all know that dreams can be very realistic, and our emotions in dreams are often even stronger than while awake. So why do we almost always mistake the dream-world for physical reality?
For one thing, the dream theory I believe holding the most credibility states that dreaming is for rehearsing threatening scenarios. This rehearsal should improve our ability to react appropriately to real-life threat scenarios (especially those which were most common in our prehistoric environments). Through practice in dreams, vigilance whilst awake for danger should be enhanced, and our reaction time should be increased. Quicker to perceive an attacker, or a cliff, or a social threat, and faster to run away, or fight back, or avoid falling, etc. Keep in mind that this is all rehearsal. And the best rehearsal is when you don't know "this is only practice." When we think that it is life or death, we tend to give it our all - hence the try and try again aspect of dreaming, even amidst horrifying or at least extremely annoying situations (i.e., trying to get somewhere but not able to or trying to use an electronic device and it keeps malfunctioning).
Also, dreams do not need conscious recall to be effective as threat rehearsals. Motor memory, etched into our reactions through repetition, actually need to be faster than conscious control of behavior can allow. When faced with mortal danger, reactions need to be quicker than the slow, rational, highly complicated mind allows for. Motor memory works regardless of whether or not we recall the practice(s) consciously. Few people remember their first times learning to ride a tricycle or swimming. In fact, almost all memories before 3-years-old are relegated solely to motor memory, yet these early-ingrained memories are the foundation for your behaviors (including thoughts, which are behaviors too).
So if dreaming is, at least in part, a virtual practice-ground for perceiving and dealing with ancient threats, and the system developed so as to be indistinguishable from "this is actually happening," how can lucidity be possible? Well, seems like this is just another hack to human biology that humans have the capacity to perform because of our special brains. Their is nothing like the human brain on Earth. Dolphins have very large brains, but they evolved along a very different path than ours (and they don't dream, by the way. Or at least, don't have REM sleep), and Chimps and Bonobos are cousins to us but cannot match our symbol or tool using abilities (not saying humans are better, but we are different). The human brain has a proficiency for what has been called secondary-consciousness. Whereas primary-consciousness deals with perceiving what is happening NOW (it could be argued that most animals and maybe even plants have this basic consciousness), secondary-consciousness can perceive memories of the past and long-range intentions for the future (and our symbol manipulating capabilities allow us to extend secondary consciousness WAY into the future, even for generations as through verbal stories, writing and pictures). When we say perceive, we are not talking about motor-memories, but rather the phenomenological experience of color, shape, sound, taste, temperature, motion, texture, smell, etc.
In dreams, primary-consciousness dominates. This makes sense: When presented with dire situations, we need to act immediately. When our emotions are heightened, particularly as in fear or anger, we tend to be more reactive than pro-active. The typical events in dreams, such as failing at doing something or aggression from a dream character, has us responding to the moment rather than contemplating the nature of our current reality. In REM sleep and therefor in most dreams, the parts of our brain which are responsible for our most advanced planning and strategizing are deactivated (relative to waking). And the evolutionarily older parts of our brain are VERY active in dreams. The scripts and schema that are tied into our fight-or-flight instincts are active in dreams and this is why we rarely question whether we should be running from that man with a knife behind us.
And yet, we can learn how to (or some people are just able to do this naturally) activate the areas of the brain responsible for "higher" thinking, and this turns out to be exactly how lucid dreaming is possible. Brain imaging has verified that when a person is lucid in a dream, their dorso-lateral pre-frontal cortex (the last part of the brain to evolve, and the last to fully develop in the 20's - later for men than women) is as active as when they are awake (or close to). Exactly how this can be, or why we can learn to stay calm enough to remain immersed in a lucid dream, remain exciting research directions in dream studies.