Sensation, the feeling of active data across the natural senses, and unnatural ones too. In parts closely linked to neuroprosthetics, in part its own field of study, with sensation we are striving to affect the senses; to give them back to those who have lost them, to provide new ones to those who would like them, and to bring the virtual to be more meaningful to those who have lived in the physical, by delivering the sensation of being.
Sections
A Dark World
It is beyond dark, it is pitch black. No motes of light, to glittering beams, no nothing. You blink and cannot tell if you did just blink or not. Away in the distance a humming begins. You hear it, and turn towards it. It is coming from your left, and down a ways. Beyond that you cannot tell. Reaching out, you grasp at empty air. The fear rises as you reach out for something, anything to grasp on, and through some flailing, you find a smooth wall.
George Berkeley - the Bishop in the Cybermall
George Berkeley was an Anglo-Irish thinker and Anglican bishop. Born March 3, 1685, he was he first existantual philosopher to publish his views; views that pertain to modern virtual reality as much as they did his world. To be means to be perceived, or esse est percipi
Mice and keyboards, two great tastes...
A good article that puts to words in simple language that which we all know: Virtual environments for the masses are only creeping along because of the interface problem. For most users, a mouse and a keyboard is all they have for input, and true interaction requires just so much more.
Multiple Sensory Modalities Proven to Reinforce Each Other
 Usually when we think of sensory input channels - sight, sound, touch, taste, smell, balance - we think of the traditional hierarchical order, under the brain. It seems that that hierarchy is not as cut and dried as we once thought, with interesting implications for virtual worlds.
Proprioception
Proprioception is one of the main standard senses of the body, and arguably one of the most important. Unlike sight, sound, touch, taste, smell and balance, proprioception is an unconscious sense. That is to say it is one we are feeling all the time, and rarely if ever creeps into the conscious mind unless we actually sit down and think about it.
A slide from the presentation Introduction to Virtual Reality which bears repeating independently, when it comes to virtual or augmented senses versus 'natural'.
Simone, a story ahead of its time of a film producer trying to save his career by employing a virtual actress to star in his films. Unfortunately, the actress is an avatar ? he has to animate her himself, which leads to a whole host of potent issues...
The Looming Shadow of DVT
DVT or Deep Vein Thrombosis is a condition which is already known to affect airline travellers, and office workers. Thrombosis occurs when people are sitting still for long periods of time. Blood clots form in the arteries where blood is settling over long periods of time - as opposed to circulating properly when moving briskly around. This tends to be more of a problem in the lower legs when sitting, but can occur in any part of the body when fully reclined.
Michael Heim, the author of this book, describes modern 'VR' as a pale imitation of real VR. On this point, he is of course, all too correct. He then goes on to state that the real thing is fast approaching. We'll soon be able to totally immerse ourselves in detail-rich, highly interactive artificial worlds....
VRD or Virtual Retinal Display is an offshoot of HMD display technology, which, instead of placing a pair of display screens in front of the eyes, actually projects an image directly onto the human retina with low-energy lasers or LCDs.
Most people are unaware of the haptic and tactile terms for virtually recreating physical touch, and sensation. Yet these two fields, and others besides, are very real, and bringing us ever closer to total sensation.
A Ghostly World
You are travelling through a forest, dappled sunlight streaming through the branches above, casting shadows on the ground. Leaves crunch underfoot, and the odour floats up to you. Closing your eyes, you reach out to pluck a flower - and feel nothing. Opening your eyes, you see your hand is in the middle of the plant you tried to pluck. Carefully, focussing with your eyes, by trial and error, you grasp and break off the flower, not feeling anything between your fingers. Suddenly, you realise you cannot even feel your fingers, you have not been feeling them, and you run a hand over your body, no sensation; you have to look to see you are touching skin.
Touchscreen Use in All Weather
Touchscreen technology has until now, had one strong disadvantage: In inclement weather, wet, freezing cold hands result from touchscreen use, as gloves and other finger protectors have always made fingers too big and bulky to effectively use touchscreen technology, whilst masking tactile feedback with the glove?s spongy surface.
Back To Top
The true power of the virtual is not infeeding data to senses we already have, but in feeding us new senses, replacement senses, totally immersing our synapses in input cycles they feed...
A True Artificial Cochlea
Constructed by engineers at MIT, a single low power radio receiver chip has been painstakingly modelled on the function and deciphering capabilities of the human ear.
Hearing with more than Just Ears
 It might well be that current experiments with binaural sound for VR 3D recreation of sounds based on the relative positions of the ears, and the head shape of the listener, are not quite getting the full picture of sound reception. It seems facial skin also has a part to play.
iPhone Based Hearing Aids
 A novel and very practical use for an iPhone as an auxiliary hearing aid has been developed. The soundAMP program takes control of the iPhone, and essentially uses the in-built microphone to boost ambient noise levels.
My Bionic Quest for Bol?ro
In October 2005, Wired magazine featured the four-page story of Michael Chorost, a man who fought to revolutionise artificial hearing, and who has relied upon a computer surgically installed inside his skull. Called a cochlear implant, this routine replacement has 16 electrodes that snake inside the inner ear, and plenty of room for improvement.
The Silent Majority
A potent look at closed captioning systems in VR and in gameworlds, including the staggering capabilities possible, the incredible customer response when it does occur, and asks why more developers aren't considering sensory substitution in designing their worlds?
One of the fun aspects of using prosthetics to replace or repair sensory issues rather than locomotive, is that the prosthesis don't always have to fit in the same place as the original. Such is the case with Sonitus Medical's mouth-implanted artificial ear.
Back To Top
A Bland World
Walking through the orchard on a warm summer day, the smells of nature in your nostrils, the soft padding of grass underfoot, you reach up a hand, and grab a juicy red apple from a tree, plucking it delicately. You bring it to your nose and smell the fragrance, then you bite into it. It is like chewing rubber. Completely tasteless. No juice, no sweet flesh, you cannot even detect it in your mouth.
A Sterile World
Smell is a sense often overlooked in a virtual world. Developers discount its importance, as a minor sense, when in truth we rely on it as an input channel to flesh out the world around us. Scents carry on the breeze, and they are with us all the time: from the perfumed odours of pollen blowing in the wind, to the pong as you pass a full dustbin, to the aroma of freshly cooked food wafting out of a pub.
Chemical Sensor as Sensitive as a Dog's Nose
 A technological scent sensor that works in almost the exact same manner as a bog's biological nose has been developed by researchers at the University of California. The sensor, the size of an adult male fingerprint, uses microfluidic nanotechnology to mimic the biological mechanism behind canine scent receptors. The device is both highly sensitive to trace amounts of certain vapour molecules, and able to tell a specific substance apart from similar molecules.
Encoding smells in the Brain: A must for recreation?
In early January 2008, a Rice University study published in the Journal of Neuroscience found that socioemotional meanings, including sexual ones, are conveyed in human sweat. This raises the question for immersive spaces, if such dynamic scents need to be synthetically reproduced to provide a firmer social environment in tele-mediation.
The BOND Nose
 The BOND nose is a project working at the opposite end of the spectrum to most attempts to integrate electronics with the sense of smell. Rather than deciper how the brain does it, BOND places olfactory proteins on an integrated chip, and decodes the signals itself. The goal? A hand-held, artificial nose more powerful than a dog's but with the same wide olfactory range.
The Brain Knows What the Nose Smells, but How?
 The sense of smell is a strange one, and its ties with memory even stranger. How does a single sniff of a scent alert even the tiniest animal immediately to which predator is nearby? How does a human know with a single whiff of an ardour, precisely what to expect when they walk into the next room?
The Symphony of Smell, A New Approach to the Sense
 We have never truly succeeded at a VR scent interface. All those we have, bar none, physically release a scent into the room around the user, because we have never been able to grasp how the brain processes the sense of smell. A novel study tracing individual nerve firings with mice in controlled conditions, reveals that the actual organisation of the sense of smell is far more complex than we ever dreamed.
Think of a concept, and you taste it
Lexical-gustatory synaesthesia is an odd condition, very rare. Its symptoms are such that when the sufferer thinks of a concept, their brain accesses its taste memories, and calls forth the taste of an associated food, as strong as it would be if they were tasting it now.
Scents and smells are ubiquitous, they are part of our everyday world. Yet, when you enter VR worlds, you almost always lose your sense of smell. Sight is covered, hearing is covered via speakers, even touch is covered via haptic gloves and other accessories. Smell usually remains abandoned, lacking hardware to cover it. The Scent Dome, manufactured by TrySenx inc, was the first serious attempt to create a smell peripheral.
Scent POP or Scent Point-of-Purchase is a small-scale scent emitter, designed primarilly for retail needs, although it can be employed as a smell emitter for large-scale virtual worlds.
Scent Wave is a single-scent dispenser, each unit holding and dispensing a single scent-stick. Using a dry-air technology that releases fragrance without sprays, aerosols or heated oils, Scent Wave is one of those systems that will work after being bunged in a cupboard for six months. Used primarily in retail environments, this system often finds use in large-scale VR such as military simulations and immersive training.
The original intent of the designers of the scent collar was to create an entirely virtual environment controlled, personal scent experience to augment the visual and auditory sensory immersion. It is worn round the neck, and has a number of scent cartridges attached to it.
Back To Top
Brain Briefings: Taste Detectors
A layperson-designed article from the American Society for Neuroscience, explaining the basics of how the sense of taste functions, from a chemical and neural point of view.
Food & VR: Expectation of a Tasty Food affects the Brain
As it stands, we can replicate smell in VR, but taste, has long been severely under funded, with only one interface available. However, additional evidence for the benefits of virtualising the taste/smell of food has come to light.
Sensory Malfunction: Taste
 A look at what taste is, as a sense, and the possible malfunctions - what can go wrong with the taste reaching the brain, attempting to explain in signal propogation terms. Intended to assist the creation of taste-based sensory systems for VR.
The Tongue Illuminated
Imagery published by Technology Review, that highlights the distribution and emergence of taste receptors on the human tongue.
Think of a concept, and you taste it
Lexical-gustatory synaesthesia is an odd condition, very rare. Its symptoms are such that when the sufferer thinks of a concept, their brain accesses its taste memories, and calls forth the taste of an associated food, as strong as it would be if they were tasting it now.
Back To Top
A Dark World
It is beyond dark, it is pitch black. No motes of light, to glittering beams, no nothing. You blink and cannot tell if you did just blink or not. Away in the distance a humming begins. You hear it, and turn towards it. It is coming from your left, and down a ways. Beyond that you cannot tell. Reaching out, you grasp at empty air. The fear rises as you reach out for something, anything to grasp on, and through some flailing, you find a smooth wall.
Action computer games can sharpen eyesight
Empirical research from the University of Rochester in the US has found that playing in fast passed, lightning reaction virtual environments can be very good for your eyes.
Augmented Vision Cops
Beat officers, pounding the streets in the UK, have unleashed an AR-inspired weapon on the war on crime.
Book Quotes: Sight Without Eyes
 A quote from the book Profiles of the Future, by Arthur C Clarke on the creation of artificial eyes by decoding the optic nerve, is compared to more recent advances doing just that.
Can culture dictate the way we see?
The way the primary visual cortex in the brain develops, may not be hardwired after all, but may develop according to experience, according to some fairly stunning revelations from the University of Illinois in Urbana, US, in early 2007.
CYCLOPS - Robot Blind Person
 CYCLOPS is a novel robot, with a noble purpose. Created by researchers at the California Institute of Technology, it is designed to be used as a surrogate for blind persons in the testing of visual prostheses.
Cystal [eye]ball: Precognitive Eyes are Normal
 It takes our brain nearly one-tenth of a second to translate the light that hits our retina into a visual perception of the world around us. Because of the 10th of a second delay, an Assistant Professor has developed a case arguing that our visual system needs to do more than just report current events - it needs to see the future as well.
High Dynamic Range Photography, and Use in Eye Emulation
H.D.R., or high dynamic range photography is a relatively new technique used by photographers to take high-quality full lighting range photos that are near-indistinguishable from paintings, for their ability to correctly replicate the lighting of a given scene, exactly how the human eye would see it.
Optical Machine Vision Navigation System Found in Flies
 A study conducted to understand how flies and bees can navigate so precisely using just natural sunlight, has interesting implications for machine vision, and adding additional sense-based navigation systems to UAVs and UGVs without adding the weight or cost of any extra hardware.
Probing the Differences Between Organic Vision and Machine Vision
 A group of researchers from UC Santa Barbara have been comparing the human visual system to machine vision systems from an architectual and programming perspective, to see what, exactly makes human vision systems more efficient - and gain some insight on how to replicate that.
Research opens way for bionic eye
Researchers at Harvard Medical School in the US discovered the sites to stimulate in the brain of monkeys to feed visual information in, opening the way for artificial, bionic eyes. This development was a major step towards the creation of true 'virtual light' systems.
The Argus devices, created by Second Sight Inc, are ocular implants designed to restore vision to those whose eyes no-longer function. For people whose sight loss occured as a result of outer retinal degenerations, such as Retinitis Pigmentosa, they offer a chance to see again, via arrays of electrodes connected to the back of the eye and the optic nerve.
The Real Virtual Light
 Virtual Light, a term coined to describe seeing without eyes, is something that has been semi-mythical for decades, whilst research quietly moves forwards. Now, we are starting to see a convergence of the technologies vital to making it a reality.
The Argus 2 implant is the successor to Argus 1. Both were created by the same company, Second Sight. Both were created for the same purpose: To restore sight to the blind. Where Argus 2 offers improvement over its predecessor is that Argus 1 could only offer 16 electrodes in a 4 x 4 array. In the same area on the back of the retina, Argus 2 offers 60 electrodes in a 10 x 6 array.
The first Argus implant was known as the Argus 16, because it consisted of sixteen electrodes attached to the back of the retina in the eye. Created by Second Sight Incorporated as a means of restoring sight to the blind, the Argus device was actually a complex little arrangement of interconnected devices.
Back To Top
Controlling Devices by Muscle Movement
Mimi Switch, a Japanese muscle monitoring device, has an interesting premise. It looks like a normal set of headphones but is fitted with a set of infrared sensors that measure tiny movements inside the ear that result from different facial expressions.
Proprioception
Proprioception is one of the main standard senses of the body, and arguably one of the most important. Unlike sight, sound, touch, taste, smell and balance, proprioception is an unconscious sense. That is to say it is one we are feeling all the time, and rarely if ever creeps into the conscious mind unless we actually sit down and think about it.
Back To Top
Balance in the Blood not just the Ear
Galvanic Vestibular Stimulation. For those of us familiar with it, it's a possible panacea to cure simulation sickness, and rope the sense of balance of the user directly into the simulation. Unfortunately, it seems there is a fly in the mixture.
Cheapening the Cost of Motion Sensors
 Motion sensors are starting to creep into a whole plethora of applications. They are the linch pins of haptics, of 3D pointers, of stress based sensor networks and locomotive VR interfaces. Yet, there's a problem. Small, discrete motion sensors, tiny enough to be built into larger devices the size say, of a Wii-remote or a 6 ounce HMD, are extremely difficult and expensive to produce.
Repairing and Virtualising Balance in one Swoop
Without the vestibular system we lose our sense of balance entirely. Balance is vital in the physical world. In the virtual, it is not yet vital, but would be handy for an increasing number of situations. What if there were ways to use prosthetics to give balance back to those without, and to then utilise that same discovery, to bring balance, into VR?
Back To Top
Podcast: Woody Norris: Inventing the next amazing thing: Directional Sound
 The talk is primarily focussed on demonstrating his new invention 'hypersonic sound'. It is essentially a way to precisely focus sound, or as the inventor puts it "put sound where you want to." This has obvious implications for 3D sound effects in virtual reality and channeled sound cones in augmented reality.
Sound in the Brain: An Orderly Orchestra of Synapses
 Unlike the mapping of the sense of smell, which has odourant maps all over the olfactory bulb in no particular order, the sense of sound is remarkably well ordered. Synapses line up in an overlapping orchestra, with multiple redundancies, a study has found.
Two-way Link Between Sound and Fear Perception
It has long been known that sound can affect our emotions. A good tune or a sound reminding us of a terrifying event, bring associated memories to mind, that bring emotional states with them. What was not realised until recently however, was that the emotional state you are in when you hear the sound, changes the sound you hear in the first place. This makes sound a surprisingly potent tool for immersion feedback.
Why Recoil at Unpleasant Noise Occurs
Nails on a blackboard. It is an unbearable noise, one that sends shivers through anyone who hears it. But why does it do this, and more importantly, how can a virtual environment replicate the effect in any sound?
|
