InformationWeek 500: Using Sound, Video To Represent Data
A University of California at Santa Barbara researcher uses sound to highlight patterns hidden in complex data sets and present a more tangible representation to users.
Data visualization has traditionally been in charts and graphs or rotational, 3D cubes that display data point distribution. At the InformationWeek 500 Conference at Dana Point, Calif., yesterday, a music composer urged that data be illustrated both in fluid, three-dimensional graphics and sound.
"Data is getting too voluminous … to compare numbers in side by side columns. We are multi-sensory beasts. We coordinate voluminous amounts of data every day through our senses," said JoAnn Kuchera-Morin, a music composer who serves as director of the AlloSphere Research Facility at the University of California at Santa Barbara.
The AlloSphere is a walk in, spherical-shaped lab where visitors mount a bridge to be treated to a three-dimensional visual and audio representation of complex data. Speakers and images on screens embedded in the wall of the sphere surround the visitor and saturate the senses with sound and moving graphics.
Kuchera-Morin in an opening presentation to the 370 attendees showed a three-dimensional representation of an individual's brain, with sound growing in intensity as the fly-through camera approached a blood rich portion of the cortex. A random or chaotic set of numbers can be represented as noise, but numbers in a definite relationship to each other can be represented by pitch. A long piece of string will vibrate twice as fast if it is cut in half, and that relationship can be applied in the form of notes to the data, she noted.
She then showed a visualization of data that represented a mapping of the human brain. In the AlloSphere, the mapping was a kind of fly-through of peaks and valleys, accompanied by sound that grew stronger as one approached a blood dense area.
With three-dimensional images representing folds in the brain and sounds representing blood movement, Kuchera-Morin's presentation took on an air of gliding through a subterranean set of passageways, accompanied by alternately increasing, then fading sounds. The images were displayed on big screens behind the speaker.
Kuchera-Morin showed how the sole electron of a hydrogen atom can be visually represented as a wobbly disk that changes shape as it orbits its nucleus. As one hydrogen atom approaches another, the disk emits a photon as the electrons pass near the other. This data can be represented by sound; likewise, hydrogen atoms combining with zinc results in electrons becoming excited and moving into higher orbital states, a change that can also be represented in sound.