| dc.description.abstract | Influenced by synesthesia, the creators of such ‘visual musics’ as abstract art,
color organs, abstract film, and most recently visualizers, have attempted to illustrate
correspondences between the senses. This thesis attempts to develop a framework for
music visualization founded on emotional analogues between visual art and music. The
framework implements audio signal spectrum analysis, mood modeling, and color theory
to produce pertinent data for use in visualizations. The research is manifest as a
computer program that creates a simple visualizer. Built in Max/MSP/Jitter, a
programming environment especially for musical and multimedia processing, it analyzes
data and produces images in real-time.
The program employs spectrum analysis to extract musical data such as loudness,
brightness, and note attacks from the audio signals of AIFF song files. These musical
features are used to calculate the Energy and Stress of the song, which determine the
general mood of the music. The mood can fall into one of the four general categories of
Exuberance, Contentment, Depression, and Anxious/Frantic. This method of automatic
mood classification resulted in an eighty-five percent accuracy rate. Applying color expression theory yields a color palette that reflects the musical mood. The color palette
and the musical features are then supplied to four different animation schemes to
produce visuals. The visualizer generates shapes and forms in a three-dimensional
environment and animates them in response to the real-time musical data. The visualizer
allows user input to actively direct the creation of a variety of different visualizations.
This personalization of the synesthetic effects of the visualizer invites the viewer to
actively consider his or her own unique associations and facilitates understanding of the
phenomenon of synesthesia and sensory fusion. | en |
