Dimensionality and the mind...

 

The very idea of an mp3 visualizer relies on a kind of synesthesia- the visual and audio input streams mix and it appears as visuals following the music. Although the visualizer's waveshape can dance along with the music it never by itself looks like the music. The deltafields are not affected by the music playing, all they do is compute the same equation on a lot of pixels, over and over again. Still it is the movement of colors on the screen, caused by the deltafield, that appears to move and follow the music. It is a mental phenomenon and illusion of sorts that can be aided and induced through certain tactics. For instance, each deltafield has a "mood" that can make it spiky, jumpy, mellow, flowing, etc. If that mood is matched up with a similar style of music, synesthesia is enhanced. Similar, if major transitions in the music are matched by a transition the visuals will appear to follow the music. It is the job of the programmer to make the computer good at sensing transitions in the music. When it is done well, the human brain is adept at seeing causation between on two simultaneous, separate events.

Mental mixation occurs at other times too. Think of a series of notes in an arpeggio, with an amount of silence in between each note. If you speed this up and make the gaps in between shrink towards zero, listeners will begin hearing a chord in addition to single notes long before the notes actually overlap. The brain has a very short-term memory storage systems for each sense. It holds the incoming information for fractions of a second to help provide continuity and allow for preliminary processing. As the notes are held in this store they can overlap with the new incoming notes, overlapping and mixing.

This is the same phenomenon that interested the Pointillists and affected their art work. They realized that by painting individual dots of color they could represent all hues and that given the right distance, the dots would fuse in the viewer's eye to create a solid color that seemed to shimmer as the individual dots were still slightly noticeable.

In creating a mp3 visualizer we used the mental mixing of audio at certain times in the song and then combined that with visual stimulation. The result is something that our brain perceives as a combination, even though the pieces exist separately.

 

Home

Play that funky music...

Beyond Banchoff

Watch the movie

How the magic happened...

Dimensional analysis

About us...