Audio Animation using Particle Systems

Kenny Buckler - CMPS161 Final Project

Introduction

Description:

My final project is an audio file animation utility that uses particle systems to visualize a stream of audio spectrum data.  

The animator configures parameters of the particle system , including properties of both emission points and particles themselves. For ease of use, the animator needs only to set properties for each emission point. Particles emitted from this point will inherit these properties once projectile. These parameters include basic particle properties such as position, size, velocity, acceleration , time to live , color,  and transparency.

Installation and Compilation:

See README.txt

Basic Algorithm:

At each frame, an array of 512 floats containing the amplitude for each frequency band is retrieved from the audio subsystem. Given a audio file encoded at 44khz, the range of frequencies represented is 22khz, due to the Nyquist property. Since the frequency ranges represented in each array element are constant, (~40Hz), the immediate visual representation obscures what our ears perceive as pitch. In other words, two neighboring pitches at indices i =16, and i =17 are MUCH more distinct that values near the higher end of the spectrum, say i = 500 and i = 501. With this in consideration, attempts are made to scale spectrum values to have similar visual representations. The scaled audio information is then filtered to provide threshold values to determine particle emission properties.

Keyboard Controls:

Images:

Sadly static images do no justice....

Small Videos:

You'll need the DiVX video codec to view these : clip1 clip2 clip3

Animation Overview

Animation 1: Spectrum

This is the raw spectrum information. No real animation here, but illustrates the basic data structure and difficulties in obtaining visual representation.

Animation 2: Tree

This is a "tree" that emits particles. Bass frequencies are represented near the very bottom of the "tree", treble near the top. Outward velocities are determined by the strength of the frequency band at that point.

Animation 3: Unnamed

For this animation, the spectral information is divided in 3 sections, bass, midrange, and treble. The left, reddish particles are emitted from the bass frequencies, the middle green are from the midrange and the right side blue particles are emitted from the treble frequencies. The height at which the particle is emitted is based on the overall volume from each output channel. The red particle are emitted at a height proportional  to the left channel volume. The blue particles are emitted proportionally to the right channel volume and the green particles are emitted based on the overall average volume.

Animation 4: Spiral

Emission points are placed on the ground in a spiral pattern. Emission points near the center of the spiral correspond to bass frequencies, points near the spiral radius correspond to treble frequencies. Upward velocities are assigned based on the strength of the frequency band, with a small x-z jitter added.

Performance Considerations:

With a particle system such as this, the processor bottlenecks occur when a large number of particles are emitted nearly simultaneously. This can lead to asymptotic behavior in the effective frame rate of the application. I impose penalties on the particle system as the number of particles reaches threshold values. In the current implementation, the is a hard upper limit of 8192 active particles in the system. The soft limit is set to 1/2 the max particles or 4096 particles. When the particle count approaches the soft limit, the particles age increases quickly. When the soft limit is surpassed, fewer particles are emitted. This keeps the system interactive and also allow for brief periods of extreme activity. An age penalty is incurred for each collision with the ground as well.

Thanks to NeHe.gamedev.net for the Windows API Fullscreen code:
http://nehe.gamedev.net/data/lessons/lesson.asp?lesson=01