PixelAudio

PixelAudio

A Library by Paul Hertz for the Processing programming environment.
Last update, 11/12/2025.

The PixelAudio maps audio signals onto images to create gesture-driven sound art, visual music and multi-media performances.

PixelAudio maps arrays of audio samples onto arrays of pixel values. You can listen to images, visualize sounds, and interact with both through drawing. PixelAudio captures your gestures and lets you play them back with a sampler-style digital instrument or a granular synthesis instrument. Gestures can be scaled in time and resampled with linear and nonlinear functions.

The example sketches include a tutorial designed to get you up and running with creating your own sampling or granular synthesis audio synth, where you draw on an image to generate non-linear samples from an audio file. You can use the GesturePlayground code out of the box, loading your own audio files instead of the ones in the examples. The ArgosyMixer and WaveSynthEditor each provide a GUI for exploring additive audio synthesis that also acts as a color organ (WaveSynth) and pattern generation engine (Argosy). ArgosyMixer and WaveSynthEditor can also output video animation files. If you are interested in combining PixelAudio output with other applications, the AudioCapture sketch provides information about audio signal routing for MacOS, and TutorialOne_05_Network provides an interface for communicating with other applications using UDP.

Keywords. Animation, Sound, Intermedia

Reference. Have a look at the javadoc reference here. A copy of the reference is included in the .zip as well.

Source. The source code of PixelAudio is available at GitHub, and its repository can be browsed here.

Tested

Platform osx,windows
Processing 4.4
Dependencies example code requires Minim audio library, Video Export + ffmpeg, oscP5, and G4P