Package net.paulhertz.pixelaudio

package net.paulhertz.pixelaudio

Core classes for PixelAudio, a Processing library that maps between sound and image data.

PixelAudio blends sounds and images by mapping one-dimensional signal arrays, typically audio samples, onto two-dimensional image arrays using space-filling curves and related traversal patterns. The same mapping can be used in both directions: sound can be visualized as image data, and image data can be interpreted as sound.

The top-level package contains the mapping framework, built-in path generators, bitmap transform utilities, image/audio transcoding tools, and WaveSynth additive synthesis classes. Gesture, sampler, granular, scheduling, I/O, and example code are documented in their own subpackages.

You can read a scholarly introduction to PixelAudio, complete with a brief review of its historical context in intermedia art, here: PixelAudio: A Toolkit for Intermedia Composition}.

Library interface and mapping framework

• PixelAudio provides the Processing library interface.
• PixelAudioMapper maps between 1D signal arrays and 2D image pixel arrays.
• PixelMapGen is the abstract base class for signal-path generators.

Signal-path generators

• HilbertGen generates Hilbert curve paths.
• MooreGen generates Moore curve paths.
• BoustropheGen generates boustrophedon paths.
• DiagonalZigzagGen generates diagonal zigzag paths.
• MultiGen combines multiple PixelMapGens into a single path.
• BuildFromPathGen builds paths from stored coordinate data.
• RandomContinousGen is an experimental random-path generator that never worked. Deprecated.

Image transforms, animation, and transcoding

• AffineTransformType labels reflection and rotation transforms.
• BitmapTransform provides bitmap rotation and reflection utilities.
• Argosy shifts pixel patterns along an image's signal path.
• Lindenmayer provides simple L-system support for Argosy patterns.
• AudioColorTranscoder provides experimental audio/color transcoding support.

WaveSynth and signal utilities

• WaveSynth combines additive audio synthesis with color-pattern generation.
• WaveData stores WaveSynth operator data.
• WaveSynthBuilder provides utilities for building WaveSynth instruments and data sets.
• WindowedBuffer provides a moving window over larger audio buffers.

Noise utilities

• ScaledSimplex wraps OpenSimplex noise with scaling helpers.
• OpenSimplex2 implements OpenSimplex noise.

Related Packages

Package	Description
net.paulhertz.pixelaudio.curves	Gesture, path, and Bézier curve modeling classes for PixelAudio.
net.paulhertz.pixelaudio.example	Example Processing sketches and demonstrations of PixelAudio library functionality.
net.paulhertz.pixelaudio.granular	Granular synthesis classes for PixelAudio.
net.paulhertz.pixelaudio.io	JSON-based persistence for PixelAudio data structures supported by file i/o classes.
net.paulhertz.pixelaudio.sampler	Digital audio sampling synthesis classes for PixelAudio.
net.paulhertz.pixelaudio.schedule	Sample-accurate event scheduling, gesture-based time management, and audio utility functions.

Class	Description
AffineTransformType	The AffineTransformType enum defines the types of affine transformations that can be applied to bitmaps in the PixelAudio library using only rotation and reflection.
Argosy
AudioColorTranscoder	Experimental helper class for PixelAudioMapper and other classes or methods that transcode audio and color values.
BitmapTransform	Provides static methods for rotating and reflecting 2D integer arrays using index remapping.
BoustropheGen	BoustropheGen generates a signal path that starts at (0,0) and reads left-to-right to (width-1, 0).
BuildFromPathGen	A PixelMapGen that loads data from an external PixelMapGen.pixelMap, such as might be saved in a JSON file.
DiagonalZigzagGen	Generates a PixelMapGen with a zigzag diagonal signal path.
HilbertGen	Generates coordinates and LUTs for a Hilbert curve over a square bitmap starting at (0,0) and ending at (width-1, 0).
Lindenmayer	Implements a simple Lindenmeyer system (L-system), a so-called DOL-system: deterministic and context-free.
MooreGen	Generates a Moore curve over a square bitmap starting at (width/2 - 1, 0) and ending at (width/2, 0).
MultiGen	A PixelMapGen child class that combines multiple PixelMapGen instances with a single signal path.
OpenSimplex2	K.jpg's OpenSimplex 2, faster variant from https://github.com/KdotJPG/OpenSimplex2
PixelAudio	PixelAudio library for Processing.
PixelAudioMapper	PixelAudioMapper is designed to handle one-to-one mappings between 1D "signal" arrays formatted as floating point values in the range [-1.0, 1.0] and 2D "image" arrays formatted as RGBA integer pixel data.
PixelAudioMapper.ChannelNames	List of available color channels, "L" for lightness, since "B" for brightness is taken.
PixelAudioMapper.ChannelNames.ChannelExtractor
PixelMapGen	Abstract class for handling coordinates and LUT generation for PixelAudioMapper.
RandomContinousGen	Deprecated.
ScaledSimplex	A wrapper for OpenSimplex2, adapted from Christian Maher's code at https://cmaher.github.io/posts/working-with-simplex-noise/
WaveData	Provides data storage for WaveSynth operators and performs optimized updating of output.
WaveData.WaveState	enumeration of wave states: muted, solo, etc.
WaveSynth	Implements a combination of color organ and additive audio synth.
WaveSynthBuilder	Utilities for generating WaveSynth "instruments" and working with WaveSynth data.
WindowedBuffer	A windowed buffer class to permit loading large audio sources into memory and stepping through them.