Package net.paulhertz.pixelaudio

Class WaveSynth

java.lang.Object
net.paulhertz.pixelaudio.WaveSynth
public class WaveSynth extends Object
Implements a combination of color organ and additive audio synth. Animates pixels using phase shifting of audio generators in waveDataList.

WaveSynth is organized around properties, such as gain (i.e. loudness or brightness) and gamma (a sort of contrast setting), and data objects. The data objects include a a bitmap, mapImage, that is a Processing PImage instance for the image representation of the WaveSynth, a PixelAudioMapper that allows the WaveSynth to mediate between audio data and image data, arrays for the WaveSynth's audio signal and the image data ordered by the PixelAudioMapper signal path, and an array of WaveData objects, waveDataList, that holds the individual sine wave components of the WaveSynth with their frequency, amplitude, phase, and other properties. There are also a series of properties concerned with animation and video output.

When a WaveSynth is used to produce color patterns, each WaveData object in the waveDataList controls a color. The colors of the various WaveData objects are added together, much as the audio sine waves are to produce audio, with the brightness of each color determined by the amplitude of the controlling sine wave. The WaveSynthEditor example code provides a graphical user interface for editing the WaveSynth properties and the individual WaveData objects. Experiment with it to get an idea of what WaveSynth can do to produce patterns and additive audio synth waveforms.

Gamma and histogram scaling are intended primarily for still-image grading and offline rendering. For realtime animation, prefer adjusting operator amplitudes and DC offsets to control brightness/contrast, and leave gamma = 1.0 and isScaleHisto = false for best performance.

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    private float[]
    activeColorB
    Pre-split color channel B for active waves (first activeCount entries valid).
    private float[]
    activeColorG
    Pre-split color channel G for active waves (first activeCount entries valid).
    private float[]
    activeColorR
    Pre-split color channel R for active waves (first activeCount entries valid).
    private int
    activeCount
    Number of active waves in the current frame (length of valid entries in activeWaves).
    private WaveData[]
    activeWaves
    Active (unmuted, non-suspended) waves for current frame.
    private float[]
    activeWeights
    Weights just for active waves (first activeCount entries valid).
    int
    animSteps
    Default number of animation steps
    float[]
    audioSignal
    Array of audio signal values corresponding to colorSignal
    int[]
    colorSignal
    Array of color values for the audio signal
    String
    comments
    comments for JSON file
    int
    dataLength
    The length of the audio and image pixeldata, w * h
    float
    gain
    Gain factor for audio and color signal
    float
    gamma
    Gamma correction factor
    int[]
    gammaTable
    Precomputed gamma table
    private int
    h
    The height of the image
    int
    histoHigh
    The high value for histogram scaling
    int
    histoLow
    The low value for histogram scaling
    (package private) boolean
    isRenderAudio
    Whether to render the audio signal
    boolean
    isScaleHisto
    Whether to scale the histogram
    processing.core.PImage
    mapImage
    The image generated by the WaveSynth
    float
    mapInc
    the increment in phase over the image pixels, typically TWO_PI / image size
    PixelAudioMapper
    mapper
    The PixelAudioMapper instance used to map audio signals to pixel values
    int
    mapSize
    The size of the image
    int[]
    maskScan
    array of color values for mask, especially useful when it is constant
    float
    noisiness
    Noisiness factor for frequency modulation, left for future development
    float[]
    renderSignal
    Array of rendered signal values
    float
    sampleRate
    The sampling frequency, the number of samples read in one second of sound.
    int
    step
    Current animation step
    int
    stop
    Stop animation at this step
    boolean
    useGammaTable
    Whether to use the gamma table for correction, or compute on the fly.
    String
    videoFilename
    Video filename as stored in JSON data, used for output
    int
    videoFrameRate
    Video frame rate as stored in JSON data, often set externally
    private int
    w
    The width of the image
    int[]
    waveColors
    array of colors associated with the WaveData operators
    ArrayList<WaveData>
    waveDataList
    List of WaveData objects
    float[]
    weights
    array of amplitudes associated with the WaveData operators
    float
    woff
    offset for normailzing signal, see renderFrame method
    float
    wscale
    scaling factor for normalizing signal, see renderFrame method

  • Constructor Summary

    Constructors
    Constructor
    Description
    WaveSynth(PixelAudioMapper mapper)
    Convenience constructor for WaveSynth that initializes with a default list of WaveData objects.
    WaveSynth(PixelAudioMapper mapper, ArrayList<WaveData> wdList)
    Preferred constructor for WaveSynth.

  • Method Summary

    Modifier and Type
    Method
    Description
    WaveSynth
    clone()
    Creates a deep copy of this WaveSynth instance.
    int
    getAnimSteps()
     
    String
    getComments()
     
    float
    getGain()
     
    float
    getGamma()
     
    int
    getHeight()
     
    static int[]
    getHistoBounds(int[] source)
    Calculate the histogram bounds (minimum and maximum pixel values) for a given array of pixel values.
    int
    getHistoHigh()
     
    int
    getHistoLow()
     
    PixelAudioMapper
    getMapper()
     
    float
    getNoiseiness()
     
    float
    getSampleRate()
     
    int
    getStep()
     
    int
    getStop()
     
    String
    getVideoFilename()
     
    int
    getVideoFrameRate()
     
    ArrayList<WaveData>
    getWaveDataList()
     
    int
    getWidth()
     
    boolean
    isRenderAudio()
    Checks if audio rendering is enabled.
    boolean
    isScaleHisto()
     
    float
    noiseAt(int x, int y)
    Experimental method, for future use.
    static float[]
    normalize(float[] sig)
    Normalize an array of float values to a limit of 1.0f.
    static float[]
    normalize(float[] sig, float limit)
    Normalize an array of float values to a specified limit.
    void
    prepareAnimation()
    Legacy code: set up mapImage for editing, load colorSignal, set mapInc.
    static ArrayList<WaveData>
    quickWaveDataList()
    Initializes a list of WaveData for use by a WaveSynth.
    private void
    rebuildActiveWaves()
    Rebuilds the list of active (unmuted, non-suspended) waves and pre-splits their colors into RGB channels.
    float[]
    renderAudio(int frame)
    Render the audio signal for a given frame, returning an array of float values representing the audio signal.
    float[]
    renderAudio(int frame, float limit)
    Render the audio signal for a given frame, returning an array of float values representing the normalized audio signal.
    float[]
    renderAudioRaw(int frame)
    Render the raw audio signal for a given frame, returning an array of float values representing the audio signal.
    processing.core.PImage
    renderFrame(int frame)
    Loop to render all the pixels in a frame.
    int
    renderPixel(int frame, int pos, ArrayList<WaveData> wdList)
    Render one pixel, return its RGB value.
    void
    setAnimSteps(int animSteps)
     
    void
    setComments(String comments)
     
    void
    setGain(float gain)
     
    void
    setGamma(float gamma)
     
    void
    setHistoHigh(int histoHigh)
     
    void
    setHistoLow(int histoLow)
     
    void
    setMapper(PixelAudioMapper mapper)
    Sets the PixelAudioMapper for this WaveSynth and initializes associated properties and data structures.
    void
    setNoiseiness(float noiseiness)
     
    void
    setRenderAudio(boolean isRenderAudio)
    Sets the audio rendering flag.
    void
    setSampleRate(float newSampleRate)
     
    void
    setScaleHisto(boolean isScaleHisto)
     
    void
    setStep(int step)
     
    void
    setStop(int stop)
     
    void
    setVideoFilename(String videoFilename)
     
    void
    setVideoFrameRate(int videoFrameRate)
     
    void
    setWaveData(ArrayList<WaveData> wdList)
    Sets the list of WaveData objects for this WaveSynth.
    void
    setWaveDataList(ArrayList<WaveData> waveDataList)
     
    static int[]
    stretch(int[] source, int low, int high)
    Apply a linear stretch to an array of pixel values based on specified low and high bounds.
    String
    toString()
    Returns a string representation of this WaveSynth instance.
    void
    updateWaveColors()
    Updates the wave colors based on the current list of WaveData objects.
    int
    weightedColor(int[] colors, float[] weights)
    Calculate the weighted color for a pixel based on the colors and weights of the active waves.
    private int
    weightedColorActive(float[] weights, int count)
    Optimized weighted color for the current active waves.

    Methods inherited from class java.lang.Object

    equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

  • Field Details

    • mapper

      public PixelAudioMapper mapper
      The PixelAudioMapper instance used to map audio signals to pixel values

    • mapImage

      public processing.core.PImage mapImage
      The image generated by the WaveSynth

    • colorSignal

      public int[] colorSignal
      Array of color values for the audio signal

    • audioSignal

      public float[] audioSignal
      Array of audio signal values corresponding to colorSignal

    • renderSignal

      public float[] renderSignal
      Array of rendered signal values

    • waveDataList

      public ArrayList<WaveData> waveDataList
      List of WaveData objects

    • isRenderAudio

      boolean isRenderAudio
      Whether to render the audio signal

    • w

      private int w
      The width of the image

    • h

      private int h
      The height of the image

    • mapSize

      public int mapSize
      The size of the image

    • dataLength

      public int dataLength
      The length of the audio and image pixeldata, w * h

    • gain

      public float gain
      Gain factor for audio and color signal

    • gamma

      public float gamma
      Gamma correction factor

    • gammaTable

      public int[] gammaTable
      Precomputed gamma table

    • useGammaTable

      public boolean useGammaTable
      Whether to use the gamma table for correction, or compute on the fly.

    • isScaleHisto

      public boolean isScaleHisto
      Whether to scale the histogram

    • histoLow

      public int histoLow
      The low value for histogram scaling

    • histoHigh

      public int histoHigh
      The high value for histogram scaling

    • animSteps

      public int animSteps
      Default number of animation steps

    • step

      public int step
      Current animation step

    • stop

      public int stop
      Stop animation at this step

    • noisiness

      public float noisiness
      Noisiness factor for frequency modulation, left for future development

    • comments

      public String comments
      comments for JSON file

    • sampleRate

      public float sampleRate
      The sampling frequency, the number of samples read in one second of sound. By default, for WaveSynth instances that are intended to be primarily visual, mapSize is the sampling frequency. This makes one period of a 1.0 Hz wave fill the entire signal curve. OTOH, if we want the image to represent an audio signal that is also produced by additive synthesis, we should set samplingFrequency to a standard such as 44100 or 48000.

    • mapInc

      public float mapInc
      the increment in phase over the image pixels, typically TWO_PI / image size

    • weights

      public float[] weights
      array of amplitudes associated with the WaveData operators

    • waveColors

      public int[] waveColors
      array of colors associated with the WaveData operators

    • maskScan

      public int[] maskScan
      array of color values for mask, especially useful when it is constant

    • woff

      public float woff
      offset for normailzing signal, see renderFrame method

    • wscale

      public float wscale
      scaling factor for normalizing signal, see renderFrame method

    • videoFrameRate

      public int videoFrameRate
      Video frame rate as stored in JSON data, often set externally

    • videoFilename

      public String videoFilename
      Video filename as stored in JSON data, used for output

    • activeWaves

      private WaveData[] activeWaves
      Active (unmuted, non-suspended) waves for current frame.

    • activeCount

      private int activeCount
      Number of active waves in the current frame (length of valid entries in activeWaves).

    • activeWeights

      private float[] activeWeights
      Weights just for active waves (first activeCount entries valid).

    • activeColorR

      private float[] activeColorR
      Pre-split color channel R for active waves (first activeCount entries valid).

    • activeColorG

      private float[] activeColorG
      Pre-split color channel G for active waves (first activeCount entries valid).

    • activeColorB

      private float[] activeColorB
      Pre-split color channel B for active waves (first activeCount entries valid).

  • Constructor Details

    • WaveSynth

      public WaveSynth(PixelAudioMapper mapper, ArrayList<WaveData> wdList)
      Preferred constructor for WaveSynth. Initializes PixelAudioMapper mapper and waveDataList and sets up the associated properties and data structures. The mapper argument is essential for the WaveSynth to function, as it defines the mapping between audio signals and pixel values. The waveDataList contains the individual sine wave components that will be used for synthesis. The constructor also initializes the mapImage, colorSignal, and audioSignal arrays based on the dimensions of the mapper.

      Note that the constructor does not prepare the animation or render any frames; it simply sets up the initial state of the WaveSynth.
      Parameters:
      mapper - the PixelAudioMapper instance that defines the mapping between audio signals and pixel values
      wdList - the list of WaveData objects to initialize with

    • WaveSynth

      public WaveSynth(PixelAudioMapper mapper)
      Convenience constructor for WaveSynth that initializes with a default list of WaveData objects.
      Parameters:
      mapper - the PixelAudioMapper instance that defines the mapping between audio signals and pixel values

  • Method Details

    • setMapper

      public void setMapper(PixelAudioMapper mapper)
      Sets the PixelAudioMapper for this WaveSynth and initializes associated properties and data structures.
      Parameters:
      mapper -

    • setWaveData

      public void setWaveData(ArrayList<WaveData> wdList)
      Sets the list of WaveData objects for this WaveSynth.
      Parameters:
      wdList - list of WaveData objects to initialize local waveDataList with

    • updateWaveColors

      public void updateWaveColors()
      Updates the wave colors based on the current list of WaveData objects.

    • rebuildActiveWaves

      private void rebuildActiveWaves()
      Rebuilds the list of active (unmuted, non-suspended) waves and pre-splits their colors into RGB channels.

    • quickWaveDataList

      public static ArrayList<WaveData> quickWaveDataList()
      Initializes a list of WaveData for use by a WaveSynth.
      Returns:
      an ArrayList of WaveData objects

    • getWaveDataList

      public ArrayList<WaveData> getWaveDataList()

    • setWaveDataList

      public void setWaveDataList(ArrayList<WaveData> waveDataList)

    • getGain

      public float getGain()

    • setGain

      public void setGain(float gain)

    • getGamma

      public float getGamma()

    • setGamma

      public void setGamma(float gamma)

    • isScaleHisto

      public boolean isScaleHisto()

    • setScaleHisto

      public void setScaleHisto(boolean isScaleHisto)

    • getHistoLow

      public int getHistoLow()

    • setHistoLow

      public void setHistoLow(int histoLow)

    • getHistoHigh

      public int getHistoHigh()

    • setHistoHigh

      public void setHistoHigh(int histoHigh)

    • getNoiseiness

      public float getNoiseiness()

    • setNoiseiness

      public void setNoiseiness(float noiseiness)

    • getAnimSteps

      public int getAnimSteps()

    • setAnimSteps

      public void setAnimSteps(int animSteps)

    • getStop

      public int getStop()

    • setStop

      public void setStop(int stop)

    • getStep

      public int getStep()

    • setStep

      public void setStep(int step)

    • getComments

      public String getComments()

    • setComments

      public void setComments(String comments)

    • getVideoFrameRate

      public int getVideoFrameRate()

    • setVideoFrameRate

      public void setVideoFrameRate(int videoFrameRate)

    • getVideoFilename

      public String getVideoFilename()

    • setVideoFilename

      public void setVideoFilename(String videoFilename)

    • getMapper

      public PixelAudioMapper getMapper()

    • getWidth

      public int getWidth()

    • getHeight

      public int getHeight()

    • getSampleRate

      public float getSampleRate()

    • setSampleRate

      public void setSampleRate(float newSampleRate)

    • clone

      public WaveSynth clone()
      Creates a deep copy of this WaveSynth instance.
      Overrides:
      clone in class Object
      Returns:
      a new WaveSynth instance with the same properties and wave data

    • toString

      public String toString()
      Returns a string representation of this WaveSynth instance.
      Overrides:
      toString in class Object
      Returns:
      a string containing information about the WaveSynth instance

    • isRenderAudio

      public boolean isRenderAudio()
      Checks if audio rendering is enabled.
      Returns:
      true if audio rendering is enabled, false otherwise

    • setRenderAudio

      public void setRenderAudio(boolean isRenderAudio)
      Sets the audio rendering flag.
      Parameters:
      isRenderAudio - true to enable audio rendering, false to disable

    • prepareAnimation

      public void prepareAnimation()
      Legacy code: set up mapImage for editing, load colorSignal, set mapInc. colorSignal = mapper.pluckPixels(...) is no longer essential, because renderFrame() overwrites every colorSignal[pos] before planting it back into mapImage.pixels.

    • renderFrame

      public processing.core.PImage renderFrame(int frame)
      Loop to render all the pixels in a frame. We want it to complete a frame before any changes to the WaveSynth, so it's synchronized.
      Parameters:
      frame - the number of the frame we are rendering in an animation sequence
      Returns:
      a PImage derived from additive audio synthesis.

    • weightedColorActive

      private int weightedColorActive(float[] weights, int count)
      Optimized weighted color for the current active waves. Uses pre-split RGB and only the first activeCount entries. Gamma and Histogram settings are primarily intended for still frame rendering. You can probably tweak operator amplitudes and perhaps DC offset to obtain the brightness and contrast you want for animation. Leave gamma = 1.0 and isScaleHisto = false for best performance.
      Parameters:
      weights - the array of weights for scaling the active waves
      count - the number of active waves (length of valid entries in weights and activeColor arrays)

    • renderPixel

      public int renderPixel(int frame, int pos, ArrayList<WaveData> wdList)
      Render one pixel, return its RGB value.

      NOTES
      Our basic equation: 
      sample amplitude = sin(initial phase + phase shift + frequency * i * (TWO_PI/n))
      Restated, in two parts: 
      wd.phaseInc = (wd.cycles * TWO_PI)/animSteps; mapInc = TWO_PI / mapSize;
      float val = (float) (Math.sin(wd.phaseTwoPi - frame * wd.phaseInc + wd.freq * freqShift * pos * mapInc) + woff) * wscale + wd.dc;
      Instead of incrementing phase at each step, we subtract (frame * phase increment) from the initial phase: instead of adding, we subtract so that animation data files give the same result in previous implementations. And yes, I have forgotten the original reasons for subtracting.

      We now let the WaveData object calculate the signal: this is much more flexible and barely affects the time. For the latest version (2026-06), the WaveData class has been refactored to have a prepareFrame() method that calculates the current phase for the frame, and a nextValue() method that uses recurrence relations to calculate the next sample value without trig functions. This should be much more efficient, especially for large numbers of waves.:

      Parameters:
      frame - number of the frame to render
      pos - index position of the pixels
      wdList - list of WaveData objects that calculate output value
      Returns:
      the color of the pixel in RGBA format

    • weightedColor

      public int weightedColor(int[] colors, float[] weights)
      Calculate the weighted color for a pixel based on the colors and weights of the active waves. This method applies optional gamma correction and optional histogram scaling to the resulting color, but you should avoid using those for animation, as they are primarily intended for still frame rendering.
      Parameters:
      colors - an array of colors
      weights - an array of weights for scaling the colors
      Returns:
      weighted color value

    • renderAudio

      public float[] renderAudio(int frame)
      Render the audio signal for a given frame, returning an array of float values representing the audio signal. This method builds the active wave list, prepares the oscillators for the current frame, and sums the contributions of the active waves to produce the audio signal. The resulting audio signal is normalized to a specified limit.
      Parameters:
      frame - the frame to render
      Returns:
      audio signal as an array of float, scaled by 0.95f normalization

    • renderAudioRaw

      public float[] renderAudioRaw(int frame)
      Render the raw audio signal for a given frame, returning an array of float values representing the audio signal. This method builds the active wave list, prepares the oscillators for the current frame, and sums the contributions of the active waves to produce the audio signal. It is a little more efficient than renderAudio() because it does not normalize the signal.
      Parameters:
      frame - the frame to render
      Returns:
      the raw audio signal, rendered but not normalized

    • renderAudio

      public float[] renderAudio(int frame, float limit)
      Render the audio signal for a given frame, returning an array of float values representing the normalized audio signal. This method calls renderAudioRaw() to get the raw audio signal, and then normalizes it to a specified limit using the normalize() method. The resulting audio signal is suitable for output to an audio file or audio output device.
      Parameters:
      frame - the frame to render
      limit - the normalization limit
      Returns:
      audio signal, rendered and normalized

    • noiseAt

      public float noiseAt(int x, int y)
      Experimental method, for future use.
      Parameters:
      x -
      y -
      Returns:

    • normalize

      public static float[] normalize(float[] sig, float limit)
      Normalize an array of float values to a specified limit. This method finds the minimum and maximum values in the input array, and then maps all values to a new range defined by -limit to limit.
      Parameters:
      sig - the array of float values representing the audio signal to be normalized
      limit - the maximum absolute value for the normalized signal
      Returns:
      an array of float values representing the normalized audio signal

    • normalize

      public static float[] normalize(float[] sig)
      Normalize an array of float values to a limit of 1.0f.
      Parameters:
      sig - the array of float values representing the audio signal to be normalized
      Returns:
      an array of float values representing the normalized audio signal

    • getHistoBounds

      public static int[] getHistoBounds(int[] source)
      Calculate the histogram bounds (minimum and maximum pixel values) for a given array of pixel values. This method iterates through the input array, extracts the RGB components of each pixel, and updates the minimum and maximum values found. The resulting minimum and maximum values can be used for histogram scaling or other image processing tasks. May not produce the desired contrast enhancement if the histogram is heavily skewed.
      Parameters:
      source - array of pixel values to analyze for min and max
      Returns:
      int[] array with min and max values

    • stretch

      public static int[] stretch(int[] source, int low, int high)
      Apply a linear stretch to an array of pixel values based on specified low and high bounds. This method iterates through the input array, extracts the RGB components of each pixel, and applies a linear mapping to stretch the values from the range defined by low and high to the full range of 0 to 255. The resulting array contains the stretched pixel values, which can be used for image enhancement or other processing tasks. You can run getHistoBounds to determine low and high values for the stretch. Not recommended for animation.
      Parameters:
      source - an array of pixel values to stretch
      low - desired lower bound
      high - desired upper bound
      Returns:
      scaled array of pixel values